Enhancing Undergraduate Retention: Strategies and Results

1. 2006 Assessment SymposiumWELCOME!May 22, 2006

2. Traditional Undergraduate Retention Cathy Alexander, Maria Kohnke, Damien Peña

3. Assessment Goal & Design • Review the status of first-time traditional undergraduate freshmen by entering cohorts • Identify best practices on campus • Implement programs aimed at improving retention and graduation rates • Data reviewed using Fall 99 through Fall 05 census snapshots • Data separated by categories identified by team • Reported data as attrition versus retention for simplification

4. Data Analysis Note: Some categories Include duplicated counts

5. Findings: Attrition After Start of Senior Year6 year Data (some data includes duplicated counts) Total Students FY 03-04 372 82 261 24 includes 17 underrepresented of which 3 are conditional admits 6 total conditional admits • CLU Overall = 38% • 35% cumulative fourth year • 3% after start of senior year • Underrepresented Students = 56% • 38% cumulative fourth year • 18% after start of senior year • Caucasian = 36% • 34% cumulative fourth year • 2% after start of senior year • SSS Students = 26% • 26% cumulative third year (0% fourth year) • 0% after start of senior year

6. Changes Implemented & Results • Created Academic Assistance Program (AAP) modeled after student success plans in Student Support Services • Funded program by assessing an advising fee • 6% improvement in retention for all freshmen and 10.4% increase for conditional admits

7. Goals for 06-07 & Beyond • Longitudinal study using quantitative and qualitative data (retention data & exit surveys) • Expand services provided beyond entering conditional admits and probation students • Monitor third year retention and graduation rates • Gather and analyze data around “stuck seniors” and determine methods to increase degree completion

8. Exercise Science and Sports MedicineProgram ReviewMichele LeBlanc

9. Exercise Science and Sports Medicine Department • Four full-time faculty • Jim Hand, M.S., A.T.C. • Kavin Tsang, Ph.D., A.T.C. • Michele LeBlanc, Ph.D. • Janie Rider, Ph.D. • Approximately 17 adjunct instructors each semester • Approximately 140 majors • About 60 different classes scheduled each semester • Offers a Bachelor of Science degree with 4 emphasis options • Athletic Training • Human Performance • Pre-Physical Therapy • Physical Education (Teaching) • Offers activity program which ALL CLU students are required to participate in for at least one semester

10. 2005 – 2006 Program Review Goal • Timeline/Context • Started Program Review during 2004-2005 AY • Repeated change in departmental personnel/leadership • Department changed from Social Science Division to Natural Science Division • Program Review Considerations • Facilities and Faculty • Curriculum • Engagement/Scholarship • Etc.

11. Qualitative Research Design • Purpose: Discover with faculty most pressing issues for PR • Process: • Conduct multiple focus group interviews • Interview faculty one-on-one • Plan to use Flashlight survey for student input also • What data was collected? • What are the major goals of our department? • What are the department’s 3 greatest strengths/ weaknesses? • What do you see as the greatest priorities in the department this year? Next 5 years? • What are your personal goals for this year? Next 5 years?

12. Analysis of Data: Emerging themes • Need for departmental unity/community • Improve communications within department • Factions of students within dept. have established community, but there is no cohesiveness to collective department • Raising the bar – upgrading overall quality of department • Involve students in activities outside the classroom (e.g. research, conferences) • Have faculty more involved across campus • Concern for over-responsibilitied faculty and high turnover, desire to do more, but how?

13. Lessons Learned/Benefits • Common ground identified • Curriculum  Community • January workshop tied these together • Enhanced curriculum created • Cannot implement without additional faculty • Moving forward… • External reviewer – coming to terms with reality • Position request – hopes for saner reality  • Future external reviews/accreditation – need to take baby steps

14. Media Habits Survey Cody Hartley Marketing for Graduate & Adult Programs

15. July 2005 Assessment Goal • Survey Purpose: To determine the media preferences of current ADEP and MBA students so that chosen media targets would be accurate • Understand current student media consumption • Investigate if promotional tactics were connecting with students in our target population • Create media tracking measurement system • Assess the deployment of resources

16. Assessment Design • Paper survey • administered in class • administered by faculty • Two week period • Survey results were entered and tabulated electronically

17. Data Collection • Logistical information • TV usage and preferences • Newspaper readership • Radio listenership • Internet preferences • Other relevant information

18. Analysis of Data • Logistical and Demographic Information • Media Information Sources • Television • Newspaper • Radio • Internet preferences • Other relevant information • Referrals • Incentives

19. Outcomes and Future Goals • Outcomes: • Confirmed low priority for television • Shifted advertising from print to internet • Refined radio advertising based on geography • Developed a “Free Books” referral program • Created media tracking measurement system • Future Goals: • Increase by 20% the total amount of traffic driven to the microsite via advertising • Radio - College Guides • Print - Internet • Harness the power of referrals

20. Sharon Docter, Chair and Professor Beverly Kelley, Professor Dru Pagliassotti, Associate Professor Russell Stockard, Assistant Professor Jing Jiang, Instructor Beth Le Poire Molineux, Associate Professor (half-time) Communications Program Review The mission of the Communication Department is to graduate majors who are highly skilled, critical, and ethical media producers, practitioners, and analysts who are able to do socially meaningful, theoretically rich work.

21. 2005 – 06 Program Review Goals • Examined: • Trends (Enrollment and Faculty-Student Ratios) • Teaching Effectiveness • Scholarly Productivity • Challenge and Engagement • Diversity Initiatives • Critique of Curriculum • Co-Curricular and Internship Opportunities • Departmental Student Learning Outcomes

22. Design:Challenge and Engagement • Examined aggregate scores on student course evaluation questions pertaining to challenge and engagement • Compared departmental score to score for all traditional undergraduates • Compared departmental grade distributions to mean grade distributions for Arts and Sciences, Creative Arts Division, and Social and Behavioral Sciences Division

23. Data Challenge and Engagement Course Evaluation Questions: Grade Distributions:

24. Analysis of Data • Slightly below traditional undergraduate average on measures of challenge • Above traditional undergraduate average on measures of engagement • Departmental grade distributions comparable to Arts and Science averages • Departmental grade distribution slightly above Social and Behavioral Sciences average and below Creative Arts average

25. Recommendations • Sustain high level of student engagement • Institute research methods course requirement • (Need additional faculty FTE) • Implement Departmental Honors Program • Require students to read more primary research and reflect on readings through response papers • Research papers at the lower division level • Provide faculty with data concerning grade distributions and work with faculty to curb grade inflation

26. A Problem-Based Learning (PBL) Approach to Teaching Information Literacy (IL) Concepts to Freshman Seminar Students Henri Mondschein ISS/Pearson Library

27. Purpose To determine information literacy concept levels of freshmen seminar students and to discover student perceptions of information literacy concepts.

28. Assessment Design and Data Collection • Quantitative Design: • Pre-test/Post-test of Information Literacy Knowledge (UCLA Library Instructional Services Advisory Committee Questionnaire) • Work samples rubric • Qualitative Design: • Direct observation of research skills 5-10 seminar participants • Interviews of 5-10 seminar participants • Data Collection • Pre-test/Post-test of Information Literacy Knowledge • Student work samples

29. Analysis of Data • Student Work: • Pre/post-tests • Samples using rubrics • Artifacts (papers, journals) • Interview Transcripts: • Transcribing, coding and analyzing interviews for patterns and themes using qualitative analysis software (Qualrus) • Identify major themes re: PBL & IL • Evidence of: • enhanced information literacy competencies • engagement, intentional learning, and self-directed learning

30. Preliminary Findings • Freshmen enter CLU with diverse information literacy competencies • Freshmen identify engaging classes as consisting of active learning, debates, discussion and group learning activities

31. Future Goals • Implement Assessment Design • Create lesson plans and assignments based on findings • Share results with faculty • Design a freshman seminar that increases information competencies and promotes student engagement through problem-based learning model

32. Computer ScienceUndergraduate Program Review Chang-Shyh Peng

33. Computer Science Department • Degrees offered (Traditional & ADEP) • B.S. in CS, CIS (48 credits) • Certificate in IT, IS (32 credits) • Minor in CS, CIS (20 credits) • Faculty • Full time: Chang-Shyh Peng, Myungsook Klassen, Fenglien Lee, Craig Reinhart • Additional Adjuncts • Milestones • Data collection & analysis: Fall 2004 ~ Fall 2005 • Completion of review report: 3/16/2006 • External reviewer’s visit: 4/25/2006 • External reviewer’s report: 5/4/2006

34. Process Included in Program Review process Reviewer’s Contribution Outsider’s viewpoint Evaluation of current status Feasibility of future plan Goals Curriculum Current structure Areas for improvement Direction for advancement Faculty Expertise Load Future growth Student body Enrollment Diversity Supporting resources Existing What it takes External Reviewer

35. Data Collection • Internal • Class evaluations • Departmental assessment plan • Data from Registrar and Provost’s Office • External • Curriculum at comparable institution • APU, Chapman, Cal Poly, CSUN, Humbolt, PLU, Pomona, Redland, Westmont, … • ACM/IEEE Curriculum Task Force Report • ACM Job Migration Task Force Report • US Dept of Labor Reports • Industry outlooks

36. Analysis and Outcomes • Commendations • Dedicated faculty • Advanced Curriculum • Innovative 5-yr BS/MS Program • Well-designed labs • Recommendations • Support and retain qualified faculty • “Fit" of the department • Recruitment of female and international students • Course redesign (discrete) and realignment (subjects with language emphasis) • Exploration of “Project Center” • Sufficient and modern lab facility • Supporting personnel

37. Lessons Learned • Know your external reviewer • Credentials • Academic experience • Program Review experience • Current knowledge of the field • Start early • Availability of report • Prepare review report and external reviewer questions long before reviewer’s arrival • Utilize valuable internal resources

38. Pearson Library’s Role with Challenge and Engagement Julius Bianchi

39. Introduction • Regents requested a strategic plan for Pearson Library (Spring 03) • Draft Plan presented Fall 04 2005-2006 Goal • What goal was set? • To process all Inter-Library Loans (ILL’s) in a timely manner • To provide easy access to and streamlined on-line forms • What aspects of student learning / service will be assessed? • Circulation transactions and ILL’s

40. Identified peer institutions to establish base line data and benchmarks Used Education Department’s National Center for Educational Statistics Bi-Annual Academic Library Census Pearson Library collects: Circulation statistics Inter-Library Loans Materials budget Pearson Library reports this data to Education Department including IPEDS Assessment Design Data Collection

43. 2006-2007 Actions to Promote ILL • Free information specialists from reference desk responsibilities • Promote liaison program - information specialist / faculty collaboration • Advertise and market ILL • Include information on ILL in all bibliographic instruction sessions

44. Course Schedule SubmissionContinual Improvement Process Melanie Clarey

45. Fall 2005 Assessment Goal • CIP identified two areas for review: • Review the processes within the two month timeline • Ensure the integrity of the data • Control of input • Control of sources • Review communication process • Sources • Budget implications

46. Assessment Design • Measures used to collect data: • Design flow charts to track processes and times • Integrity of data count: • The number of corrections to adjunct contracts necessary. • The number of times the data was being entered into a spreadsheet or Datatel. • The number of people “handling” the data throughout the current process.

47. Data Collection • Each course required input on information into 7 separate screens of Datatel. • Each semester information was being entered into a spreadsheet or Datatel no fewer than 5 times. • Each semester 4 people have to enter the same information into different systems are different times. • Each semester the Provost office received “endless” phone calls from adjunct professors regarding changes to contracts.

48. Analysis of Data • The number of times information was entered increased the chances for error and discrepancies. • Program Directors and Deans were being held accountable for their adjunct budget yet were not given the tools or control over the system necessary to monitor these things appropriately. • Discrepancies were found in data housed in Datatel. • The entire review process took the Committee 2 months to complete.

49. Recommendations & Implications • Recommendations: • Create one screen in Datatel for information input. • Input information directly into Datatel by the Schools and College. • Retrieve data from Datatel. Provost and Registrar Offices each obtain identical information. • Implications: • Timeline for the entire process has been cut in half. • Datatel will be able to give an accurate account of historical data. • Fewer corrections will be needed for adjunct contracts. • More accurate predictions of budget implications of adjunct contracts.

50. Summer Orientationto Academic Resources The SOAR Program:A summer bridge component of the Student Support Services Program Tuula Mattson, MS SSS Director Damien A. Peña, MSW Dir. of TRIO Programs Funded by grants from the U.S. Department of Education & the James Irvine Foundation