In Search of a Grail:

1. In Search of a Grail: Identifying Best Practices for Attracting and Retaining Students Don Hossler (hossler@indiana.edu) Mary Ziskin (mziskin@indiana.edu) Jacob P.K. Gross (paugross@indiana.edu) Indiana University – Project on Academic Success Lauck Parke (Lauck.Parke@uvm.edu) Vice President for Undergraduate Studies University of Vermont

2. Overview • What student retention efforts seem to work (or not)? • But why? • Where does this leave us? • What are we learning from pilot studies on student retention?

3. A Sampling of Retention Initiatives • Supplemental instruction • Major/Career counseling programs • Learning communities • First-year experience seminars

4. Participants more than twice as likely to persist year-to-year as nonparticipants Undergraduates were trained in cooperative learning strategies & stuck to this approach They had a training manual In other words, the institution devoted time & resources to training – it continues to do so Supplemental Instruction

5. Program for undeclared students at a church-affiliated residential institution Participants 6 times more likely to persist to degree Strong institutional focus – campus has a director for this program Could success be linked to faith-based approach? Program is now required & we are continuing study of the effects Program for undecided students at a commuter campus run by career center Participants were over 8 times more likely to persist year-to-year as non participants Participation was strongly encouraged by campus Program connects students to staff advisors, faculty mentors, and potential employers Major/Career Counseling

6. Learning Communities on a Commuter Campus • Had positive impact at a nonselective institution on first-to-second-semester persistence • No effects on year-to-year persistence • We know little of how the program was implemented and sustained over time • Has this been a success?

7. First-Year Seminars at a Community College • Program implemented at three separate campuses within one region • IPAS undertook program evaluation studies using both qualitative and quantitative methods • No evidence of program impact from these studies • Little training of or coordination among faculty instructors • No additional institutional resources devoted to program • Why would we expect this to work?

8. Some Caveats • Student self-selection into programs likely contributes to positive findings • The time commitments of participating campuses & their lack of experience with doing evaluation research resulted in their not collecting key data elements needed for rigorous assessment

9. Where Does This Leave Us? • We remain interested in better understanding how campuses can intervene to positively influence persistence. • Because the commitments of campuses to the way they deliver programmatic initiatives is so variable, we are also interested in how campuses organize themselves to address issues of student persistence. We think this might be an important factor.

10. And…We Are Trying To Do This! • To understand how campuses can adopt policies that enhance persistence • We next report on two pilot study efforts we have underway, both funded by the College Board. • One is a student survey that helps campuses identify the policy levers they can use to improve persistence. • The other survey looks at how institutions organize themselves to enhance persistence & how this might differentially affect persistence rates.

11. College Board Pilot Study on Student Retention • A survey of first-time, full-time, first-year students at 8 four-year institutions • Students surveyed at the end of their first year (spring 2006) • Web-based instrument, or • In-class paper-and-pencil administration • Response rates varied widely from under 10% to over 35% • Follow-up data collected from institutions to show enrollment in fall 2006 • Allows us to look at persistence

12. A few examples of survey questions • How certain are you that you have received useful academic advising at this college? • How often have you participated in classroom discussions… that included contributions from students with diverse backgrounds? • How certain are you that you've taken advantage of all federal and state financial aid programs you are eligible for? • How often have you been unable to register for a course that you needed…? • How many times per semester have you received prompt feedback…from Instructors? • How often have you felt out of the loop with regard to campus policies and procedures?

13. Participating Campuses • Campuses included • 3 commuter campuses • 2 small private liberal arts colleges • 2 residential public universities • 1 public HBCU • Institutions in four states

14. Institution-Specific Analyses • Descriptive information • Participation in student programs • Classroom experiences • Time diary items • Satisfaction • Inferential analyses • Explore factors associated with intent to persist • Merge data with SAT Questionnaire program and fall 2006 enrollment data to explore covariates of persistence

15. Preliminary Results from Residential Campuses • A high proportion of students (87-92%) intend to persist • Factors capturing aspects of academic engagement emerge on one campus • Campus 2: High Academic Engagement (α=.629 ) • Campus 2: Use of Public Space for Learning (α=.607) • Logistic regressions showed that a traditional persistence model enhanced the prediction of which students did not intend to persist • Variables that contribute significantly to intent to persist • Campus 1: development of friendship networks, class attendance, and positive perceptions about placement practices • Campus 2: high combined SAT score • Variables that detract significantly from respondents’ intent to persist • Campus 2: distance of residence from campus, time spent preparing for class

16. Results from Commuter Campuses • A high proportion (84-93.5%) intend to persist • Logistic regressions showed that a scaled-down traditional persistence model enhanced the prediction of which students did not intend to persist • Campus 1: 71.4% correctly predicted • Campus 2: 70.0% correctly predicted • However… • Overall variance explained by the models was relatively low (14.5-18.3%), though comparable to other research on persistence • Academic engagement variables included in the models did not show a significant effect • Variables that contribute significantly to intent to persist • Campus 1: development of friendship networks • Campus 2: certainty of being able to pay for college

17. Factor Analysis

18. Cross-Case Findings • Differences across campuses are evident • Robust factors emerge • Early analyses show how policies, practices and environment play into intent to persist, institution-by-institution

19. Lessons Learned: Pilot Year-1 • Small residential campuses have the highest response rates, commuter campuses the lowest. • Paper-and-pencil administration on commuter campuses • Increased cost • Much higher response rate • More complicated to administer, but may also be a good indicator of…how serious campuses are about improving student persistence • Timing: We hope earlier administrations will improve response rates for residential campuses that use a Web-based survey. • Studying and improving student persistence is difficult. It takes institutional commitment.

20. Next Steps • We will merge fall enrollment records to see who actually returned and then re-run our current set of analyses. • Then we will also merge financial aid data, and SAT questionnaire data to see how these data affect our results. • We will readminister the survey in the winter of 2007.

21. Institutional Survey Sneak Preview • Survey of four-year institutions in California, Georgia, Indiana, New York, and Texas • Web-based administration, summer 2006 • 32.8% response rate • Preliminary findings • 57.1 % of responding institutions have a retention coordinator • 97.3% of institutions analyze retention data annually • Annual analyses, broken out by race/ethnicity, 87.8% • Annual analyses, broken out by major, 70.8%

22. Institutional Survey Preliminary Findings (Continued) • 43.5% report having semester-long orientation programs • 60.5% reported average class size for first-year students at 30 or lower • 82.9% require students to meet with advisors each term • 44.2% of retention coordinators rated the availability of academic support at their institutions as higher than at similar institutions

23. Final Thoughts • Both surveys are works in progress, but show potential • We will refine both instruments and re-test them early second semester • We hope to shed light on how institutions organize themselves and what they can do to enhance student persistence

24. Contact Us Indiana University Project on Academic Success http://pas.indiana.edu

25. Appended Slides

26. Data & Case Descriptives • A Public Residential University • Public, residential, research-extensive university • Approximately 9,000 undergraduates • 92% White; next highest group: APA (2.5%) • 49% expressed confidence that their families would be able to pay for college • 87% reported an intent to persist • Data for this institution • Response rate 22% • Men underrepresented among respondents (46% of population; 27% of respondents)

27. A Public Residential University

28. Findings in 4 cases • Campus A (Nagelkerke=.245) • (+) Friendships/Social network (p<.01) • (+) Positive perceptions of English placement (p<.01) • (-) Missing class (p<.05) • Campus B (Nagelkerke=.342) • (+) SAT (p<.05) • (-) Distance (p<.05) • Campus C (Nagelkerke=.320) • (+) Certainty of major (p<.05) • (-) Discussions with peers (p<.05) • Campus D (Nagelkerke=.209) • (+) Friendships/Social network (p<.05) • (+) Feedback from instructors (p<.05)