Judith Scott-Clayton Teachers College, Columbia University Higher Education in Michigan:

1. On Money and Motivation:A Quasi-Experimental Analysisof Financial Incentives for College Achievement Judith Scott-Clayton Teachers College, Columbia University Higher Education in Michigan: Looking Back and Looking Ahead University of Michigan, Ann Arbor December 10, 2009

2. Background • We now lose as many students between college entry and completion, as we lose between HS graduation and college entry • Financial incentives for achievement are increasingly popular policy intervention • 13+ states have programs similar to WV PROMISE scholarship • Programs provide both money and motivation • We know that aid improves entry, less known on persistence • Learning about mechanisms is critical for optimal policy design • But we have only a limited understanding of if/how these programs work • Administrative data like WVHEPC database offers the opportunity to address these knowledge gaps with rigorous research

3. Research Overview • Do large financial incentives for student achievement improve college students’ outcomes, and if so, what are the mechanisms? • Alleviation of financial constraints? • Incentives for increased student effort? • I examine these questions in the context of a $40 million dollar state-funded merit scholarship program in West Virginia (the PROMISE scholarship)* • Program covers tuition/fees at WV college for eligible students, as long as they maintain a minimum GPA & courseload • I exploit discontinuities in the eligibility formula and timing of implementation to identify causal effects, use detailed data to tease out mechanisms • Summary of results: • Significant and robust positive effects on range of outcomes • Incentives are key to results; scholarship with no strings attached unlikely to produce same pattern *My research follows the first two cohorts of recipients, entering in the fall of 2002 and 2003. The cost of the program has risen, and eligibility rules have changed since then.

4. West Virginia’s PROMISE Scholarship • WV began Providing Real Opportunities to Maximize In-state Student Excellence (PROMISE) in 2002 [context] • To qualify initially: must be a WV high school graduate, with 3.0+ high school GPA overall and within “core” courses, and score a 21+ on the ACT (or 1000+ SAT)* • No income limitations • Must enroll full-time in a WV public or private 2/4 year college • Covers tuition and fees at any WV public univ* • To renew: must complete 30+ credits per year and maintain 3.0 GPA (2.75 in 1st year) • 30-credit requirement corresponds to “on-time” progression • Of 13 similar state programs, only WV & SC have this req.

5. The WVHEPC Data • Comprehensive, administrative data from WV state higher ed commission • Covers all 1st time freshmen entering WV public 2/4 yr colleges between 2000-2003 (~13,000 students per entry cohort). • Includes hsgpa, ACT/SAT scores, basic demographics (not income); enrollment, transcript, financial aid and graduation records for five years following entry • Linked with admin employment records from WV state employment office (but still rather early for post-college emp.) • How did this data partnership come to pass? • Persistence on both sides, plus luck • Support from the top, time/expertise from data coordinator • WV contacts provided more than just data: also provided critical contextual information, and on-the-ground perspective

6. Research Methods • Simple comparison of recipients and non-recipients does not provide causal estimates of program’s impact • Instead, two strategies based on discontinuities • Initial eligibility rules set up a regression discontinuity (RD) analysis for 2002, 2003 entry cohorts [figure] • Availability of data for pre-program entry cohorts sets up a cohort analysis (before-after) [figure] • Both strategies made possible by rich admin data • Concern: differential selection into the “treated” group • An explicit goal of the program was to attract “best & brightest” • But additional data sources suggest that switchers from out of state are too small in number to drive overall impacts

7. Results • RD graphs: key thresholds and graduation • Cohort graphs: key thresholds and graduation • RD table: main results and falsification check (RD before PROMISE) • Cohort table: main results • Mechanisms: [credit PDFs][credit CDFs][GPA CDFs][table]

8. Concluding Remarks • Financial incentives can have significant positive effects on end-of-college outcomes incl. graduation & TTD • Scholarship of same value with no strings (or different strings) attached is not likely to produce same effects • Courseload requirements appear particularly important • Does not imply that money is unimportant • Knowing something about mechanisms makes results useful beyond just WV context • Social cost-benefit analysis is positive, but upfront cost to state is high – future research needed to examine whether smaller scholarships can produce similar effects

9. The End

10. West Virginia Context [BACK] • WV public system accounts for about 85% of in-state first-time entrants • Total of about 20 public, 20 private 2/4-yr colleges • WVU/Marshall account for 43% of public entrants (66% of public PROMISE recips) • WVU is only ranked school (“Tier 3”) • Tuition is lower than national average • WVU is most expensive public (currently $5,100/yr, compared to national avg. of $6,200 for 4-yr public) • In 2002 average public tuition in WV was ~$3,000 • PROMISE represented major increase in financial aid • Prior to PROMISE, average young WV entrant received $1,350 in Pell+HEGP (mostly Pell), with 46% receiving one or both • After PROMISE average entrant received $1,200 in PROMISE, with 38% receiving a grant • PROMISE receipt generally does not affect Pell/HEGP eligibility [back]

16. NOTES: All regressions include indicator controls for gender, race/ethnicity, age, and a quadratic function of high school GPA and indictors for each ACT score.

20. Table 9