Career- and College-Ready Cut Scores For MEAP and MME

1. Presentation to the Michigan State Board of Education September 13, 2011 Career- and College-Ready Cut Scores For MEAP and MME

2. ACT Research & Development Unit National Center for Educational Achievement Michigan Technical Advisory Committee MDE/BAA Measurement Research & Psychometrics Unit Contributors

3. Identify a score for Proficient on the MME that represents being on track for career and college • Identify a score for Proficient on the highest grade level of MEAP that represents being on track to high school success • Identify a score for Proficient on the lower grade levels of MEAP that represent being on track to success in the next higher grade • Identify a score in each grade level of MEAP or MME that represents attainment of Partial Proficiency • Identify a score in each grade level of MEAP or MME that represents attainment of Advanced skills Issues to Address

4. Assumed that if a student is on track to success in college, then the student is also on track to success in technical career training • Therefore, focus on success in college • Included 2-year college programs (including job training programs at 2-year institutions) • Success could mean many things: • A, B, or C in college? • In a 2-year or 4-year college? • Conducted analyses of all of these scenarios • Conducted analyses only of academic success, not of every factor that leads students to be successful in college Defining On Track for Career and College

5. 2-year versus 4-year colleges • Separate analyses were run regarding college success in 2-year and 4-year institutions • The cut scores identified for 2 year versus 4 year institutions were within measurement error of each other • Therefore, all final analyses combined all students from 2-year and 4-year colleges into a single group Defining College Success

6. Separate analyses were run using students achieving an A versus B versus C in their first credit-bearing freshman courses • A and C analyses did not produce usable results • Therefore, all final analyses used the criterion of B or better as the measure of college success • This is also the criterion for success used by ACT in its college readiness benchmarking study Defining College Success

7. Three types of analyses conducted • Logistic Regression (LR) • Signal Detection Theory (SDT) • Equipercentile Cohort Matching (ECM) Types of Analyses Conducted

8. Started with data from Michigan Public Institutions of Higher Education • Identified appropriate credit-bearing freshman courses against which to analyze the relationship between MME scores and course grades • Thanks to 2- and 4-year institutions for providing the data • Thanks to President’s Council for providing listings of courses appropriate to tie to MME scores • Using those data, conducted SDT analyses to connect MME to college freshman grades Identifying Proficient (On Track)Cut Scores on MME

9. Used SDT to map backward from 11th grade MME to 7th and 8th grade MEAP • Identified the score on MEAP that would maximize the consistent classifications from MEAP to MME • Used SDT to map from 7th grade MEAP to all other MEAP grades • Identified the score on each MEAP grade that would maximize consistent classifications from grade to grade Identifying Proficient (On Track) Cut Scores on MEAP

10. Recommend retaining the labels for the four performance categories • Not Proficient (Considered “Off Track”) • Partially Proficient (Considered “Off Track”) • Proficient (Considered “On Track”) • Advanced (Considered “On Track”) Achievement Levels

11. SDT can also identify scores on the MME scale that give certain probabilities of obtaining a B or better in the first credit bearing freshman course • Identified two probabilities that have strong meaning and give cut scores sufficiently far from the MME Proficient (On Track)cut scores. • 1/3 probability of B or better (Partially Proficient cut score) • 2/3 probability of B or better (Advanced cut score) • These cut scores were identified using SDT for the MME • These cut scores were also mapped back to the MEAP using ECM Identifying the Other Cut Scores

12. Recommended Cut Scores for MEAP and MME Mathematics

13. Impact of Recommended Cut Scores for MEAP and MME Mathematics (From Spring 2011 MME and Fall 2010 MEAP Data)

14. Mathematics Passing Rates (with Current and Recommended Cut Scores)

15. Approximate Percent Correct Scores needed to pass the Fall 2010 MEAP and Spring 2011 MME Mathematics tests with Current and Recommended Cut Scores

16. Recommended Cut Scores for MEAP and MME Reading

17. Impact of Recommended Cut Scores for MEAP and MME Reading (From Spring 2011 MME and Fall 2010 MEAP Data)

18. Reading Passing Rates (with Current and Recommended Cut Scores)

19. Approximate Percent Correct Scores needed to pass the Fall 2010 MEAP and Spring 2011 MME Reading tests with Current and Recommended Cut Scores

20. Recommended Cut Scores for MEAP and MME SCIENCE

21. Impact of Recommended Cut Scores for MEAP and MME Science (From Spring 2011 MME and Fall 2010 MEAP Data)

22. Science Passing Rates (with Current and Recommended Cut Scores)

23. Approximate Percent Correct Scores needed to pass the Fall 2010 MEAP and Spring 2011 MME Science tests with Current and Recommended Cut Scores

24. Recommended Cut Scores for MEAP and MME Social Studies

25. Impact of Recommended Cut Scores for MEAP and MME Social Studies (From Spring 2011 MME and Fall 2010 MEAP Data)

26. Social Studies Passing Rates (with Current and Recommended Cut Scores)

27. Approximate Percent Correct Scores needed to pass the Fall 2010 MEAP and Spring 2011 MME Social Studies tests with Current and Recommended Cut Scores

28. MDE staff will apply the cut scores retroactively as if the new cut scores had been in place • Applied to four years of data • Within one month of approval • No retroactive accountability implications for schools, just to allow for following trends over time Reporting Implications

29. Joseph Martineau • Executive Director • Bureau of Assessment & Accountability • Michigan Department of Education • martineauj@michigan.gov Contact Information