The Excellence Achievement Gap: State-Level Data

1. The Excellence Achievement Gap:State-Level Data Jonathan A. Plucker, Ph.D. Terry Spradlin, M.P.A. Amber Esping Indiana University American Psychological Association August 13, 2006

2. Current Policy Landscape • Tremendous focus on minimum competency. • Strongly codified in NCLB, although many pre-NCLB state accountability systems focus on minimum competency.

3. Minimum Competency • AYP = Adequate Yearly Progress • AYP determined by % students passing state-defined content standards • Complex calculation … • Most states have minimum competency (MC) standards • Although some claim to have “high standards,” they’re generally MC standards

4. MC Systems Are Not a Bad Thing • Low passing rates within MC systems are deplorable • Especially the large achievement gaps • Eliminating these gaps and getting every student to MC is an ethically justifiable goal.

5. But is it Enough? • What does 100% MC by 2014 get us? • NAEP description of “Basic” achievement level • Partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade. • Minimum proficiency will have many societal and humanitarian benefits, but it will probably not be the economic engine for which many hope. • A better goal is to get as many students as possible into the highest levels of achievement.

6. What About Excellence? • There is growing concern that the tight focus on MC neglects the needs and development of academic excellence. • This was predictable at the time NCLB passed. • Counterargument: Rising tides carry all ships. • But do they?

7. The Excellence Gap • We are interested in whether MC achievement gaps are mirrored in achievement disparities at high levels of academic performance. • Research Question: Do states with small and shrinking MC gaps also have small and shrinking excellence gaps?

8. History of AddressingExcellence Gaps • Javits Act has focused on eliminating excellence gaps, but evaluations of funded programs provide very limited evidence of effectiveness. • Javits Act remains very, very small and faces annual calls for its elimination. • Some state efforts, almost always short-lived and generally not well-evaluated. • Many local efforts, but small scale and also rarely evaluated and disseminated.

9. Research Question:Do states with small and shrinking MC gaps also have small and shrinking excellence gaps?

10. Method I • Identified states who are credited with small/shrinking and large/growing MC gaps • Based on Education Trust and NAEP reports • Examined student performance on state criterion-referenced tests • Disaggregated by race • Examined pass rates for top scoring category • E.g., Pass+ or Advanced • Examined student performance on NAEP reading tests • Disaggregated by race • Examined pass rates for Advanced Proficiency category

11. Method II • Looked at trends over three year periods • Focused on reading tests

12. Sample for State Tests • States with shrinking MC gaps • Biggest decreases in Black-White elementary reading gaps: • Colorado, Delaware, Florida (all 5%) • Biggest decreases in Hispanic-White gaps: • Delaware (18%), Virginia (7%), Florida (5%) • Source: Ed. Trust, Measured Progress

13. State Data on Black-White Excellence Gaps: Colorado Grade 4 Reading

14. State Data on Black-White Excellence Gaps: Delaware Grade 5 Reading

15. State Data on Black-White Excellence Gaps:Florida Grade 4 Reading

16. State Data on Latino-White Excellence Gaps: Delaware Grade 5 Reading

17. State Data on Latino-White Excellence Gaps: Virginia Grade 5 Reading

18. State Data on Latino-White Excellence Gaps: Florida Grade 4 Reading

19. Summary: State Data • Little evidence that state’s lauded for shrinking MC gaps have also been able to shrink excellence gaps • … unless the rate of White students achieving advanced scores decreases.

20. Problems with State Data • Wide range in … • implementation of NCLB • quality of standards and assessment systems • definition of advanced performance • Which make state comparisons problematic • Potential solution: Use state NAEP data

21. Method • Selected states with the 10 highest and lowest MC achievement gaps • Examined % of students achieving Advanced level • Disaggregated by race, specifically Black and White students • Focused on Grade 4 Reading test

22. Basic Partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade. Proficient … Advanced Superior performance. Basic Grade 4 Reading [Students] demonstrate an understanding of the overall meaning of what they read. … They should be able to make relatively obvious connections between the text and their own experiences and extend the ideas in the text by making simple inferences. Advanced Grade 4 Reading [Students can] generalize about topics in the reading selection and demonstrate an awareness of how authors compose and use literary devices. When reading text appropriate to fourth grade, they should be able to judge text critically and, in general, to give thorough answers that indicate careful thought. NAEP AchievementLevel Descriptions

23. 2003 NAEP Grade 4 Reading

24. 2003 NAEP Grade 4 ReadingBlack-White Excellence Gaps Note. DC data not included due to size of gap.

25. 2005 NAEP Grade 4 Reading

26. 2003 NAEP Grade 4 ReadingBlack-White Excellence Gaps Note. DC data not included due to size of gap.

27. 2003 and 2005 NAEP Grade 4 ReadingExcellence Gap Ratios Note. DC data not included due to size of ratio. WV in 2003 and OH in 2005 had 0% Black students score at the Advanced level, hence the missing data.

28. Summary: NAEP Data • States with smaller MC gaps generally have smaller excellence gaps • But the gaps are still very, very large • Difference between states studied appears to be the Advanced rate for White students • i.e., Advanced rate for Black students is uniformly and consistently low, ranging from 0 to 4 compared to 5 to 29 for White students in 2005

29. Conclusions • This study provides evidence that … • States with smaller MC achievement gaps generally have smaller excellence gaps. • Excellence gaps in all of the states studied are quite large and improving gradually, if at all. • Much of the improvement may be due to declines in performance by White students rather than increases by students of other racial backgrounds.

30. Future Research • Expand to … • other content areas and assessments • additional years, additional racial and demographic groups, such as SES, gender, ESL • other outcomes • Graduation rates, attendance, college attendance and performance, student engagement • Refine tools for estimating excellence gaps • e.g., use 3-year moving averages for ratios

31. References and Data Sources • Education Trust, Measured Progress Report • http://www2edtrust.org/edtrust/images/MeasuredProgress.doc.pdf • Colorado data • http://www.cde.state.co.us/cdeassess/index_assess.html • Delaware data • http://dstp.doe.state.de.us/DSTPMart/default.asp#Disagg • Florida data • http://www.fcatresults.com/demog/GetReport.aspx • Virginia data • http://pen2.vak12ed.edu/cgi-bin/broker?_service=doe_prod&_program =prodcode.doerp101rcdp001.sas#gr5 • Florida data • http://www.fcatresults.com/demog/GetReport.aspx

32. More References and Data Sources • Generic NAEP Achievement Level Descriptions • http://nces.ed.gov/nationsreportcard/reading/achieve.asp • Test- and Grade-Specific NAEP Achievement Level Descriptions • http://nces.ed.gov/nationsreportcard/reading/achieveall.asp • 2003 and 2005 NAEP Data • http://nces.ed.gov/nationsreportcard/pubs/stt2003/20044564.asp and http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006452

33. CEEP Contact Information: Jonathan A. Plucker, Ph.D. Director 509 East Third Street Bloomington, Indiana 47401-3654 812-855-4438 Fax: 812-856-5890 http://ceep.indiana.edu