Critical Markers of High Quality Child Outcomes Data

1. Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012 DRAFT

2. Topics • Why identify critical markers of high quality data? • Anticipated use • 3 areas of focus • Draft critical markers • Where these might be discussed in the APR • Reactions? Early Childhood Outcomes Center

3. Looking for… Proposed markers that suggest a state’s EC outcomes systems may produce high quality data Use markers to: - systematically track over time - track internally (within states) - to produce a national picture Early Childhood Outcomes Center

4. Anticipated Use • Tracking state progress (based on APR info) • Compare data from each state to a series of critical markers for summaries • X % of states met standard on each marker nationally • X% of states met standard on 7 out of 10 markers • NOT a state by state report card • Share detailed info with each state, upon request Early Childhood Outcomes Center

5. 3 Areas • Completeness of data Early Childhood Outcomes Center

6. Completeness of data • Missing data concepts • Are whole forms missing? (Have data from all kids expect to have data from) • Are the forms that you have complete? Early Childhood Outcomes Center

7. 3 Areas • Completeness of data • Accuracy of data Early Childhood Outcomes Center

8. Accuracy of Data • Do the data reflect the “true” performance of children in the program • Unknowable, so…. • Are the patterns in data what you would expect to see if the data were accurate? • Do they make sense? Or, are there red flags that raise more questions? Early Childhood Outcomes Center

9. 3 Areas • Completeness of data • Accuracy of data • State efforts related to tracking quality of data Early Childhood Outcomes Center

10. Criteria for Selection of Markers • Important • necessary for or indicative of high quality data • Accessible to ECO • Information to determine presence or absence of marker is available Early Childhood Outcomes Center

11. Draft Markers: Completeness • State calculates and publicly reports number of children missing outcomes data • Percent of missing outcomes data is less than 5%. • Is this reasonable? • Percent of missing data by proxy calculation is less than. • 40% of exiters for Part C • 20% of child count for 619 Early Childhood Outcomes Center

12. Part C percent of exiters2009-10 Early Childhood Outcomes Center

13. Draft Markers: Accuracy/Patterning • State % in a is not overly high (GT 5%) • State % in b is not overly low (LT 5%) or high (GT 50%) • State % in c is not overly low (LT 5%) or high (GT 50%) • State % in d is not overly low (LT 5%) or high (GT 50%) • State % in e is not overly low (LT 5%) or high (GT 65%) Early Childhood OutcGTomes Center

14. Knowledge and SkillsPart B 619 proportion of children that make no progress (progress category a) Early Childhood Outcomes Center

15. Knowledge and SkillsPart B 619 proportion of children who improved functioning but not sufficiently to move nearer to functioning comparable to same age peers(progress category b) Early Childhood Outcomes Center

16. Knowledge and SkillsPart B 619 proportion of children who improved functioning to a level nearer to same age peers but did not reach it(progress category c) Early Childhood Outcomes Center

17. Knowledge and SkillsPart B 619 proportion of children who improved functioning to reach a level comparable to same aged peers(progress category d) Early Childhood Outcomes Center

18. Knowledge and SkillsPart B 619 proportion of children who maintained functioning at a level comparable to same aged peers(progress category e) Early Childhood Outcomes Center

19. Percent of states meeting the Critical Markers for 2009 - 10 Early Childhood Outcomes Center

20. Draft Markers: State Quality Review of Data • State conducts data quality checks • State’s own analyses provide evidence of high quality data Early Childhood Outcomes Center

21. Location in Suggested APR Template • http://www.fpg.unc.edu/~eco/pages/fed_req.cfm#ECOSuggestedFormats Completeness/ Missing Data Accuracy/ Patterns Early Childhood Outcomes Center

22. Location in Suggested APR Template • http://www.fpg.unc.edu/~eco/pages/fed_req.cfm#ECOSuggestedFormats State Quality Review of Data Early Childhood Outcomes Center

23. Questions for you • Are these markers important? • Related to high quality data • Things you value and might track or are already tracking them? • Would tracking them help you improve your system? • Is there anything else you can think of as a good marker of high quality data Early Childhood Outcomes Center

24. Summary – Draft Critical Markers • State calculates and publicly reports number of children missing outcomes data • Percent of missing outcomes data is less than 5% • Percent of missing data by proxy calculation is less than X • State % in a is not overly high (GT 5%) • State % in b is not overly low (LT 5%) or high (GT 50%) • State % in c is not overly low (LT 5%) or high (GT 50%) • State % in d is not overly low (LT 5%) or high (GT 50%) • State % in e is not overly low (LT 5%) or high (GT 65%) • State conducts data quality checks • State’s own analyses provide evidence of high quality data

25. Comments, Reactions, Questions?