Preview: State Child Outcomes Data Quality profiles

1. Preview: State Child Outcomes Data Quality profiles National Webinar February 2014

2. Objectives • Introduce the new state child outcomes data quality profiles • Explain how to interpret the charts and graphs in the profiles • Review the data quality criteria used to analyze the national child outcomes data • Provide an opportunity for you (states) to have specific questions regarding the profiles answered by TA providers

3. Overview • The profiles include information about: • State vs. national • Data quality criteria used for national analysis • Completeness of data • Progress categories patterns • Trends over time

4. National vs. State: State A

5. National vs. State: State B

6. National vs. State: State C

7. Questions? Feel free to ask your questions via the chat box Or by unmuting your line by pressing *6

8. Quality Child Outcomes Data Criteria for quality state-wide data: • Reporting data on enough children • Within expected patterns

9. Completeness of Data • Reporting data on enough children • Part B 619: 12% or more of child count • Number of children reported / Child count • Part C: 28% or more of exiting children • Number of children reported / Total exiters • Note: Some states have established more accurate methods for estimating the number of children receiving services

10. State A Part B 619: Percent of Child Count Included in Child Outcomes Data Plotted +-1 standard deviation (SD) above and below the national average.

11. State B Part B 619: Percent of Child Count Included in Child Outcomes Data

12. State C Part C: Percent of Exiters Included in Child Outcomes Data

13. Questions? Feel free to ask your questions via the chat box Or by unmuting your line by pressing *6

14. Patterns in the Data • Within expected patterns in the data • Criteria for inclusion in national analysis • Category ‘a’ not greater than 10% • (However, less than 5% recommended as more stringent data quality criteria for states) • Category ‘e’ not greater than 65%

15. Progress Category Patterns: State A

16. Progress Category Patterns: State B

17. Progress Category Patterns: State C

18. What Types of Change are Important?

19. Trends over Time SS1: State A

20. Trends over Time SS1: State B

21. Trends over Time SS1: State C

22. Trends over Time SS2: State A

23. Trends over Time SS2: State B

24. Trends over Time SS2: State C

25. Summary • State Child Outcomes Data Quality Profiles present data on: • State compared to national • State data in relation to quality criteria used for national analysis • Reporting data on enough children • Within expected patterns in the data • Trends over time • Profiled 3 states • 1 consistently high quality child outcomes data • 2 states with some different potential data quality issues • Patterns in progress categories • Patterns over time

26. Questions? Feel free to ask your questions via the chat box Or by unmuting your line by pressing *6

27. How to Use the Child Outcomes Profiles • Use to identify trends in the data and data quality issues that require further exploration • Use graphs in other materials to share with various audiences • Can be used as part of broad data analysis

28. Data Quality • Not the focus of the SSIP • But must be addressed in the SSIP • Describe data quality issues identified through data analysis • Describe data quality efforts

29. Looking for Patterns in Data • What capacity do states have to do more pattern checking than nationally available? • Some possibilities: • By entry scores • By disability subgroup • By family characteristics • By geography • By age at entry • Other child or service characteristics

30. SSIP Child Outcomes Broad Data Analysis Template NEW! Tool to assist in conducting an initial broad data analysis using data you currently use for reporting in the APR to look at: • State relative to national • Data Across years • Patterns within the state • Comparisons across programs within the state http://ectacenter.org/~docs/eco/SSIP_child_outcomes_broad_data_analysis_template_FINAL.docx

31. Meaningful Differences Calculator • Are differences meaningful? • This tool can be used to… • Compare current year summary statement values to the previous year for the state • Identify local programs that are meaningfully different from the state • http://ectacenter.org/eco/pages/summary.asp

32. Other resources • Data quality resources • http://www.ectacenter.org/eco/pages/quality_assurance.asp • Child Outcomes Measurement Framework • http://www.ectacenter.org/~docs/eco/selfassessment.doc • Guidance for looking for patterns in data • http://www.fpg.unc.edu/~eco/assets/pdfs/Pattern_Checking_Table.pdf • Data analysis for program improvement resources • http://www.ectacenter.org/eco/pages/usingdata.asp

33. SSIP Phase I Webinar Series for Part C and 619 • Hosted by the Regional Resource Centers Program (RRCP) Part C/619 State Accountability Priority Area, in collaboration with DaSy, ECTA and IDC. The webinar topics are as follows: March 6, 2014,3:00pm ET: SSIP Overview • April 8,2014, 3:00pm ET: SSIP Data Analysis • May 1, 2014, 3:00pm ET: SSIP Infrastructure Analysis

34. Save the Date! • Early Childhood National Conference • September 8-10, 2014: Improving Data, Improving Outcomes • Hosted by DaSy, ECTA, and IDC in New Orleans, LA. • DaSywill be sponsoring up to four people from each state to attend.

35. How ECTA Can Help! • Contact us for support for data quality analysis and quality assurance activities • Contact us for support for program improvement planning and data analysis Abby Winer: abby.winer@sri.com

36. Questions?