Measuring Progress: Improved Outcomes for Children and Families

1. Measuring Progress: Improved Outcomes for Children and Families Kathy Hebbeler The Early Childhood Outcomes Center SRI International Virginia EI Conference Creating Connections: Strengthening Partnerships between Families and Providers Roanoke, VA, March 2007 Early Childhood Outcomes Center

2. Objectives • Review history and rationale related to the selection of child indicators • Describe what we have learned about implementing measurement systems • Describe why child indicators are important for improved services Early Childhood Outcomes Center

3. Goal of the ECO Center Promote the development and implementation of child and family outcome measures for infants, toddlers and preschoolers with disabilities that can be used in local, state, and national accountability systems Early Childhood Outcomes Center

4. Being clear… • For VA, the word “indicator” refers to the 3 OSEP outcomes • to distinguish them from IFSP outcomes • ECO Centers materials use the word “outcomes” to refer to both • I will try to use “indicator” for the OSEP outcomes unless I mean all kinds of outcomes Early Childhood Outcomes Center

5. What are Outcomes? Early Childhood Outcomes Center

6. The Concept of Outcomes • An outcome is the result of some action or set of actions • Often expressed in the form of a statement • Examples: • All passengers will wait 10 minutes or less in the airport security line. • All children will be fully immunized by 2. Early Childhood Outcomes Center

7. The Concept of Outcomes Two parts: • The expected result • The action(s) that produces the result • May not be stated but always present Outcomes are the “effect” in cause and effect Early Childhood Outcomes Center

8. Outcomes Closer to Home • EI programs provide high quality services and supports • EI services are family centered • Families help their child develop and learn • Children take appropriate action to meet their needs Early Childhood Outcomes Center

9. Different Kinds/Levels of Outcomes Program or systems outcomes • EI programs provide high quality services and supports • EI services are family centered • EI services are coordinated What are the actions/activities that produce these results? Early Childhood Outcomes Center

10. Different Kinds/Levels of Outcomes Child or family outcomes • Families know their rights and advocate effectively for their children • Children have positive social relationships What are the actions/activities that produce these results? Early Childhood Outcomes Center

11. Logic Model for Child and Family Outcomes Adequate funding Good outcomes for children and families High quality services and supports for children 0-5 and their families Good State policies and programs Good Local policies and programs Good Federal policies and programs Strong Leadership • Supports • Preservice training • Inservice training

12. Two Critical Outcome/Indicator Questions for Building High Quality Programs • What is the intended result? • What is the action/set of action that will produce the intended result? Early Childhood Outcomes Center

13. How Looking at Child Indicators for EI Became Important Early Childhood Outcomes Center

14. Focus on Accountability • Followed an era of focusing on and measuring actions/activities • Call for interest in the “ultimate result”: child and family outcomes/indicators • Cut across both the public and private sectors • Funders demanded data to determine whether a program was doing what it was supposed to do Early Childhood Outcomes Center

15. Critical Events in EI Accountability • 1992 –Osborne and Graebler, Reinventing Government • 1993 – GPRA (Government Performance and Results Act) passed • Intervening years…. • 2002 – PART finds there are no data on outcomes for Part C • 2003 – OSEP begins to ask states for EI child outcome data (and funds the ECO Center!) • 2005, 2006 – OSEP releases revisions to the reporting requirements Early Childhood Outcomes Center

16. PART Review for Part C and Part B Preschool • Results Not Demonstrated Part C “While the program has met its goal relating to the number of children served, it has not collected information on how well the program is doing to improve the educational and developmental outcomes of infants and toddlers served.” Part B Preschool “The Department has no performance information on preschool children with disabilities served by this program.” • Read more at Expectmore.gov Early Childhood Outcomes Center

17. Intervening Years • Special Education – • National study found poor outcomes for special education adolescents Results • Push to include students with disabilities in statewide assessment systems • Early Childhood • Debate about whether child outcomes should be measured at all • Discussion of all the problems in trying to measure outcomes for young children with disabilities Early Childhood Outcomes Center

18. Measuring Child Indicators for EI • The PART findings put an end to the debate about whether or not to do it • Unfortunately, almost no progress had been made in the intervening years as to HOW to do it Early Childhood Outcomes Center

19. Meanwhile: Re-thinking Assessment in Early Childhood • Major changes in last 15 years in how assessment of young children is viewed • Old position: Do not test little kids • New position: Ongoing assessment is part of a high quality early childhood program Early Childhood Outcomes Center

20. What changed • New and different tools became available • Curriculum-based assessments were developed • General EC: Tools for 3-5 came first; 0-3 tools are coming now • Interesting sidebar: Curriculum-based assessments for programs serving children 0-5 with disabilities have been around for years Early Childhood Outcomes Center

21. What changed • The purpose of assessment was redefined • Not about: sorting, labeling, using to deny access (“the unready”) • Now about: Getting a rich picture of what children can do and can’t do and using that information to help them acquire new skills • “progress monitoring” Early Childhood Outcomes Center

22. What changed • Assessment had always been seen as a process with multiple purposes • Distinctions were made between good and bad uses of assessment with young children • Good uses are now promoted • For more information: NAEYC web site (Position statement on Curriculum, Assessment and Evaluation) Early Childhood Outcomes Center

23. Interesting Irony • Even though the disability community had developed many curriculum-based assessment tools, currently **many** programs do not practice ongoing assessment • Assess for eligibility only • The push for ongoing assessment to monitor how a child is doing and plan for instruction/intervention is coming from the general education community Early Childhood Outcomes Center

24. Where We Are Now • Push for measuring outcomes for accountability (top down – but not just federal) • Push for ongoing assessment of progress (monitoring outcomes) as best practice in EC programs • Done well this could be a powerful blending of forces to improve outcomes for children Early Childhood Outcomes Center

25. Why These 3 Indicators? Early Childhood Outcomes Center

26. OSEP Reporting Requirements: Child Indicators • Positive social emotional skills (including positive social relationships) • Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) • Use of appropriate behaviors to meet their needs Early Childhood Outcomes Center

27. Origin of the Child Indicators • ECO Center stakeholder meetings followed by public comment period (2004) • First, collected themes and ideas • Then, drafted and re-drafted indicator wording • Made recommendation to OSEP (2005) Early Childhood Outcomes Center

28. Themes from the Stakeholders: Child Indicators • Consistent with IDEA and legislative intent • Reflect what EI and ECSE are trying to do • One set for birth to 5 • Reflect what is known about development and learning Early Childhood Outcomes Center

29. Themes from the Stakeholders • Be compatible with best practice (esp. transdisciplinary service models, functional behaviors) • Do not base them on domains • Have potential to influence practice in a positive way • Incorporate universal design • Be readily understood Early Childhood Outcomes Center

30. Make Outcomes/Indicators Functional • Functional refers to things that are meaningful to the child in the context of everyday living • Refers to an integrated series of behaviors or skills that allow the child to achieve the outcomes • They are not • a single behavior, nor are they • the sum of a series of discrete behaviors Early Childhood Outcomes Center

31. Functional • What does a child typically do? • Actual performance across settings and situations • How child uses his/her skills to accomplish tasks • Not the child’s capacity to function under unusual or ideal circumstances Early Childhood Outcomes Center

32. Recommendations • Decision: One set of indicators birth to 5 • Decision: Functional outcomes • Decision: Global, not specific Early Childhood Outcomes Center

33. The need for an introductory statement • Family and child outcomes are linked • There are overarching goals for children and family that cut across the outcomes • Helpful to frame the outcomes with these overarching goals Early Childhood Outcomes Center

34. Concepts in the introduction: Goal for children The ultimate goal is for young children to be active and successful participants now and in the future in a variety of settings– in their homes, in their child care, preschool or school programs, and in the community http://www.fpg.unc.edu/~eco/pdfs/eco_outcomes_4-13-05.pdf Early Childhood Outcomes Center

35. Concepts in the introduction: Goal for children • Active and successful participants • Now and in the future • In variety of settings Early Childhood Outcomes Center

36. Concepts in the introduction: Goal for families • Enable families to provide appropriate care for their child • Have resources they need to participate in community activities Early Childhood Outcomes Center

37. Concepts in the introduction:Effective programs • Support families in their quest for a satisfactory quality of life • Provide needed supports and services in a timely fashion ***Remember: Outcomes are the result of actions*** Early Childhood Outcomes Center

38. Concepts in the preface: Outcomes and accountability • Outcomes determined by variety of factors • Not all families and children will achieve all outcomes • BUT, system should still strive to achieve them Early Childhood Outcomes Center

39. Understand their child’s strengths, abilities, and special needs Know their rights and advocate effectively for their children Help their children develop and learn Have support systems Access desired services, programs, activities in their community ECO Family Outcomes Early Childhood Outcomes Center

40. The OSEP Reporting Categories Early Childhood Outcomes Center

41. OSEP Reporting Categories Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged peers but did not reach it d. Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers Early Childhood Outcomes Center

42. Functioning 13 Early Childhood Outcomes Center

43. Entry 14 Early Childhood Outcomes Center

44. Entry Exit 15 Early Childhood Outcomes Center

45. Entry Exit 16 Early Childhood Outcomes Center

46. CISF Numbers to OSEP Reporting Categories • Each possible combination of numbers (and answer to the “b” question at entry and exit generates a line, i.e., a developmental trajectory) • Each trajectory corresponds to an OSEP category • Taking measurements more frequently than entry and exit generates more powerful information Early Childhood Outcomes Center

47. Functioning 13 Early Childhood Outcomes Center

48. OSEP Reporting Categories: Child Outcomes Percentage of children who: • Did not improve functioning • Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers • Improved functioning to a level nearer to same-aged peers but did not reach it • Improved functioning to reach a level comparable to same-aged peers • Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers Early Childhood Outcomes Center

49. Comparing to Same Aged Peers • Deficit model? Not individualized? • Goal of EI: Active and successful participation now and in the future • Kindergarten, school readiness, having friends, community participation • Setting high expectations • Moving from 1 to 2 = moving from 4 to 5 • How much progress is enough? Early Childhood Outcomes Center

50. COSF/CISF Implementation Issues Early Childhood Outcomes Center