Session Outline

1. Family Outcomes:Overview of Statewide and Local Early Intervention Family Survey Data and Best Practices Sharing For Local Improvement PlanningMay 2009

2. Session Outline • Overview – Measuring Family Outcomes • Statewide and Local Data Collection/Results • Using Family Outcome Measures for Local Improvement Planning • Best Practices Sharing • Additional Resources • Brief Overview – Preschool Parent Survey • Feedback on DRAFT Family Outcome documents

3. Definition of Family Outcomes Part C Indicator #4 Percent of families participating in Part C who report that early intervention services have helped the family: a) know their rights, b) effectively communicate their children’s needs, and c) help their children develop and learn.

4. The NCSEAM Family Survey for Early Intervention • The family survey for Part C includes 2 scales: • Impact on Family: Impact of Early Intervention Services on the Family – 22 items • Family-Centered Services: Quality of Early Intervention Services – 25 items

5. Collecting the DataSurvey Distribution • Surveys sent out October 15, 2008 through December 1st, 2008 to all 7,078 Part C families • Surveys sent out in Spanish to families who indicated that Spanish is their primary language (from the Part C database) • 8 jurisdictions chose to have the surveys directly mailed to families by the vendor (3,170) • 16 jurisdictions chose to deliver the surveys by hand (3,908)

6. Collecting the DataResponse Rates • Response Rate for mailed surveys – 18.4% • (ranges from 11% - 36%) • Response Rate for hand delivered surveys – 25.2% • (ranges from 12% - 64%) • Overall Response Rate of 22.2% • Overall Effective Response Rate of 22.1% • With a 1.9% Margin of Error at a 95% Confidence Level

7. Impact on Family Scale

8. Impact on Family Scale Item Calibrations • An item’s calibration indexes the amount of positive impact needed to achieve agreement with the item. • When item calibrations were examined separately for different subgroups, (e.g., families in different states or of different ethnicities), the item calibrations were highly similar.

9. Relative Position of Standards for Indicator 4a, 4b and 4c Indicator 4b: Effectively communicate their children’s needs. FS Item: “Communicate more effectively with the people who work with my child and family.” [556] 556 Indicator 4a: Know their rights. FS Item: “Know about my child's and family's rights concerning Early Intervention services.” [539] 539 Indicator 4c: Help their children develop and learn. FS items: “Understand my child's special needs.” [516] “Do things with and for my child that are good for my child's development.” [498] 516

10. Reporting the Data 2007-2008 Results • A 95% likelihood of a response of “agree,” “strongly agree,” or “very strongly agree” with these items on the NCSEAM’s Survey’s Impact of Early Intervention Services on Your Family scale: Percent of families participating in Part C who report that Early Intervention services have helped the family: • 4a: Know their rights 78% • 4b: Effectively communicate their children’s needs 75% • 4c: Help their children develop and learn 86%

11. Maryland Percents on Indicators 4a, 4b, and 4c

12. Distribution of Impact on Family Measures: All Participating MD Families

13. Distribution of Quality of EI Services Measures: All Participating MD Families

14. Comparing Demographics and the Impact on Family and Quality of EI Services Measures

15. Sex of the Child

16. Relationship to Child

17. Child’s Age at Time of Survey Completion

18. Child’s Age When First Referred

19. Length of Time in the Program

20. Survey Language

21. Ethnicity

22. Extent Results are Representative

23. What does it all mean?? • Early intervention is helping a large majority of Maryland families to achieve positive family outcomes. • Maryland’s performance on Indicators 4a, 4b, and 4c was between 75% and 86%. There has been a 3 – 4% increase from baseline on all three indicators. • A significant minority of families reported that they did not benefit from early intervention. • The extent to which the survey responses are representative of the demographics varies across the state.

24. Using Family Outcome Measures for LocalProgram Improvement

25. Using Statewide Data for Program Improvement

26. Using Statewide Data for Program Improvement • The Impact on Family Measure Raw Score for Maryland is 661 (with a 95% confidence that the real number statewide is between 669 and 653). • Using the lower score of 653, item calibrations and the measure interpretation guide, Maryland should focus on activities to increase the number of families who agree with the following statements: • Over the past year, Early Intervention services have helped me and/or my family to know where to go for help and support to meet my family’s needs. • Over the past year, Early Intervention services have helped me and/or my family to know about services in the community.

27. Using Local Data for Program Improvement • Based on the mean measure for your jurisdiction on the Family Impact Scale and the Quality of Early Intervention Services Scale, specific activities have been identified for your local jurisdiction to focus on for this year (Description of Family Outcomes Current Issues/Activities) • Continue local strategies that are working for program improvement around family outcomes. • Incorporate best practices strategies that make sense for your local program improvement efforts.

28. Best Practices Sharing • Queen Anne’s County • Frederick County • Charles County • Prince George’s County • Baltimore County

29. Queen Anne’s County • Primary focus - communication with families • Talk about parent rights at every opportunity • Provide a welcome packet which includes a discussion of the family survey • Expect families to attend at least one Family Training during the first six months in the program • Provided training around the “Family Visit Note” • Provided training using the Indicators of Impact on Families and Quality Services as a self-reflection tool to assess strengths of staff • Focus on Functional Outcomes • Reminder call about the survey to every family (FSN Coordinator)

30. Frederick County • Relationship-Based Approach • Set a goal – 50% response rate • “Every Family Counts” banner and barometer • Hand delivering surveys an effective approach • Information in the newsletter and a letter from the Program Director introducing the survey • Professional Development Activities – particularly the shift to writing integrated functional outcomes • Surprise celebration for staff

31. Charles County • Consistent approach concerning family outcomes with all partners and most importantly families • Discuss family outcomes and the family survey from the very beginning with the focus on partnership • Emphasize the parent teaching/training component • Randomly evaluate aspects of the I & T program by phone surveying parents • Educate LICC and all community partners about the survey and the use of consistent terminology • Provided training to all partners - “If you have a problem do you know what to do or who to talk to?”

32. Prince George’s County • Provided professional development to all staff around family outcomes as part of a system change from child-centered EI service delivery to a family support model. • Statewide and local FSS Coordinators assisted with the development and delivery of the family outcomes/family survey training (copy in packet).

33. Baltimore County • Presentation about family outcomes/family survey to LICC. • BCITP team leaders developed written guidelines and a fact sheet about the survey and this was distributed to all I & T staff and contractual providers. • Provided training to improve family and child outcomes, local response rates, and data, and gave out copies of the state family survey . • BCITP parent coordinators made individual family contacts. • BCITP staff including parent coordinators reviewed the BCITP Family Guide and provided strategies for effectively sharing this information with families as part of their ongoing services.

34. Family Outcomes – Improvement Planning as part of the CLIG Every jurisdiction must complete an improvement plan for OSEP Performance Indicator #4. The Improvement Plan must include: Steps to improve the response rate and representativeness of responses to the statewide family survey, such as identifying ways that family support staff and service coordinators can assist families to complete the survey.

35. Family Outcomes – Improvement Planning as part of Monitoring OSEP Performance Indicators #4a, b, c will require an improvement plan if the LITP does not achieve the State target. The Improvement Plan, if required, must include: Steps to involve local stakeholders to understand the purpose of the survey and the importance of family benefit based on each outcome. Steps to improve the response rate and representativeness of responses to the statewide family survey, such as identifying ways that family support staff and service coordinators can assist families to complete the survey; and Steps to identify targeted improvement activities based on local survey results and other local sources of information on family benefit in order to improve the ability of families to help families know their rights (4a), and/or communicate their children’s needs (4b), and/or help their children develop and learn (4c).

36. Preschool Parent Survey • Report • FAQ for the Parent Survey • Copy of the Preschool Parent Survey • Update on current Parent Survey distribution

37. Additional Resources • www.nectac.org • www.fpg.unc.edu • www.taalliance.org/publications/curricula.asp • Child and Family Outcomes Brochure • Family Survey Frequently Asked Questions