CMP Statewide Evaluation Measurement Plan Summary of HB1451 Outcomes Workgroup Efforts

1. CMP Statewide Evaluation Measurement PlanSummary of HB1451 Outcomes Workgroup Efforts Presentation to IOG Members

2. Goals of presentation • Provide background (rationale, legislative goals, objectives) for measurement plan • Review the measurement plan and address questions • Discuss next steps

3. Measurement issues • CMPs vary widely in: • Outcomes • 144 indicators in 31 outcome areas in FY09-10 • Quality of indicators • Measurement processes • E.g., level of reporting (county vs. program vs. ISST) • Difficult to aggregate data across CMPs to demonstrate statewide impact on any of the legislative goals

4. Why we are doing this work • Make stronger statements about the Program’s efforts and successes • Improve the collection of outcome data • Improve service delivery and programs (ability to connect CMP practices with outcomes) • Provide evidence that collaborative management approaches lead to improved child and family outcomes • Promote greater likelihood of ongoing support at the state level

5. Outcomes Workgroup activities • Discussed ways to improve measurement processes • Identified CMP core components for measurement • Developed a measurement approach to implement across CMPs • Selected process and outcome indicators in child welfare and juvenile justice areas • Surfaced issues, challenges, solutions

6. Improving measurement: Alignment of interdependent relationships Population Program Model Outcomes

7. Population • The intended core target population of the CMP is children and families who are currently involved, or at risk for involvement, in multiple service agencies

8. Program Models • IOGs • Specified participating service agencies • Includes family advocacy organization • Focus is on establishing ISST • Developing shared processes • ISSTs • Development of a single plan that integrates services across multiple agencies • Family involvement in service planning

9. Outcomes • Key process outcomes • e.g., reduced duplication, cost shifting • Improvements on child and family outcomes • SSC outlined four domains • There are overarching goals • Ensure safety, stability, and well-being for children • Reduce further penetration into systems

10. Minimally, we need to be able to: • Accurately count number of CMP youth and families served by ISSTs • Show that CMP efforts are leading to positive outcomes across the state in at least a few key areas • Measurement plan was developed to meet these minimum requirements • Recognize that this does not capture all of CMPs’ efforts

11. Measurement Plan • Measures will focus on children and families referred to and served by ISSTs • A minimum set of defined process indicators will be uniformly measured across all CMPs • A small set of CW and JJ outcome indicators will be measured on relevant cases • Education and health/mental health outcome indicators will be defined next year

12. Overview of data collection • Process indicators will be collected through individual client-level database • ETO (web-based system) • Or modified local database (provide dataset) • Child welfare and juvenile justice outcomes will be collected from state data systems, where possible • Collected by OMNI at end of fiscal year • Annual reporting will continue in ETO

13. Process Indicators • Number of families referred to and served by ISST and date of ISST meeting • Type of outcome domain each family is enrolled under (CW, JJ, etc.) • Number and type of agencies or systems • Participating in the family’s service planning • Providing services to families • Number of families participating in service planning • Whether or not an integrated treatment plan was developed • Date the case was closed and/or end of service delivery • Outcomes (from selection of common outcomes) that are aligned with case

14. Process indicators: Data collection • Coordinators collect process data from whoever runs ISST meeting • Enter data into ETO sometime after the first ISST occurs • Coordinators will establish processes to: • Locate and record existing client IDs in Trails and Eclipse

15. Outcome Indicators - CW • Number (rate) of CMP youth with new open involvements in Trails after CMP services began • Number (rate) of CMP youth with no substantiated abuse finding after CMP services began • Number of moves that CMP youth experience when in out-of-home placement • Number (rate) of CMP youth discharged to a permanent home

16. Outcome Indicators - JJ • Number (rate) of CMP youth who successfully complete probation and/or parole • Number (rate) of revocations by technical violations where case resulted in unsuccessful termination among CMP youth • Number (rate) of CMP youth who recidivate, resulting in unsuccessful completion of probation and/or parole

17. Measurement of other legislative outcomes • Annual report and brief mid-year report in ETO (no more quarterly reports) • Strategies to reduce duplication and fragmentation, involve families, etc • Costs and cost savings • Local performance measures • Yearly collaboration surveys to IOGs

18. Next steps – short-term May-June • OMNI will build and train CMPs on ETO database July • CMPs will enter FY10-11 annual report data into ETO • OMNI will work with CMPs who choose to modify local databases • CMPs will begin tracking process indicators on ISST-referred families

19. Next steps – long-term FY11-12 • Develop plan to identify education and health/mental health outcome indicators • Identify steps to explore use of data in incentive formulas • Continue to streamline annual reporting

20. Questions? • What is needed for our discussion about MOU indicators for next year? • What kind of processes do we need to put in place so that we are collecting the data that needs to be collected?