C ontinuous Q uality I mprovement

1. MIECHV Grantee Meeting Continuous Quality Improvement Michelle P. Hill mhill@pa.gov 717.772.4850 6/5/2013 Departments of Education and Public Welfare | www.education.state.pa.us | www.dpw.state.pa.us

2. Session Agenda • Introduction to Quality Improvement • Pennsylvania’s MIECHV CQI Plan • CQI Team • Culture of Quality • CQI Plan • Model for Improvement

3. Introduction to Quality Improvement • Pennsylvania’s MIECHV CQI Plan • CQI Team • Culture of Quality • CQI Plan • Model for Improvement

4. Brainstorm:What is CQI?

5. The BIG picture • Quality improvement comes from a desire to make things better • At first glance, quality improvement might seem like more work piled on to an already impossible work load • But if quality improvement can ultimately lessen that workload, and allow you to provide more efficient/better home visits for your families, leading to better outcomes, isn’t it worth it?

6. Quality Improvement  Quality Assurance

7. Shifting Away fromQuality Assurance Quality assurance (QA) measures compliance against certain necessary standards. QA can be reactive, retrospective, policing, and in many ways punitive. It often involves determining who is at fault after something goes wrong. However, standards and measures developed for quality assurance can inform the quality improvement process.

8. So, what is quality improvement? • a process of continuousimprovement • involves both prospective and retrospective reviews • improvement -- measuring where you are, and figuring out ways to make things better • create systems to prevent errors from happening • improving how things work • trying to find where the “defect” in the system is • figuring out new ways to do things • “think outside the box”

9. Introduction to Quality Improvement • Pennsylvania’s MIECHV CQI Plan • CQI Team • Culture of Quality • CQI Plan • Model for Improvement

10. An Opportunity Comprehensive, multi-level CQI is a federal grant requirement and a commitment Pennsylvania has made in its MIECHV CQI Plan . . . but more importantly, it is an opportunity: • To refine and share best practices; • To enhance decision-making; • To individualize services for participating families across sites and programs

11. A Shared Responsibility • CQI leverages the expertise and perspective of project participants across roles, levels, and agencies • To work well, CQI needs to be a safe space for constructive input from all • It can’t just be something the state, the programs, or the “data people” are doing alone • Community input is vital • Communication is key – challenges and successes need to be freely shared

12. Pennsylvania’s MIECHV CQI Plan • 3 teams • Bottom-up approach

13. Survey Results 30 different agencies implementing MIECHV programs 25 agencies responded to the survey 21 have a CQI plan, CQI lead and/or a CQI team 3 have a CQI lead and/or a CQI team, but no CQI plan 1 does not have a CQI lead/team or a plan

14. Discussion:Share current CQI processes

15. Introduction to Quality Improvement • Pennsylvania’s MIECHV CQI Plan • CQI Team • Culture of Quality • CQI Plan • Model for Improvement

16. Initial Activities (of all Teams) • learn about CQI • define the role and function of a CQI team • determine the frequency of CQI team meetings and activities • identify the people/positions responsible for elements of the CQI plan • provide oversight of the CQI Plan • define quality for the MIECHV program • identify goals and objectives

17. Team Membersshould include the following: • Lead – oversees the team; facilitates the meetings • Scribe – captures all documentation • Member – provides information and helps make good decisions • Technical Expert – subject matter experts

18. MIECHV State CQI Team • Provides an opportunity to address statewide issues and consolidate information and issues from all other teams. • Meetings will be held quarterly in-person.

19. MIECHV State CQI Team 1 Members are optional attendees at meetings. 2 Denotes subcontractors.

20. MIECHV State CQI Team Possible issues to be addressed are: • Monitor data collected across programs • Discuss unresolved issues from Program and Local Teams • Ongoing comprehensive program evaluation • Monitors comprehensive program model fidelity • Communicate shared successes and resolved issues • Report on aggregated data • Assist in the development of the PA MIECHV State CQI plan Through tracking the output and process indicators related to the constructs, the MIECHV State CQI Team will be able to identify specific agencies that require additional support. CQI activities will focus on ensuring that state goals are met. Data will be analyzed for progression with enrollment, provider training, frequency of home visits, and data entry compliance, in addition to monitoring whether one site or model is doing better than others on certain benchmarks and why, including identification of best practices and lessons learned.

21. MIECHV Program CQI Team • Provides an opportunity to address issues that impact the MIECHV programs as a whole and any unresolved issues presented by the MIECHV Local CQI Teams. • Meetings will be held every other month via webinar/phone (in-person when possible) .

22. MIECHV Program CQI Team

23. MIECHV Program CQI Team Possible issues to be addressed by the MIECHV Program CQI Team are: • Discussion of common issues (and shared successes) across agencies and models • Discussion of unresolved issues brought up from MIECHV Local CQI Teams • Discussion of data • Discussion of model fidelity • Review and provide feedback to State on the use of assessment tools, program questionnaires, and implementation of the MIECHV program The overall goal of the MIECHV Program CQI Team is to continuously improve the CQI process so that the MIECHV Local CQI Teams have fewer challenges and greater success in optimally serving participants. Members of the MIECHV Program CQI Team will work directly with the MIECHV Local CQI Teams.

24. MIECHV Local CQI Teams • Provide an opportunity to address issues that impact individual MIECHV agencies. • Meetings will be held monthly in-person.

25. Discussion:Local CQI Team Members

26. MIECHV Local CQI Teams

27. MIECHV Local CQI Teams The MIECHV Local CQI Teams will be required to: • Have a CQI plan, which will be revised, at a minimum, annually • Focus on agency level data • Monitor model fidelity at the agency level • Determine areas of improvement • Address quality improvement challenges (i.e. staff retention, participant attrition) • Identify issues with program evaluation(s), communication, information systems, hiring, safety, etc. Issues that are not resolved can be brought to the MIECHV Program CQI Team. The EDUCATIONAL RESEARCH ASSOCIATE 2 and STATISTICAL ANALYST 1 will be tasked with regular dialogue with local sites around CQI. 

28. Introduction to Quality Improvement • Pennsylvania’s MIECHV CQI Plan • CQI Team • Culture of Quality • CQI Plan • Model for Improvement

29. Components of a Culture of Quality • Generate buy-in at all levels: movement from an ‘us’ to a ‘we’ • Understanding the system: process steps that lead to the outcome • Data collected that are relevant and meaningful • Utilization of data to monitor progress towards established target • Small-scale tests of interventions

30. Understanding of Current Culture • Mission, vision and values support culture • Determination of where we are today vs. where we want to be • Consistent and frequent methods of communication • Lines of accountability • Expectations for performance • Processes and procedures

31. Agencies strive to reach pre-determined targets. Every situation can be a learning experience. Data are essential to day-to-day work. • All members are committed. Key members are trained in QI methods and support strategic initiatives. Frontline practitioners are supported  by leadership. Improvement is guided by team  approach and accountability. Data are readily accessible. Data collection infrastructure exists. Reporting metrics are meaningful. Data elements are relevant, accurate, and important. Reports are timely and recent. Reports show performance relative to targets. Reports show changes over time. Reports break out small units. • Outcomes are clearly articulated and understood. Outcomes are measured against an established target. All process measures are tied to outcomes. There is a deep understanding of systems, processes, and procedures. Performance is reported frequently. • Practices and methods are shared. Results and outcomes are shared. Individual contributions are recognized. Open communication and critical scrutiny are welcomed. Shared learning decreases the need to ‘reinvent the wheel.’

32. Organizational Culture of Quality Checklist • Transparent • Curious and eager to learn • Open to failure as a learning opportunity • Committed to improving processes and outcomes • Data-driven with sufficient MIS capacity • Trained from top to bottom in CQI methods

33. MIECHV CQI Process Map 1: Data Collection Process MIECHV CQI Process Map 1 provides an overview of the data collection process, from client intake to data extracts from PA MIECHV and NFP ETO data systems. Data collected by home visitors Intake Client enrolled Data brought back to local MIECHV agency Data extracts from PA MIECHV & NFP ETO data systems Data entered into PA MIECHV & NFP ETO data systems

34. MIECHV CQI Process Map 2: Data Sharing Process Once all data are received , the STATISTICAL ANALYST 1 will analyze the data and create trend reports. These reports will be shared each month with the MIECHV Local CQI Teams. After reviewing the data at the agency level, the MIECHV Local CQI/Data lead will work with the STATISTICAL ANALYST 1 to improve data quality. The STATISTICAL ANALYST will then upload all data into an Excel spreadsheet to perform data clean- up while observing for outliers and continuing to follow-up as needed with sites for missing / questionable data. The INFANT TODDLER SPECIALISTS and Model TAs will also follow-up with sites on any program issues/TA needs. The STATISTICAL ANALYST 1 will conduct data analysis and create reports for the MIECHV Local CQI Teams; this will enable the local agencies to see how data quality can affect results, as well as to see their final results.

35. MIECHV CQI Process Map 2: Data Sharing Process Reports will also be shared with the MIECHV Program CQI team (every other month), MIECHV State CQI team (quarterly), Home Visitation Stakeholder Committee (quarterly), and the Early Learning Council (twice a year). While reports for the MIECHV Program and State CQI teams will include data that is identified at the agency level, the other two teams will only see data that is de-identified. The goal of sharing data with the Home Visitation Stakeholder Committee and the Early Learning Council is to provide them information about the children and families being served through MIECHV as well as Pennsylvania’s progress towards meeting the benchmarks.

36. Introduction to Quality Improvement • Pennsylvania’s MIECHV CQI Plan • CQI Team • Culture of Quality • CQI Plan • Model for Improvement

37. Discussion:CQI plan

38. Introduction to Quality Improvement • Pennsylvania’s MIECHV CQI Plan • CQI Team • Culture of Quality • CQI Plan • Model for Improvement

39. Model for Improvement 2 parts: (1) THREE fundamental questions • What are we trying to accomplish? Set an aim. • How will we know that a change is an improvement? Establish measures. • What changes can we make that will result in improvement? Take action. • We know that improvements require change, however not all changes are an improvement. We have to test whether a change is an improvement, and that’s where the PDSA cycle comes in.

40. Model for Improvement 2 parts: (2) Plan-Do-Study-Act (PDSA) cycle • Plana change. Clarify the objective, make a prediction, figure out the who, what, where, when, and how. • Do (Test) the change. Carry out the plan, document any unexpected problems or challenges, begin analyzing data. • Study (Review) the tests. Complete data analysis, compare results to theory and prediction, summarize and present data. • Take Action based on what has been learned. • If the change did not work, go through the cycle again with a different change. If the change was successful, use what has been learned to begin planning new improvements.

41. How Do Tests Lead to Improvements? Changes that result in improvement The cycles build on each other… Implementation of change Wide-scale tests of change Follow-up tests Very small scale test

42. Things to Remember • Keep it small – design the first test for one home visitor, 5 clients • Scale down the time frame (weeks, days, hours) • Keep the first tests simple • See what happens, act on that knowledge, and then scale-up the test • And remember You’ll never make a change if you don’t… actually make a change.

45. Time to Practice

46. Survey Results Top eight ranked areas of improvement (in order from the highest to the lowest ranked): • retention of families • improvements in family self-sufficiency/stability • families are connected to needed services and social supports • recruitment of families • increase in screenings for domestic violence • improved parenting skills • community partnerships • prevention of injuries and maltreatment

47. Next Steps • Identify agency-level CQI team members • Share current CQI plan • Plan initial agency-level CQI meeting • Identify initial measures and areas of focus • Seek technical assistance