Measuring Public Health System Performance: Model from WA State September 12, 2013

1. Measuring Public Health System Performance: Model from WA StateSeptember 12, 2013 Marni Mason, MarMason Consulting, LLC MarMason Consulting

2. Marlene (Marni) Mason • More than 30 years in private healthcare and public health as clinician, manager and national consultant • Consultant for performance measurement and quality improvement (20+ years) • Consultant PH performance standards and improvement since 2000 and for all 3 Multistate Learning Collaboratives (2005-2011), including more than 50 QI teams from > 6 states • National trainer and presenter for QI and Accreditation in more than 20 states and for ASTHO, NACCHO, NIHB, NNPHI, and RWJF • Consultant for PHAB Standards Development and training of site reviewers (2008-2010) • Surveyor for National Committee for Quality Assurance-NCQA (15 years) and Senior Examiner for WA State Quality Award for Baldrige Performance Criteria MarMason Consulting

3. Performance Review: No single tool provides all the information needed-A variety of tools provides a full picture Quality Improvement Which efforts are working best for us? What is not working? How can we tell? Is the service efficient? Standards: Are we doing all the things we need to do? Are we doing them well? Are we missing important aspects of our work? Performance Results Indicators:How healthy are people now – and is that changing? Is public health service driving change – or something else? MarMason Consulting

4. Putting the pieces together Identify the weak spots in public health practice. Example: We have not systematically evaluated immunization efforts. Our immunization rates for 2 year olds appear and we have seen increased pertussis. Can we improve the effectiveness of this service? Standards and Measures Quality Improvement Efforts Improve what is not achieving the results we need Example: Outreach to medical providers, parents and day care to address immunization. Better data collection. Increased outreach to parents. Health Indicators Monitor Results Example: Did the strategies work? Immunization rates up? Pertussis down? If not – why not? Was success achieved one place – and why? MarMason Consulting

5. Accreditation and Improvement Cycles PLAN PLAN PLAN Accreditation Standards Accreditation Standards Target Improvements ACT DO DO ACT DO ACT Improvement work Areas for Improvement Modify or Implement Evaluate Areas for Improvement Evaluate STUDY STUDY STUDY Report/Recommend Study Results Report/Recommend Statewide Assessments 2002 (baseline) & 2005, 2008, 2011 remeasurement) Performance Improvement Cycles (statewide plus many individual agency efforts) Statewide Assessment (2014-2015 remeasurement) MarMason Consulting

6. Standards Apply Many Programs Domains or Standards Family Planning Programs Immunizations Monitor Health Status Communicable Disease Community Involvement Workforce Development Health Policy & Plans Performance Management Food Safety Chronic Disease MarMason Consulting

7. Washington Model • Review all local HDs and state DOH every three years at the same time (within 3 months) • Give options for scope of review: • PHAB Accreditation (meets performance review requirements) • Undergo full WA Standards review (majority of PHAB measures plus a few WA measures) • Undergo “Limited Set” review from WA Standards (approximately 45% of measures from full set) MarMason Consulting

8. What is Reviewed Varies by Measure • Once – review once to demonstrate agency performance • Strategic Plan • Health Assessment and Improvement Plan • Sample – review a sample of programs to determine consistency of performance across the agency • Use of performance measures • EH enforcement • All – review all programs or processes for performance with the standard • Confidentiality Agreements, HIPAA • Staff evaluations MarMason Consulting

9. Child Care First Steps Immunizations Nutrition & Physical Exercise Tuberculosis Selection from Program Menus Four programs (2 EH and 2 P&P) from program menus and Communicable Disease were reviewed. • Food Safety • Drinking Water • Wastewater Management • Water Recreational Safety Prevention & Promotion Environmental Health MarMason Consulting

10. Site Visit Process • Varying length and number of reviewers, depending on size • Document review scoring • Arrange for electronic copies of documents that are exemplary practices • Director/team closing session, including a review of strengths and opportunities for improvement related to each topic area of standards MarMason Consulting

11. Correlation of Budget & FTEs • Relationship of budget and FTEs to overall performance in the Standards is nearing random (little or no correlation) • Five non-urban LHJs with budgets of $2 million or less had > 60% demonstrated • There is variability not connected to budget or size, other drivers of high performance are local priority-setting; leadership; local funding; staff skill, training and experience; and documentation and data systems MarMason Consulting

12. Correlation of Annual Budget to Overall Performance MarMason Consulting

13. Correlation of Per Capita Budget No correlation or relationship of per capita budget to overall performance in the Standards MarMason Consulting

14. Overall LHJ and State Agency Performance by Standard, 2008 Demonstrated Partially Demonstrated Not Demonstrated 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% DOH 86% 12% 2% 1. Community Health Assessment 78% 14% 8% 1. Community Health Assessment LHJ DOH 88% 12% 2. Communication to the Public and Key Stakeholders 83% 14% 3% 2. Communication to the Public and Key Stakeholders LHJ DOH 3% 77% 21% 3. Community Involvement 41% 32% 28% 3. Community Involvement LHJ DOH 80% 20% 4. Monitoring and Reporting Threats to the Public's Health 83% 13% 4% 4. Monitoring and Reporting Threats to the Public's Health LHJ DOH 33% 63% 3% 5. Planning For and Responding to Public Health Emergencies 50% 31% 19% 5. Planning For and Responding to Public Health Emergencies LHJ DOH 79% 20% 1% 6. Prevention and Education 50% 33% 17% 6. Prevention and Education LHJ DOH 67% 26% 8% 7. Helping Communities Address Gaps in Critical Health Services 30% 13% 57% 7. Helping Communities Address Gaps in Critical Health Services LHJ DOH 63% 27% 10% 8. Program Planning and Evaluation 34% 31% 35% 8. Program Planning and Evaluation LHJ DOH 47% 16% 37% 9. Fiscal and Management Systems 35% 54% 11% 9. Fiscal and Management Systems LHJ DOH 59% 38% 3% 10. Human Resource Systems 58% 28% 14% 10. Human Resource Systems LHJ DOH 45% 50% 5% 11. Information Systems 50% 36% 13% 11. Information Systems LHJ DOH 75% 25% 12. Leadership and Governance 34% 38% 29% 12. Leadership and Governance LHJ Statewide System Results MarMason Consulting

15. Exemplary Practices- MarMason Consulting

16. Relevance to Oregon Scenarios • Can assist in implementation of 3 Scenarios: • 3. Adopt the revised standards document, keep triennial review in place as is, add full review (optional or required) of all accreditation domains • 4. Adopt the revised standards document, replace components of the Triennial Review with components of an accreditation domain review • 5. Replace current triennial review process with review of the 12 domains, work with PHD programs to identify documentation required for federal funding (this required documentation will remain in the review process) MarMason Consulting

17. What Questions Do You Have? Marni Mason 425-466-7965 marni@marmason.com MarMason Consulting