Estimating Return on Investment for Public Health Improvements Introduction to a new t ool

1. Open Forum Milwaukee, Wisconsin June 12th, 2013 Karl Ensign, Director of Evaluation Association of State and Territorial Health Officials (ASTHO) Estimating Return on Investment for Public Health ImprovementsIntroduction to a new tool

2. Helps answer the following questions: • Are we making the right investments? • Are we becoming more efficient? • What bang are we getting for our buck? • What is our budget accomplishing? • Are we being good stewards? • Must be done thoughtfully and carefully ROI is one way of measuring and communicating public health effectiveness in a manner that is particularly salient for policymakers, funders, administrators and the general public

3. ROI is a Specialized Form of Analysis • Outcomes are monetized: • Compares the costs of an intervention with its benefits – in financial terms • Yields the net return on investment – over time • Sensitivity analysis can be conducted – different levels of investments and their benefits

4. ROI Calculated: • Net benefit = Benefits – Costs • ROI = Benefits – Costs Costs • Hypothetical values: • $5 = ($400 + $500 + $300) – ($150 + $50) ($150 + $50) • Or …

5. “…a dollar spent on pediatric immunization is estimated to save $5 in treating preventable illness.”

6. Where and How has ROI been used in Public Health? • Public health programs (injury prevention, tobacco prevention and control) • Aggregate public health spending (Glen P. Mays, UKY) • Improvement projects undertaken by agencies

7. Challenges from an Agency Perspective • New data probably need to be collected • An investment may take years to produce benefits • Benefits may accrue outside the agency (population health) • Benefits may be difficult to link back to a specific public health program(s) or function(s)

8. ROI of QI – Advantages and Disadvantages • ROI is more immediate • Timeframe is shorter • ROI accrues more directly to the agency • Tends to be narrower in scope • Return is more modest

9. Hot off the PressesTHE NEW ROI TOOL!! • Funding from Prevention & Public Health Fund, ACA • Measures ROI for improvement projects • QI projects undertaken through the National Public Health Improvement Initiative (NPHII) • ASTHO led workgroup provided input and guidance • CDC, state and local agencies, foundations, academia • Developed by Glen Mays, University of Kentucky • Beta tested by Connecticut, Maine, and Virginia

10. Pathways to Realizing ROI for QI • Reductions in standard operating costs • Greater efficiencies realized • Revenue enhancements • Increased cost reimbursement • Increased productivity of agency functions • Increased service encounters • Decreased time to produce outputs • Reduced cycle time process

11. Planning and Development Costs

12. Routine Operating Costs

13. Outcomes/Outputs

14. How the Tool Calculates ROI • ROI = Benefits – Costs Costs Improvements in Routine Operations – Investment Costs Investment Costs (Routine Operations + Other Outcomes) – Investment Costs Investment Costs

15. Incorporates Standard Accounting Practices in ROI Calculation • Amortization • The cost of an investment should not be absorbed entirely in the first year • Amortization rate spreads the agency’s cost/investment over the useful life of the product • Present value • The relative worth of a single dollar changes over time • Accurate comparisons are made by applying a discount rate (inflation) to • Costs • Returns

16. Tool Can Be Used throughout Project • Prospectively – Planning Phase • Implementation Phase • Retrospectively – Post Implementation

17. Apply ROI Correctly • Build evaluation methods – including ROI – into program inception • Clearly specify intended purpose and use of ROI

18. Conduct ROI through a transparent process • Conduct ROI through an inclusive process • Implementers and end users of the analysis

19. Testing the New ROI Tool in Connecticut: A Prospective Look Susan Logan, MS, MPH Connecticut Department of Public Health June 10, 2013

20. Improving the Connecticut Department of Public Health Databases • AIM: Make three DPH databases compliant with the CT DPH policy on collecting sociodemographic data (similar to federal OMB-15 directive in 1997) • Increase the percentage of compliant databases from 4% to 10% • Put a plan in place to modify the remaining databases • Make sure that all newly-designed databases are aligned with the DPH data collection policy. • The modification process for the targeted databases was implemented and this strategy will be followed on a continuous quality improvement basis through 2014. Background on Quality Improvement Initiative

21. Process

22. Determining Investment Costs: QI Planning and Implementation • Pre-implementation/Baseline Period • Project team mid-point annual salaries x FTE spent (plus Fringe) • Include employees assisting with planning • Office operations costs (printing, supplies) • Travel for grant award • Implementation and Post-Implementation Periods • Project team mid-point • annual salaries x FTE spent (plus Fringe) • Include employees assisting with implementation • Cost of contracting out services • Office operations costs (printing) • Travel for grant award • Modifications to Databases • Communications (storyboards, easels, DPH Newsletter)

23. Approximately how many meetings did you attend regarding this QI project? ________ • Approximately how much time did you spend on the following activities related to this QI project? • Attending formal or informal meetings? hours • Making changes to data collection forms? hours • Making changes to databases? hours • Generating cost estimates for changes? hours • Other activities related to this QI project? hours • Please briefly describe these activities: ________________________________________________________________________________________________________________________________________ Customer Satisfaction Survey on QI Process Collected Data on Time Spent on Planning and Implementation Activities:

24. Takes into account time prds (4.7mos baseline) Salary Costs for QI Initiative Team

25. Salary Costs for QI Initiative Team: Implementation Prds 2 and 3 Data quality committee reviewing databases and overseeing fixes (ImplPrds 2 - 3)

26. Non-Personnel Investment Costs: Planning and Implementation Periods

27. Determining Costs and Savings for Outcomes and Outputs: ROI Produced • Baseline and Post Periods • Measures of Production Time • Time to analyze data with standardized sociodemographic categories (expected reduction) • Time to review databases (improves over time) • Improve Health Outcomes: Based on finding new cases in sociodemographic groups • Taking into account the Investment and Routine Operating Costs Only • Implementation Period 1 • Benefit (savings) was 64 cents for every dollar spent • Implementation Period 2 • Cost was $2.21 for every dollar spent • Adding in Outcomes • Implementation Period 1 and Overall • Savings was $17.05 ($18.16 overall) for every dollar spent

28. Outcome/Output Measures: Production Time Costs and Savings

29. Did not include data in ROI calculations (unchecked) Outcome/Output Measures: Reach Outcomes: Rates Used these rows as a worksheet for the health outcomes on next slide

30. Include data in ROI calculations (checked) Outcome/Output Measures: Health Outcomes Savings

31. Return on Investment Analysis

32. Used prospectively in this case • Seemed daunting at first, but once data is collected, calculations are done for you • Used customer satisfaction survey to gather data on time spent on planning and making modifications to databases • Customized to fit our needs Highlights of Using the ROI Tool for a Connecticut DPH QI Project

33. Can be applied for many different types of QI projects • Data quality projects may not have a large return, but it is the “right thing to do” • Projects that improve efficiency and reduce process steps/time will see a greater return on investment • Future Use of the ROI Tool: • ASTHO National Demonstration Project (3/1/12 – 10/31/13) • Make data accessible to local health departments – reduce time for state DPH staff to handle data requests • Save time cleaning up data Applications of the ROI Tool

34. Determining Return on Investment (ROI) Josh Czarda, Performance Improvement Manager, Virginia Department of Health

35. Example 1: Justifying Continuing Operations Increase Enrollment in Plan First – A Brief Background (A Medicaid Family Planning Program) Knowledge Individuals Providers Enrollment Process Eligibility Application Process/ Form Length Registration WebVision DSS System Understanding & Function Birth Control Knowledge Demographic, Age, Income, Insurance No More Needed Eligible LHD STD Testing Possible - Prompt for Plan First. Clerk still must know parameters Service Eligibility Levels Knowledge Other – Family Planning Ask Questions to Determine Eligibility Title X & Payment Issues Service Provided Auto-Enrollment & Payment External Provider If eligible – fill out application/ self populates some fields WIC Clinic Billing Not Interested Application Given to Individual If Interested Individual Signs & Dates Tracking/ Notification LHD Bills Medicaid Post Partum Tracking & Follow Up Capability No Incentive LHD Submits Printed App to DSS Individual Follows Up

36. Background - Implementing Changes and Greater Challenges Realizing Health Outcome ROI

37. Step 1: Calculating All Costs Pre-Implementation – Implementation Phases Initial PI Teams Costs DMAS, DSS & Local Health District Staff Additional Time, Billing & Enrolling Hired 2 FTEs to Run Program Marketing Outreach, Focus Groups, etc

38. Step 1: Calculating All Costs Pre-Implementation – Implementation Phases

39. Step 2: Measuring Immediate Impact

40. Initial View of Short Term ROI Calculations Using the Tool

41. Step 3. Measuring ROI of Long Term Health Outcome Impact

42. Step 3. Measuring ROI of Long Term Health Outcome Impact • Working with DMAS to Determine Cost Per Pregnancy to Medicaid • Calculating Variation in Pregnancy Rate to Determine Savings • Calculating ROI Compares All Costs Including Medicaid Payments for Plan First

43. Example 2: Proactively Determining Is A PI Project Worth It? Automating the VDH Travel Process Traveler Number of TARs (1000) Estimated $130,000 Programming Costs Supervisor X Business Manager Time Spent Per Unit Office Director Optimistic Future State T&E $40,293 VS. X Executive Advisor Salary Cost Per Unit Deputy Commissioner = Commissioner Travel Cost Increase From Approval Delay = 73K in Time and Effort + Secretary HHS

44. Sample Routine Operating Cost Calculation

45. TAR Automation ROI

46. Lessons Learned • The tool helps you recognize unanticipated costs and can be great for both prospective and retrospective decision making • ROI Analyses Must be Dynamic • PI Projects vary in complexity and metrics • ROI Tools must be flexible • Amortization of project costs can be challenging as change processes fluctuate • Illustrating & Showing ROI is Critical to Drive & Promote Continuous PI • Measuring Health Outcome ROI Remains A Critical Challenge • ROI for public health remains un-quantified for Public Health • A National Standardized Cost Analyses Calculating Cost Per Incident/Unit should be established