Evidence and Value as Foundations for Quality Measurement

1. Evidence and Value as Foundations for Quality Measurement Advanced Issues in Healthcare Performance Measurement Hospital Report Research Collaborative November 20, 2003 Alan M. Garber Stanford University

2. Defining Quality Improvement High quality care is Safe Effective Patient-centered Timely Efficient Equitable Institute of Medicine, Crossing the Quality Chasm, National Academy Press, 2001

3. High-Tech, High Cost

6. New Tech, Modest Cost

8. Judging Health System Performance �Quality� is often used as metric for level of performance How, if at all, should costs be considered in performance measurement?

12. Quality Measures and Performance Direct outcome-based measures used when possible Measures based on intermediate endpoints (e.g., blood glucose control in diabetics) and process (e.g., adherence to preventive care recommendations) often used instead Indirect measures often explicitly derived from evidence criteria

13. Choosing Quality Indicators: AHRQ HCUP Approach Which indicators are currently in use or described in the literature that could be defined using hospital discharge data? What are the quality relationships reported in the literature that could be used to define new indicators using hospital discharge data? What evidence exists for indicators not well represented in the original indicators�pediatric conditions, chronic disease, new technologies, and ambulatory care sensitive conditions?

14. Which indicators have literature-based evidence to support face validity, precision of measurement, minimum bias, and construct validity of the indicator? What risk-adjustment method should be suggested for use with the recommended indicators, given the limits of administrative data and other practical concerns? Which indicators perform well on empirical tests of precision of measurement, minimum bias, and construct validity?

15. AHRQ Prevention Quality Indicators � Ambulatory Sensitive Conditions � Bacterial pneumonia � Hypertension � Dehydration � Adult asthma � Pediatric gastroenteritis � Pediatric asthma � Urinary tract infection � Chronic obstructive pulmonary disease (COPD) � Perforated appendix � Diabetes short-term complication � Diabetes long-term complication � Lower-extremity amputation among patients with diabetes � Uncontrolled diabetes � Low birth weight � Angina without procedure � Congestive heart failure (CHF)

16. National Committee for Quality Assurance (NCQA) Accredits managed care organizations based on performance criteria in five areas: Access and Service Qualified Providers Staying Healthy Getting Better Living with Illness Also accredits other types of health plans

17. Sample NCQA Effectiveness Criteria Adolescent Immunization Status Antidepressant Medication Management Beta-Blocker Treatment After a Heart Attack Breast Cancer Screening Cervical Cancer Screening Childhood Immunization Status Chlamydia Screening Cholesterol Management After a Heart Attack

18. Specific NCQA HEDIS Measures the percentage of children who, by the age of 13, have received recommended immunizations for the continued protection against childhood diseases. the percentage of Medicare members over the age of 65 who received an influenza vaccination to prevent the flu (only for health plans serving Medicare beneficiaries). the percentage of women ages 52-69 who received a mammogram with the past two years to detect breast cancer early. the percentage of adult women who received a pap smear within the past three years to detect cervical cancer early. the percentage of pregnant women who received their first prenatal care visit during the first three months of pregnancy. the percentage of new mothers who received a check-up within eight weeks after deliver

19. Economist�s View of Efficiency Productivity analysis: how much output can be obtained from given input Distinguish between allocative inefficiency and x-inefficiency Quality measurement is incorporated in the output measure

20. Comparative Productive Efficiency

21. Can �Evidence-Based� Quality Measures Identify the Efficiency Frontier? Cost-effectiveness can be a tool to identify efficient health interventions Uncertainty about details of implementation Evidence-based criteria often � but not always � proxy for cost-effectiveness

22. Application of Evidence Evaluation In U.S., evidence evaluation is basis for approval of regulated drugs and devices, coverage decisions, and often for guideline development Canadian and U.S. preventive care evaluations helped promote the use of explicit evidence evaluation as basis for recommendations

23. Private plans and Medicare usually cover medical care that is known to be effective Categorical exclusions (e.g., cosmetic surgery, experimental therapy) Commercial plans cover all �medically necessary� services Medical necessity criteria variable Do not usually consider costs Typically evidence-based today Role in Coverage

24. What is �medically necessary�? Based upon prevailing practices/community standards in past Today explicit processes are usually evidence-based

25. The First and Second Questions of Evidence Evaluation Does it improve health? Are we sure it does?

26. Why incorporate value explicitly in performance measurement?

27. Limitations of Evidence Criteria Application sometimes ambiguous Specific criteria can appear arbitrary New (and old) technologies may meet evidence standards but provide little improvement over alternative forms of care

28. PET myocardial perfusion imaging appears to be more accurate than other noninvasive tests for coronary artery disease. Should it be considered a marker of quality care, or even a covered health benefit?

29. Implantable cardioverter-defibrillators prevent sudden cardiac death. Should they be used in everyone at elevated risk of sudden cardiac death? What risk is sufficient, and how should we identify individuals at elevated risk? Should implantation rates be incorporated in performance measures?

30. Should usage of expensive (as much as $US 20,000/ yr.) injectable medications for rheumatoid arthritis, such as Remicaid and Enbrel, be considered a quality marker?

31. The traditional Pap smear, used to prevent cervical cancer, costs as little as $US 8. Enhancements to Pap testing cost laboratories $10 or more � is their usage a good measure of the quality of care?

32. Technology and Expenditures

33. How New Technologies can Increase Expenditures New medical technologies do not meet a conventional market test Demand is highly subsidized: most consumers of medical services pay only part of the cost Moral hazard promotes innovation as well as overutilization Tend to seek quality improvement rather than price reduction

37. Evidence of effectiveness may not be adequate as sole standard for adoption

38. Evaluation of Health Technologies Cost-cost analysis: lowest cost approach to achieve given outcome Has limited applicability Cost-benefit analysis: Monetize both costs and benefits More versatile than, and more theoretically grounded than cost-effectiveness analysis Limited acceptance in health care

39. Measuring Value with Cost-Effectiveness Analysis Cost-effectiveness ratio measures cost per unit health effect gained Numerator: Difference between costs of the intervention and costs of the alternative Denominator: Difference between health outcomes with the intervention and health outcomes with the alternative

41. Steps to Economic Evaluation Selection of intervention and the comparison (alternative) strategy Definition of the clinical context Comprehensive measurement of costs of both intervention and alternative Selection of appropriate outcome measure There is not a unique recipe for performing the analysis

42. Include costs of intervention, as well as all costs that it might change Future costs induced by intervention Savings in future health care costs due to disease averted Possibly changes in future (non-health-related) consumption Costs to Include

43. Controversies in Cost-Effectiveness Analysis Whose costs count? What is valid measure of costs (marginal, average, long-term marginal costs)? Which future costs matter? How to estimate future costs? How to infer effectiveness from limited data?

44. Which Perspective to Use for the Analysis Whose costs matter? Societal perspective usually recommended May sometimes want to consider more limited perspective, e.g., perspective of government agency Perspective of ideal insurer (acting as a perfect agent for enrollees) Obtain greatest value for premium cost Most relevant for range of decision makers?

45. Controversies: Measuring Effectiveness Effectiveness measure needs to be comprehensive enough to capture all relevant health benefits When using an intermediate or �surrogate� endpoint, must have strong evidence linking the endpoint to a comprehensive health measure Often RCTs measure partial but not comprehensive outcomes precisely

47. Measuring Outcomes: Quality-Adjusted Life Years (QALYs) QALYs analogous to life expectancy except that each year is given an average �quality weight� Quality weight takes value between 0 (equivalent to death) and 1 (perfect health) QALYs always take values that are no greater than life expectancy Less weight is placed on future years (discounting)

48. Accounting for Uncertainty Traditional sensitivity analysis seeks to determine effects of uncertainty in model parameters Vary these values and show how the changes affect final results Probabilistic CE analysis seeks global measures of model uncertainty Medical community skeptical about precision and validity of results of models

50. Applying cost-effectiveness analysis to performance measurement

51. Incremental Analysis Cost-effectiveness ratio always between two relevant alternatives Never calculate an average cost-effectiveness ratio (C/E) When there are multiple alternatives, rank order � do not calculate all cost-effectiveness ratios relative to a single alternative

52. Dominance and Extended Dominance (Strict) dominance: One intervention more effective and less costly than an alternative Extended dominance: More expensive intervention has a lower cost-effectiveness ratio than the lower-cost option. Alternatively, one intervention is strictly dominated by a linear combination of two other interventions. Rational decision maker will not choose an option that is dominated by another, whether under strict or extended dominance

53. Testing for Coronary Artery Disease

55. Enhanced Cervical Cancer Screening Tests

57. Treatments for Advanced Prostate Cancer

59. Cost-Effectiveness �League Table� Source: Harvard Center for Risk Analysis Cost-Effectiveness Registry http://www.hsph.harvard.edu/cearegistry/1976-2001_CEratios_comprehensive.pdf

60. General Limitations of CEA Cost-effectiveness analysis results are only as valid as the underlying cost and effectiveness information The most important uncertainties usually arise from limitations of effectiveness data Cost-effectiveness analysis makes formerly implicit assumptions explicit CEA does not obviate the need to make tough decisions

61. Use in Performance Measurement Provide information that can inform deliberative processes for determining which interventions should be considered markers of quality care Identify forms of care that represent extremely poor uses of resources In practice, application of cost-effectiveness criteria most acceptable for diagnostic and screening tests, preventive interventions Application of a fixed cost-effectiveness threshold (�equalization at the margin�) moves to efficiency frontier, and the level of the threshold identifies point on the frontier

62. Comparative Productive Efficiency

63. Application Challenges Avoid double-counting costs or savings Need consensus about threshold cost-effectiveness level Need to address uncertainty better � complement evidence standards Application least controversial when identifying dominated alternatives and extreme CE values Address implementation when benefits are highly heterogeneous Explicit CE criteria can enhance the relevance of performance criteria