Evidence and evaluation in health (and health care)

1. Evidence and evaluation in health (and health care) SIPR/SDHI Seminar 5th December 2008 Anne Ludbrook

2. Evidence about what? • Is the activity or service worthwhile? • Requires comparison of costs and benefits • Cost-benefit analysis or cost-consequence analysis • Is this an efficient way to achieve a given outcome? • Compare cost per unit of output across alternatives • Cost-effectiveness analysis • Are the health (policing) benefits of this activity or service justified? • Compare costs and health outcomes (QALYs) or policing outcomes • Cost-utility analysis

3. Key steps • What is already known? • Reviewing existing evidence • What are the alternatives? • Relevant to decision to be made • Evaluation design • Attribution of effects to interventions • Identifying, measuring and valuing costs and benefits

4. The first three steps are evaluation tasks • Economic evaluation centres on the comparison of costs and benefits • Good economic evaluation requires good evaluation • Key to success in health and health care has been the development of evidence based approaches

5. How can we generate evidence? • Primary research • Include economics at an early stage • Ensure that appropriate economic evidence is collected • Targeted evaluations • Well designed • Sufficient size • Adequate resources • Secondary research • Understanding and measuring systematic linkages • Enhancing secondary data sets

6. Making better use of existing knowledge • “Cost-benefit analysis would also be a useful contribution to the evidence on neighbourhood policing … limited in the NRPP because cost data were only available for one site and no agreed estimates of the cost of fear of crime or low public confidence are currently available.” (National Reassurance Policing Programme) • Economic evaluation is not a substitute for decision-making but rather a framework for organising what we know, highlighting what we don’t know, assessing uncertainty and making explicit value judgements.

7. Presenting information • Cost per intervention site - £100k on average • Example effects for one site • Change in recorded crime • 125 fewer offences • 36% points better than control • Feeling safe after dark • 11% points better than control • Public confidence in police • 12% points better than control • Does this look like value for money? What else could be bought with the same budget?

8. How could we value the benefits? • Traditional approach – avoided costs of crime • Create a policing ‘QALY’ • Consumer values – • Direct – willingness to pay • Indirect – discrete choice methods provide weights or monetary values

9. Adapting the economic evaluation framework • Particular challenges in health (and policing) • Multiple outcomes • Outcomes may be long term • Measuring outcomes • Valuing outcomes • Willingness to pay; Discrete choice experiments • Individual costs and benefits not just public sector • Attribution of outcomes • Standards of evidence for decision making • Distributional issues / inequalities

10. Wider contribution of economics • Priority setting frameworks • Workforce issues • Community values / preference elicitation