THE USE OF SYSTEMATIC REVIEWS IN EVIDENCE-BASED DECISION MODELLING

1. THE USE OF SYSTEMATIC REVIEWS IN EVIDENCE-BASED DECISION MODELLING Nicola Cooper, Alex Sutton, Keith Abrams, Paul Lambert Department of Epidemiology & Public Health, University of Leicester.

2. BACKGROUND • Increasingly decision models are being developed to inform complex clinical/economic decisions • Parameters can include: • clinical effectiveness, • costs, • disease progression rates, and • utilities • Evidence based - use systematic methods for evidence synthesis to estimate model parameters with appropriate levels of uncertainty

3. SOURCES OF UNCERTAINTY IN DECISION MODELS • Statistical error • Systematic error • Evidence relating to parameters indirectly • Data quality, publication bias, etc.

4. MARKOV MODEL – TAXANE vs. STANDARD (2nd line treatment of advanced breast cancer) QR , CR QS , CS Response Stable PSR PR PS PRP PSP  Progressive QP , CP PP PPD Probability (P) Death Quality of Life (Q) Cycle length 3 weeks Cost (C) QD = 0

5. GENERAL APPROACH • Meta-analyse available evidence to obtain a distribution for each model parameter using random effect models • Transform the pooled results, if necessary, and input into the model directly as a distribution and evaluate the model • All analyses (decision model and subsidiary analyses) implemented in one cohesive statistical model/program • Implemented in a fully Bayesian way using Markov Chain Monte Carlo simulation within WinBUGS software • All prior distributions intended to be ‘vague’. Where uncertainty exists in the value of parameters (i.e. most of them!) they are treated as random variables

6. MARKOV MODEL – TAXANE vs. STANDARD (2nd line treatment of advanced breast cancer) QR , CR QS , CS QP , CP Quality of Life (Q) Cost (C) QD = 0 Response Stable PSR PS PR PRP PSP  Progressive PP PPD Probability (P) Death Cycle length 3 weeks

7. MODEL PARAMETER ESTIMATION e.g. PSR, TAX – The probability of moving from stable to response in a 3 week period 1) M-A of RCTs: Annual ln(odds) of responding • 2) Pooled ln(odds) distribution Chan Nabholtz Sjostrom Bonneterre -0.3 (-0.9 to 0.3) PSR Combined .1 .25 1 5 Respond Stable Odds - log scale 3) Transformation of ln(odds) distrn to transition probability 4) Apply to model Progressive Death

8. THE REMAINING PARAMETERS • The Transition Probabilities need estimating for each intervention being compared • Costs and Utilities can be extracted from the literature and synthesised using a similar approach within the same framework

9. META-ANALYSES OF LITERATURE (where required)

10. TRANSITION PROBABILITIES FOR MODEL (Derived from M-As)

11. EVALUATION OF THE MODEL • A cohort of 1,000 persons is run through the model over 35 3-weekly cycles (until the majority of people are dead) for each treatment option • Costs and utilities are calculated at the end of each cycle and the average cost and utilities for an individual across all 35 cycles for each treatment option are calculated • This process is repeated 4,000 times (each time different values from each parameter distribution are sampled)

12. COST-EFFECTIVENESS PLANE Bayesian (MCMC) Simulations £10,000 £8,000 £6,000 Taxane more Standard effective but £4,000 dominates more costly Incremental cost £2,000 £0 -0.50 -0.40 -0.30 -0.20 -0.10 0.00 0.10 0.20 0.30 0.40 0.50 -£2,000 Taxane less Taxane costly but less dominates -£4,000 effective Incremental utility

13. CLINICAL NET BENEFIT- Warfarin for non-rheumatic atrial fibrillation • Evidence that post MI, the risk of a stroke is reduced in patients with atrial fibrillation by taking warfarin • However, there is a risk of a fatal hemorrhage as a result of taking warfarin • For whom do the benefits outweigh the risks?

14. EVALUATION OF NET BENEFIT ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Net Benefit

15. EVALUATION OF NET BENEFIT Meta analysis Multivariate risk equations of RCTs ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Meta analysis of Net Benefit QoL study RCTs obs studies

16. 10 6 8 6 4 4 2 2 0 0 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -1.5 -1.0 -0.5 0.0 0.5 1.0 reduction in relative risk 300 250 0.4 200 150 0.3 100 0.2 50 0.1 0 0.002 0.004 0.006 0.008 0.010 0.012 0.014 risk of bleed per year 0.0 0 20 40 60 80 100 Outcome ratio EVALUATION OF NET BENEFIT Meta analysis Multivariate risk equations of RCTs ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Meta analysis of Net Benefit QoL study RCTs obs studies

17. 10 6 8 6 4 4 2 2 0 0 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -1.5 -1.0 -0.5 0.0 0.5 1.0 reduction in relative risk 300 250 0.4 200 150 0.3 100 0.2 50 0.1 0 0.002 0.004 0.006 0.008 0.010 0.012 0.014 risk of bleed per year 0.0 0 20 40 60 80 100 Outcome ratio Multivariate Risk Equation Data Net Benefit (measured in stroke equivalents) No. T hrombo - Clinical No. of % of embolism Mean Median Probability of risk patients cohort rate (% (s.e.) (95% Benefit > 0 Simulated PDF factors per year CrI) (95% CI)) 6 5 4 2 or 3 68 12 17.6 (10.5 - 0.0004 0.06 54.2 % 3 to 29.9) (0.15) ( - 0.29 to 2 0.20) 1 0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 2 or 3 Clinical factors EVALUATION OF NET BENEFIT Meta analysis Multivariate risk equations of RCTs ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Meta analysis of Net Benefit QoL study RCTs obs studies

18. ADVANTAGES OF APPROACH • Synthesis of evidence, transformation of variables & evaluation of a complex Markov model carried out in a unified framework • Facilitates sensitivity analysis • Provides a framework to incorporate prior beliefs of experts • Allows for correlation induced where studies included in the estimation of more than one parameter • Uncertainty in all model parameters automatically taken into account • Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propagated

19. FURTHER ISSUES • Handling indirect comparisons correctly • E.g. Want to compare A vs. C but evidence only available on A vs. B & B vs. C etc. • Avoid breaking randomisation • Necessary complexity of model? • When to use the different approaches outlined above? • Incorporation of Expected Value of (Perfect/Sample) Information • Incorporation of all uncertainties