1 / 75

Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Re

Using Randomized Trials to Quantify Treatment Effects. Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited. 26 October 2006 Michael A. Kohn, MD, MPP. Quantifying the Benefit of a Treatment: Take Home Points.

arlo
Télécharger la présentation

Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Re

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Randomized Trials to Quantify Treatment Effects Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael A. Kohn, MD, MPP

  2. Quantifying the Benefit of a Treatment: Take Home Points • RCT Checklist: Importance of patient-oriented effect measures (POEMs), randomization, intention-to-treat analysis, good follow-up, blinding, and between-groups comparisons. • The Relative Risk or Relative Risk Reduction associated with an intervention is of minimal use* without a baseline prevalence of bad outcomes. • You need to have an absolute risk reduction to calculate number needed to treat. (NNT = 1/ARR) • For undesired effects of treatment, calculate the absolute risk increase (ARI), and the number needed to harm (NNH = 1/ARI) • Back-of-the-envelope CEA: Cost per bad outcome prevented = Treatment Cost x NNT *Unless the RR is 1 and RRR is 0.

  3. RCT Checklist

  4. RCT Checklist for Study Validity* • Patient-Oriented Effect Measures (POEMs) vs. surrogate outcomes • Randomization to address issues of confounding • Intention-to-treat analysis (once randomized always analyzed) • Good follow-up to eliminate differential losses to follow-up • Blinding of patients and clinicians to prevent differential co-interventions • Blinding of outcome assessors to prevent bias • Between groups rather than within groups comparison • Compare entire randomization groups, not subgroups *For checklist on study validity, see Chapter 1B1 “Therapy”, in Guyatt and Rennie (eds.), Users Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; AMA Press; 2002. (Or try http://www.cche.net/usersguides/therapy.asp#Valid )

  5. RCTs of Orthopedic Treatments • Kirkley A, Griffin S, Richards C, Miniaci A, Mohtadi N. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder. Arthroscopy. Jul-Aug 1999;15(5):507-514 2. Parker MJ, Khan RJ, Crawford J, Pryor GA. Hemiarthroplasty versus internal fixation for displaced intracapsular hip fractures in the elderly. A randomised trial of 455 patients. J Bone Joint Surg Br. Nov 2002;84(8):1150-1155.

  6. Endpoints: Arthroscopy vs. immobilization for 1st shoulder dislocation Kirkley A, Griffin S, Richards C, Miniaci A, Mohtadi N. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder. Arthroscopy. Jul-Aug 1999;15(5):507-514.

  7. Outcomes Affected by Treatments* • Dichotomous (e.g. recurrent dislocation) • Continuous (e.g. WOSI**) Endpoints • Patient relevant (e.g., ability to return to sports) • Surrogate (e.g., MRI findings) * Example: Arthroscopy vs. conservative tx for 1st Anterior Shoulder Dislocation (Arthroscopy. 1999 Jul-Aug;15(5):507-14. ) **Western Ontario Shoulder Disability Index

  8. Outcomes Affected by Treatments • Dichotomous (e.g. recurrent dislocation) • Continuous (e.g. WOSI) Endpoints • Patient relevant (e.g., ability to return to sports) • Surrogate (e.g., MRI findings)

  9. Randomization, Intention-to-Treat Analysis, and Follow-up: Arthroplasty vs. Internal Fixation? • Pt is a 81-year-old woman with a displaced, intracapsular femoral neck fracture. • Pt’s son is a physician. He asks about hemiarthroplasty vs. internal fixation. Pubmed search  Parker MJ, Khan RJ, Crawford J, Pryor GA. Hemiarthroplasty versus internal fixation for displaced intracapsular hip fractures in the elderly. A randomised trial of 455 patients. J Bone Joint Surg Br. Nov 2002;84(8):1150-1155.

  10. Randomization, Intention-to-Treat Analysis, and Follow-up: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Randomized controlled trial of the effects of hemiarthroplasty vs. internal fixation on re-operation and other outcomes in > 70-year-old patients with displaced, intracapsular femoral neck fractures.

  11. Randomization: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Why do a randomized experiment? Why not do an observational study comparing mortality, re-operation rates, etc. between hemiarthroplasty patients and internal fixation patients?

  12. Intention-to-Treat: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Some patients randomized to the hemiarthroplasty group ended up getting internal fixation. Why not include these patients’ outcomes in the internal fixation group or at least exclude them from the hemiarthroplasty group?

  13. Losses to Follow-Up: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.* If each treatment group had 20% loss to follow-up, there could still be bias. What if those in the internal fixation group were lost to follow-up because they got better and those in the hemi-arthroplasty group were lost because they died? *In fact, there were no losses to follow-up in this study.

  14. Blinding Blinding of Patients • Addresses placebo effect Blinding of Patients and Clinicians • Eliminates differential co-interventions Blinding of Outcome Assessment • Eliminates biased outcome assessment

  15. Blinding Blinding less important when opportunity for cointerventions that affect outcomes is minimal, and outcome is not subjective. • Arthroplasty vs Internal Fixation for hip fracture, with endpoints of mortality and re-operation: patients, clinicians, and outcome assessors not blinded. • Arthroscopy vs. non-operative management of shoulder dislocation, with endpoints of re-dislocation, and WOSI*: patients not blinded, but clinicians and outcome assessors (therapists) were blinded. *Western Ontario Shoulder Disability Index **Moseley JB, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002;347(2):81-88.

  16. Between-groups Comparison Nissen SE, Tsunoda T, Tuzcu EM, Schoenhagen P, Cooper CJ, Yasin M, et al. Effect of recombinant ApoA-I Milano on coronary atherosclerosis in patients with acute coronary syndromes: a randomized controlled trial. Jama 2003;290(17):2292-2300.

  17. Sub-group Analysis: ISIS II* 30-day mortality *Lancet 1988;2(8607):349-360.

  18. DONE: RCT Checklist for Study Validity* • Patient-Oriented Effect Measures (POEMs) vs. surrogate outcomes • Randomization to address issues of confounding • Intention-to-treat analysis (once randomized always analyzed) • Good follow-up to eliminate differential losses to follow-up • Blinding of patients and clinicians to prevent differential co-interventions • Blinding of outcome assessors to prevent bias • Between groups rather than within groups comparison • Compare entire randomization groups, not subgroups *For checklist on study validity, see Chapter 1B1 “Therapy”, in Guyatt and Rennie (eds.), Users Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; AMA Press; 2002. (Or try http://www.cche.net/usersguides/therapy.asp#Valid )

  19. Effect Size (Dichotomous Outcomes) RR RRR ARR NNT ARI NNH

  20. Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. This study was properly randomized but not blinded, used an intention-to-treat analysis, and had NO losses to follow-up. Results follow…

  21. Reduced Re-operation

  22. Measures of Treatment Effect RR= Relative Risk or Risk Ratio = RR < 1 tx is associated with decreased risk, as is the usually the case for a primary endpoint. RR>1 means tx is associated with increased risk, as is usually the case for a side effect. RRR = Relative Risk Reduction = 1-RR

  23. Beware of the Odds Ratio RR= Relative Risk or Risk Ratio = (a/b) (a/c) OR = Odds Ratio = ------- = -------- = ad/bc (c/d) (b/d)

  24. Measures of Treatment Effect ARR = Absolute Risk Reduction = c/(c+d) - a/(a+b) NNT = Number Needed to Treat (to prevent 1 bad outcome) = 1/ARR

  25. NNT Practice In patients < 30 years old with first-time acute anterior shoulder dislocation, prompt arthroscopic surgery (vs. standard conservative therapy) reduces the 2-year re-dislocation rate by almost 33% in absolute terms (from about 50% to about 17%).* How many first-time dislocation patients do we need to treat with arthroscopy to prevent one having re-dislocation at 2 years?

  26. Problem with the Relative Risk The relative risk (RR) or relative risk reduction (RRR = 1-RR) associated with a treatment is of minimal use without knowing the baseline level of risk*. *The RR is not completely useless without the baseline risk. If RR=1, the tx is useless regardless of the baseline risk. If RR << 1, then the treatment is beneficial; if RR >> 1, the treatment is harmful. Also, if you already know the baseline risk in your own population, the RR may be all you need.

  27. Problem with the Relative Risk The selective estrogen receptor modulator raloxifene (Evista®) at a dose of 60 mg /d for 3 years reduces vertebral fracture risk by 33% in women with osteoporosis.* How many women with osteoporosis do we need to treat with raloxifene to prevent a vertebral fracture? *JAMA. 1999 Aug 18;282(7):637-45. Numbers rounded for exposition.

  28. Problem with the Relative RiskNeed Baseline Risk The selective estrogen receptor modulator raloxifene (Evista®) at a dose of 60 mg /d for 3 years reduces vertebral fracture risk by 33% in women with osteoporosis. Baseline 3-year risk of vertebral fracture = 10% How many women with osteoporosis do we need to treat with raloxifene to prevent a vertebral fracture?

  29. Problem with the Relative RiskNeed Baseline Risk Baseline 3-year risk of vertebral fracture = 10% RRR = 0.33 ARR = 0.1 x 0.33 = .033 NNT = 1/0.033 = 30 Need to treat 30 osteoporotic women with raloxifene for 3 years to prevent a vertebral fracture.

  30. Effect of Flu Vaccination on All-Cause Mortality During the Flu Season The study population included almost 300,000 subjects at least 65 years old, of whom about 58% were vaccinated. Among vaccinated and unvaccinated subjects, 1.2% and 2.0% respectively died during the flu season. Pooled computerized data from 3 large managed care organizations. Nichol et al. N Engl J Med. Apr 3 2003;348(14):1322-1332. This was NOT an RCT, lots of confounding to address, but it’s related to the flu.

  31. No Flu Shot? How about Tamiflu?

  32. Mechanism of Action of Neuraminidase Inhibitors Moscona A. Neuraminidase inhibitors for influenza. N Engl J Med 2005;353(13):1363-73.

  33. Flu Prophylaxis? • Pt is a 14-year-old girl with fever, myalgias, cough and sore throat X 1 day • Should you rx prophylactic Tamiflu® for the pt’s grandparents (in their 70s) who live in the same household and didn’t get the flu shot this year? Pubmed search  Welliver R et al. Effectiveness of Oseltamivir in Preventing Influenza in Household Contacts: A Randomized Controlled Trial. JAMA 2001; 285:748-754.

  34. Prophylactic Oseltamivir: Index Case Flu+* *Welliver R et al. Effectiveness of Oseltamivir in Preventing Influenza in Household Contacts: A Randomized Controlled Trial. JAMA 2001; 285:748-754.

  35. Prophylactic Oseltamivir: Index Case Flu+* *Welliver R et al. Effectiveness of Oseltamivir in Preventing Influenza in Household Contacts: A Randomized Controlled Trial. JAMA 2001; 285:748-754.

  36. Number Needed To Harm NNH is really number needed to treat to cause one undesired effect.

  37. Number Needed to Harm Not an apt term for number needed to treat to cause one bad outcome. Would prefer NNTc (“Number Needed to Treat to cause”) vs. NNTp (“Number Needed to Treat to prevent”), but NNH is well established.

  38. Ratio of Undesired to Desired Effects “Harms” / Bad Outcome Prevented = ARI/ARR = NNT/NNH Or Bad Outcomes Prevented / Harm Caused = ARR/ARI = NNH/NNT

  39. Number Needed to Harm

  40. Number Needed to Harm

  41. Ratio of Desired to Undesired Effects Bad Outcomes Prevented / Harm Caused = ARR/ARI = NNH/NNT* Arthroplasty vs. Internal Fixation for Hip Fx Risk Difference for re-operation: ∆ Risk Re-Op = 5.2% - 39.8% = -34.6% Risk Difference for transfusion: ∆ Risk Trx = 19.7% - 1.8% = +17.9% Re-operations prevented/Transfusion Caused: -34.6/17.9 = -1.93 ≈ -2 *Easier here to divide ∆ re-operation by ∆ transfusion, rather than use NNH or NNT.

  42. Ratio of Undesired to Desired Effects Cases of Nausea / Flu Case Prevented = 2.9%/ 11% = 0.25 Or Flu Cases Prevented / Nausea Caused = 11%/2.9% = 4

  43. BOTE CEA Back-of-the-Envelope Cost Effectiveness Analysis

  44. Back-of-the-Envelope Cost Effectiveness Analysis How many patients do I need to treat (at the treatment cost) to prevent 1 bad outcome? Number Needed to Treat (NNT) = 1/ARR Cost of preventing one bad outcome = NNT x Treatment Cost* *This is just ∆$Cost /∆Risk .

  45. BOTE CEA: Oseltamivir Index Case Flu + • NNT = 9 (Treat 9 household contacts, prevent 1 flu case) • NNT x Treatment Cost* = 9 x $35 = $315/flu case prevented • Cost of Tamiflu 75 mg #10 = $59.99 www.drugstore.com 3/4/2004 • $79.99 10/25/06

  46. BOTE CEA Example Raloxifene vs. placebo in women with osteoporosis

  47. Raloxifene vs. Placebo Raloxifene (Evista®) 60 mg/d x 30 d = $87* 36 months of treatment = 36 x $87 = $3132 Need to treat 30 patients to prevent 1 fx 30 x $3132 ≈ $93,960 per vertebral fx prevented. *Drugstore.com 10/25/06

More Related