1 / 24

Is it True? Evaluating Research about a Therapy

Is it True? Evaluating Research about a Therapy. Allen F. Shaughnessy, PharmD, MmedEd Tufts University School of Medicine Department of Family Medicine David C. Slawson, MD The University of Virginia, Department of Family Medicine. Task 3: Clarification.

carolsimon
Télécharger la présentation

Is it True? Evaluating Research about a Therapy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Is it True? Evaluating Research about a Therapy Allen F. Shaughnessy, PharmD, MmedEd Tufts University School of Medicine Department of Family Medicine David C. Slawson, MD The University of Virginia, Department of Family Medicine

  2. Task 3: Clarification • Now let’s say that the same patient has heard from a friend that there is a vitamin that will help prevent migraines. What study design could answer the question of whether there is a vitamin that is useful in preventing migraine headaches in this patient?

  3. Study Methodsto Answer This Question • Epidemiology: Patients taking a vitamin are less likely to have migraines • Pharmacology: Drug x affects cerebral vasculature in rat brain isolates • Case report: “It worked on one patient” • Case-series: “It worked on a bunch of patients” • Randomized controlled trial: 1/2 get drug, 1/2 placebo. No one knows who ‘til the end who took what

  4. The “Danger” of Natural History Studies • For each 1% decline in hemoglobin A1c, decrease in mortality, stroke, CHD, . . .. • Women taking HRT and CVD mortality.

  5. The “Danger” of Natural History Studies • Compliance Effect! • Known as Factor X

  6. Validity • Internal validity: How well was the study done? Do the results reflect the truth? • External validity: can I apply these results to MY patients?

  7. Was it arandomized controlled trial? Randomization is the best protection against being mislead

  8. The value of randomization • 32 controlled trials of anticoagulation in acute MI • Results by type of study: Chalmers TC, et al. N Engl J Med 1977;297:1091-6.

  9. Was allocation assignment “concealed”? Did investigators know to which group the potential subject would be assigned before enrolling them?

  10. Importance of concealed allocation Trials with unconcealed allocation consistently overestimate benefit by ~40% Schulz KF, Chalmers I, Hayes RJ, et al. JAMA 1995;273:408-12 Schulz KF, Grimes DA. Lancet 2002;359:614-18. Pildal J, et al. Int J Epidemiol 2007;36:847-857 Moher D, et al. Lancet 1998;352:609-13.

  11. Was allocation assignment “concealed”? • Concealed allocation  blinding • Blinding can occur without concealed allocation • Surfactant in the NICU • Allocation can be concealed in an unblinded study • PT vs surgery for knee DJD Moseley JB, O'Malley K, Petersen NJ, et al. N Engl J Med 2002; 347:81-8.

  12. Concealed Allocation Conducting a Study Potential Subjects Trial starts Actual Subjects Randomization Blinding, etc A B

  13. Importance of concealed assignment • Meta-analysis of trials evaluating screening mammography • In studies in which allocation wasn’t concealed • Higher SE status, education level in screened group • Age disparity (average 6 mo older in the unscreened group) • Richer, smarter, younger • Trials with concealed allocation = screening harmful! • No effect or increased mortality • 20% more mastectomies Lancet Jan 8, 2000; Oct 20, 2001

  14. Conducting a Study Concealed Allocation Mammography Study Sign-up Number Group Patient name 1 Mamm. 2 No Mamm 3 Mamm 4 No Mamm 5 No Mamm 6 Mamm 7 Mamm 8 No Mamm 9 Mamm 10 No Mamm 11 Mamm 12 Mamm 13 No mamm 14 No mamm 15 Mamm Sara Smith Potential Subjects Jill Jones Wendy Walsh Linda Lucky Linda Lucky Trial starts Actual Subjects Randomization Blinding, etc A B

  15. Technical Nitpicking? Could this really make a difference? • Cumulative database: ~500,000 women • Current policy is based on very small differences: • Deaths in unscreened women 902 • Deaths in screened women 837 • Death difference (of 456,349) 65! • Systematic bias is not “random error” for which meta-analysis can compensate

  16. Mundus Vult Decipi “The world wishes to be deceived” People would rather be deceived than have the truth cause anxiety Caleb Carr, Killing Time

  17. “YOU WANT ANSWERS??!!! “I WANT THE TRUTH!!” “YOU CAN’T HANDLE THE TRUTH!!” Jack Nicholson and Tom Cruise “A Few Good Men”

  18. Nonfebrile Seizure Incidence

  19. Were all the patients properly accounted for at its conclusion? • Complete follow-up? • “Intention to treat” analysis? • Patients are analyzed in the groups to which they are assigned • Attempts to reflect “real world” clinical situations in which not all patients follow treatment recommendations • Watch when they compare only compliers with compliers and non-compliers • Compliant subjects always do better overall

  20. Was study “double-blinded”? • Did the patients know to which group they were assigned? • Did the treating physician know? • Did investigators assessing outcomes know (“triple-blinding” – up to 7 levels!)? • Judicial assessor blind + allocation concealment = surgery RCTs Schulz KF. Ann Int Med 2002;136:254-9.

  21. Were intervention and control groups similar? • See Table 1 of most studies • Randomization is best way to avoid bias, though imbalances still can occur (especially if allocation was not concealed) • Small differences sometimes are important

  22. Task 4.

More Related