Are you at risk?

2. Are you at risk? falling victim to misleading presentations as a result of conflict of interest of investigators Not at all A little Moderate risk High risk

3. Spinning • get in groups of 2 to 5 • write down three ways to avoid being mislead • 3 minutes

4. Conclusions and Funding, Als-Nielsen, JAMA 2003 • 25 Cochrane reviews, 370 RCTs • random selection of 167 of 1081 (issue 2, 2001) • excluded • 101 < 5 studies • 6 no concealment variability • 2 no binary outcome • 16 non-pharmacological • abstracted • sources of funding • effect on primary outcome • adverse effects • methodological quality (concealment, blinding)

5. Scale Used to Grade Conclusions in Trials 6 Points Experimental intervention highly preferred and should now be considered the standard intervention in all patients, or similar 5 Points Experimental intervention preferred to control, but further trials still indicated, experimental may be more costly, or similar 4 Points Experimental and control intervention about equal, but the experimental cheaper, easier to administer, or similar minor advantage 3 Points Experimental and control intervention about equal, but the control may be cheaper, easier to administer, or similar minor advantage 2 Points Control intervention preferred to experimental intervention, but experimental intervention might be promising under some circumstances, or similar 1 Point Control intervention highly preferred and should now be considered the standard intervention in all patients, or similar

6. Relation Between Funding Source and Conclusions in 370 Randomized Drug Trials Abbreviation: IQR, interquartile range. *Conclusions in trials were assessed by a 1-6 point scale. If the conclusions recommended the experimental drug as the treatment of choice without disclaimers, 6 points was assigned, and if not, 1-5 points was assigned. +p<.001, using Kruskal-Wallis test (medians) or X2 test (proportions)

7. Als-Nielson, JAMA, 2003 • bigger effect size, more likely stronger recommendation • blinding, more likely stronger recommendation • after adjustment, industry funding, odds ratio 5.3 (95% CI 2.0 to 14.4)

8. 1 Read methods and resultsbypass the discussion

9. Use of placebo when active comparator optimal Do you manage patients with type II diabetes? Is there any agent you would routinely suggest for patients with diabetic nephropathy? Angiotensin receptor blockers for diabetic nephropathy vs. ACE inhibitors Parving H-H et al. N Engl J Med 2001;345(12):870-878 Brenner BM N Engl J Med 2001;345(12):861-869 Lewis EJ et al. N Engl J Med 2001;345(12):851-60. Hostetter TH. N Engl J Med 2001;345(12):910-912

10. Incomparable comparators • 8 RCTS of 2nd generation neuroleptics vs. 20 mg/d haloperidol • fewer extrapyramidal symptoms • standard dose haloperidol < 12 mg • RCT paroxetine qd vs. amitriptyline bid • less daytime somnolence • standard amitriptyline qhs Safer J Nerv Ment Dis 2002;190(9):583-92. Geddes J et al BMJ 2000;321(7273):1371-6. Christiansen PE, et al. Acta Psychiatr Scand 1996;93(3):158-63

11. Beware faulty comparators 2

12. Irbesartan vs amlodipine in diabetic nephropathy • in comparison to amlodipine, irbesartan reduced the combined endpoint of all cause mortality, progression to end stage renal disease, and doubling of serum creatinine (RRR 20%, 95% CI 7.5% to 32%) • did irbesartan reduce all-cause mortality?

13. Risk reduction with irbesartan (vs. amlodipine) Doubling of creatinine concentration RRR 33% (16-47%) End-stage renal disease RRR 23% (-0.5-41%) All-cause mortality RRR -3% (-35-22%) Composite endpoint RRR 20% (7.5-32%) -40 -24 -8 8 24 40 56 RRR (95% CI) RCT 1715 DM 2 nephropathy, HTN irbesartan vs amlodipine NEJM 2001; 345: 851

14. What has gone wrong? • widely varying importance • biggest effect on least important • most important no effect • criteria for use of composite • similar patient-importance • similar effect

15. Beware composite endpoints 3

16. 5,269 patients with abnormal glucose tolerance test randomized to lifestyle advice, or advice + rosiglitazone followed for 3 to 5 years

21. What is the authors’ message? • rosiglitazone to prevent diabetes: • strong indication (for all) • weak indication (for some) • not indicated

22. Doctor, what do I gain by taking rosiglitazone? • Doc: less chance of diabetes • Pt: what happens if I get diabetes • Doc: you have to take a drug • Pt: the same drug I’m taking to prevent diabetes? • Doc: I could give you a drug with less problems • Pt: I’ll take a drug every day for 3 years to lower my risk of taking the same or a less toxic drug from 25 to 10%???

25. What is your view? • rosiglitazone to prevent diabetes: • strong indication (for all) • weak indication (for some) • not indicated • what has gone wrong here?

26. Other problematic surrogates • tests of cognitive function instead of function and behavior in Alzheimer’s • bone density instead of fractures in osteoporosis • oxygenation instead of mortality in ARDS • asymptomatic DVT instead of symptomatic thromboembolism • pulmonary function instead of exacerbations and qol in respiratory disease • lipids instead of CV events in atherosclerosis

27. Beware substitute endpoints 4

28. Five vs Four Courses of Therapy for Acute Myeloid Leukemia Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70

29. Five vs Four Courses of Therapy for Acute Myeloid Leukemia Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70

30. Five vs Four Courses of Therapy for Acute Myeloid Leukemia Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70

31. Five vs Four Courses of Therapy for Acute Myeloid Leukemia Wheatley K, Clayton D. Controlled Clinical Trials 2003;24:66-70

32. Stopping boundary True beneficial effect No effect

33. stop Stopping boundary True beneficial effect No effect Look after every patient or event

34. stop stop stop Stopping boundary True beneficial effect No effect Interim analyses every q patients or events

35. Beware trials stopped early 5

36. Users Guides to Spin read methods and results; bypass the discussion section beware faulty comparators beware composite endpoints beware of substitute endpoints beware trials stopped early

37. Wall St. Journal, November, 2006

38. Wall St. Journal, November, 2006

39. New York Times, May 9, 2007

41. Available at: www.mhprofessional.com/jama