Evidence-based Practice Centers

1. Evidence-based Practice Centers • Created in 1997; now 13 centers • Produce • “evidence reports” • systematic reviews • technology assessments • “rapid reviews” • meta-analyses and cost analyses • analysis of large databases • Work with public and private sector partners

2. Evidence-based Medicine Mark Helfand, MD Director Oregon Evidence-based Practice Center

3. What is the kind and strength of the evidence you are relying on to make a recommendation? The Question:

4. What does evidence-based mean? • A comprehensive, systematic, open minded review of all the evidence • The evidence determines the conclusion, not vice versa • Not, the citation of papers supporting a preformed conclusion (and trashing of those that don’t) • Not, the use of evidence when it is ‘positive’ but judgement when it isn’t

5. Systematic literature reviews • Are systematic to remove bias in finding and reviewing the literature.

6. Systematic literature reviews • Are systematic to remove bias in finding and reviewing the literature. • Experts may interpret the data (and their own experience) differently.

7. How sure are we?Expert estimates of breast implant rupture rates 0% 0.2% 0.5% 1% 1% 1% 1.5% 2% 3% 3% 4% 5% 5% 5% 5% 5% 5% 5% 5% 6% 6% 6% 8% 10% 10% 10% 10% 13% 13% 15% 15% 18% 20% 20% 20% 25% 25% 25% 30% 30% 40% 50% 50% 50% 62% 70% 73% 75% 75% 75% 75% 80% 80% 80% 80% 80% 80% 100% Source: Dr. David Eddy

8. 0% 25% 50% 75% 100% Experts estimates of the effect of colon cancer screening on chance of dying Source: Dr. David Eddy

9. Experts’ estimates of probability of acute retention in men with BPH Source: Dr. David Eddy

10. Systematic literature reviews • Are systematic to remove bias in finding and reviewing the literature. • Studies with disappointing results may get less attention

11. *Excludes 5 mg bid group

12. Trial 114

13. Systematic literature reviews • Are systematic to remove bias in finding and reviewing the literature. • Experts may underplay controversy or select only supportive evidence

14. Simpson et al, 2004

15. Simpson et al, 2004

17. In a double-blind study vs risperidone… GEODON sustained control of positive symptoms at 1 year 1

18. In a double-blind study vs risperidone… GEODON sustained control of positive symptoms at 1 year 1

19. Systematic literature reviews • Are systematic to remove bias in finding and reviewing the literature. • Experts may underplay controversy or select only supportive evidence • Emphasize the best evidence

20. The best evidence • Reflects patients’ concerns • By addressing health outcomes patients, their caregivers, and families care about

21. The best evidence • Reflects patients’ concerns • By addressing health outcomes patients, their caregivers, and families care about • Help you feel similar to other people • Help you feel less lonely and removed from others • Help you feel more hopeful and happy • Allow you to think and express yourself more clearly

22. Selecting questions • Researchers often use their own curiosity or research interest as the basis for selecting questions. • They often use “standard” scales and measures instead of seeking a deeper understand of the patient’s well-being and quality of life.

23. Selecting questions • Our premise is that important questions arise from practice, and from life. “Experts in practice”--and patients--select the populations, interventions, and outcome measures of interest.

24. The best evidence • Reflects patients’ concerns • By addressing health outcomes patients, their caregivers, and families care about • By using simple measures of benefit and risk

25. Example

26. Why use systematic literature reviews? • Define the strengths and limits of the evidence. • Clarify what is based on evidence and what is based on other grounds. • Do not necessarily tell you what to do when the evidence is limited. Other factors, such as equity, clinical judgment, values, and preferences play a role in using the evidence.

27. Rules for linking evidence to recommendations Systematic Reviews + + local judgments and values = Evidence-based decision-making

28. An evidence-based decision process • Makes use of an independent, systematic review of the evidence • Employs rules for linking evidence to recommendations • Produce explicit, defensible recommendations

29. Oregon ApproachWhat are we after? • Systematic drug-class reviews should address questions that reflect clinicians’ and patients’ concerns. • Decision-makers should begin to wrestle with the idea of what is good evidence. • Manufacturers should gain market share if they produce good evidence of superiority over other drugs in a class. • Patients, caregivers, payers (and NAMI) should demand better evidence about outcomes that matter !

31. Drug Class Review on Atypical Antipsychotics

32. Included Drugs Clozapine not posted risperidone (1993) not posted olanzapine (1996) not posted quetiapine (1997) not posted ziprasidone (2001) posted aripiprazole (2002) posted

33. Eligible Outcomes

34. Results • 196 studies included overall • 33 head-to-head • 24 placebo-controlled • 58 active controlled • 63 observational studies • 18 systematic reviews • 427 study publications excluded

35. SchizophreniaHead to Head Trials • 3 Effectiveness Trials • 12 month pragmatic trial of olanzapine, risperidone or continuing typical AP • One 12-month switching study of olanzapine & risperidone • InterSept trial of clozapine and olanzapine to prevent suicidality found clozapine superior • 30 Efficacy Trials

36. Head to head trials in outpatients

37. Summary: Benefits • Clozapine, olanzapine and risperidone had similar efficacy with two exceptions • Clozapine > olanzapine in suicidality/suicide prevention • Olanzapine > risperidone in reducing rates of relapse • Aripiprazole, quetiapine, and ziprasidone: Evidence too limited to say

38. Summary: Harms • Weight gain • Greater risk for olanzapine than risperidone • Results mixed in long-term observational studies • Diabetes mellitus • Risk greater for olanzapine than risperidone, but studies had mixed results • Risk with clozapine relative to others not clear • Limited evidence on quetiapine • Other long-term safety • No conclusions about comparative safety can be made

39. Other harms • Movement disorders • Somnolence • Hyperprolactinemia/sexual dysfunction • Long QT interval • Bone marrow problems

40. Outpatient studies Better head-to-head comparisons of antipsychotics are needed to discern the relative efficacy and safety profiles of these compounds.

41. What we can do together • select and refine questions that puts patients’ and caregivers’ concerns center stage • Rely on unbiased reviews to inform patients, families, and clinicians • Promote an evidence-based process, not just systematic reviews. • Promote higher standards for evidence about treatments for mental illnesses

42. Observational Studies: Long-term Safety • 48 studies,  6 months in duration • primarily schizophrenia patients • 8 head-to-head cohort studies • 10 AAP versus typical AP cohort studies • 29 descriptive epidemiologic studies • 1 case-control study • Death: Rates ranged from 0.1% to 3.3% for clozapine, quetiapine and risperidone (7 uncontrolled studies)

43. Criticism • “By adhering to rigorous rules of inclusion, the process maximizes the validity of assessing proven treatment efficacy (strength), while it ignores or discards other germane but less statistically rigorous evidence of real-world effectiveness and cost-effectiveness (weakness).

44. Our response • We agree controlled trials ignore important aspects of effectiveness…

45. Limitations of RCTs • There aren’t enough of them. • They test interventions that may or may not fit easily into practice. • They often don’t tell you about important subgroups. • They may not extend for a long time.

46. More limitations of RCTs • Design features are poorly adapted to the purpose of assessing average effectiveness • Populations • run-in periods • Exclusions • Comparators and comparisons • Outcome measures • Followup period • Feasibility • Implementation costs • Maintenance costs

47. Most common problems with head-to-head trials • Doses of the different drugs aren’t equivalent. • Strategies for using the drugs aren’t realistic. • Usually, focus on efficacy or harms but not on both • Do not address all important outcomes

48. RCTs & harms • Design features are poorly adapted to the purpose of assessing harms • run-in periods • exclusions of susceptible people • Reporting is poor • unreported • Selectively reported • Misleadingly reported • Lack of severity data

49. Applicability: How to bias an efficacy study and stillget a “good-quality” rating • select compliant patients • dilute the control group interventions • measure only certain outcomes • cheat • selective use of cut-off dates • what are the norms?

50. We agree controlled trials ignore important aspects of effectiveness… • and agree on what information we’d like to have.