New Era Of Comparative Effectiveness Research

1. New Era Of Comparative Effectiveness Research Hyatt Regency Capitol Hill Washington, D.C. October 5, 2010

2. With thanks to:

3. An Evaluation Of Recent Federal Spending On Comparative Effectiveness Research Priorities, Gaps, & Next Steps Joshua S. Benner Marisa R. Morrison Erin K. Karnes S. Lawrence Kocot Mark McClellan

4. The Recovery Act provided unprecedented federal spending on CER—in a 2 year period *FY09/10: Based upon ARRA actual/projected obligation amounts **FY09/10: Based upon actual and projected spending amounts Sources: AHRQ: http://www.hhs.gov/recovery/reports/plans/pdf20100610/AHRQ%20CER%20June%202010.pdf; http://www.ahrq.gov/about/cj2011/cj2011.pdf; NIH: http://report.nih.gov/rcdc/categories/; Office of Secretary: http://www.hhs.gov/recovery/reports/plans/pdf20100610/OS%20CER%20June%202010.pdf

5. Not accounted for As of Aug 4, nearly all of the $1.1B was spent, but tracking it is challenging • Sources: • Recovery.gov • Grants.gov • FBO.gov • NIH Research Portfolio Online Reporting Tools Expenditures and Results (RePORTER) • NIH Research, Condition, and Disease Categorization (RCDC) system • AHRQ Grants On-Line Database (GOLD) • AHRQ website

6. More than half of funds were spent on improving capacity to do CER Recovery Act CER spending, by project type(As of Aug 4, 2010)

7. Recovery Act CER studies funded through NIH span the disease areas prioritized by IOM

8. Implications for future CER spending • Areas where more funding is needed • Experimental research • Evaluation of system-level reforms • Identification of effects in subgroups • Inclusion of understudied populations • Dissemination of results • Need timely, comprehensive, “dashboard” • Opportunities and ongoing projects • Specific aims, including priorities areas addressed • Methods and data sources • Budget amount, duration, and granting agency • Linked to CER Inventory

9. Creating A High Performance System For Comparative Effectiveness Research Lynn Etheredge Rapid Learning Project George Washington University

10. A Rapid Learning Health System • Designed to learn as rapidly as possible about the best treatment for each patient • Key concept: in silico research (on large computerized databases) • Complements in vitro and in vivo methods • Studies many more patients, richer (EHR) data, more researchers; more, different, faster studies • Researchers: multi-year data collection  log-on to world’s evidence base. Databases designed for CER

11. A High-Performance CER System • Learning about new technologies • Registries, research plans, reporting • E.g. new cancer therapies, off-label Rx use, surgery • Comparability standards for studies • To accelerate comparative research, data networking • A National Database For Effectiveness Research Studies • Clinical studies submit standardized data-sets, a growing world “evidence base” for CER • Established by Presidential order

12. A High-Performance CER System • A national network of clinical research registries and databases • Fill gaps in the evidence base for elderly, children, pregnant women, persons with multiple chronic conditions, minorities, rare diseases, surgery • Getting results • HHS public work plans, w/ due dates, accountability • Create a national rapid-learning culture for CER, w/ websites, networks, rapid-learning communities

13. Comparative Effectiveness And Personalized Medicine: Evolving Together or Apart? Robert S. Epstein J Russell Teagarden

14. The Worry • Comparative Effectiveness can help determine the average effects between alternative treatments • But this may mask results from important sub-populations (e.g. Personalized Medicine)

15. There is some evidence to support this concern • A recent random sample of 5 major journals and 319 trials – revealed: • Only 29% of the studies tested comparative effects within sub-populations Source: Gabler NB et al: Dealing with heterogeneity of treatment effects: is the literature up to the challenge? Trials 2009;10:43.

16. Comparative trial showed 16% reduction in CV events if on high dose statin vs. std. dose at 2 years Source: Cannon CP et al: Intensive versus moderate lipid lowering with statins after acute coronary syndromes. NEJM 2004;350(15):1495-1504.

17. Personalized medicine finding:40% CV drop in genetic sub-population! Source: Iakoubova OA et al: Polymorphism in KIF6 gene and benefit from statins after acute coronary syndromes: results from the PROVE-IT TIMI 22 study. JACC 2008:29;456-8.

18. Comparative effectiveness can also provide evidence of clinical utility Source: Mallal S et al: HLA-B*5701 screening for hypersensitivity to abacavir. NEJM 2008;358:568-79.

19. Conclusions • Comparative effectiveness can • Uncover findings not available elsewhere due to studying diverse patient populations • Provide evidence of utility in the ‘real world’ • Must remember to structure CE studies with broad entry criteria and include genetics – to help each discipline grow together

20. A Patient Advocate’s Perspective On Patient-Centered Comparative Effectiveness Research Tony Coelho Former Chair, Board of Directors, Epilepsy Foundation of America

21. A Flexible Approach To Evidentiary Standards For Comparative Effectiveness Research Louis P. Garrison, Jr. Department of Pharmacy, University of Washington, Seattle WA Peter J. Neumann Center for the Evaluation of Value and Risk in Health, Tufts Medical Center, Boston, MA Paul Radensky McDermott Will & Emery LLP, Washington, D.C. Sheila D. Walcoff McDermott Will & Emery LLP, Washington, D.C. Research support: National Pharmaceutical Council

22. Patient-Centered Outcomes Research Institute (PCORI)--Responsibilities • Setting priorities • Developing methodological standards • Communicating research results to decision makers Question: How can the institute support the different standards for acceptable evidence used by various government agencies, providers, patients, and other decision makers?

23. Evidentiary Standards Rules that decision makers apply in using different types of evidence to carry out their mission Evidentiary standards vary among decision makers: • FDA for new drugs: “substantial evidence” • FDA for devices: “reasonable assurance” • FDA for promotion: “substantial clinical experience” • CMS: “reasonable and necessary” • FTC: “Competent and reliable scientific evidence”

24. Comparative Effectiveness Research in the Decision Maker-Evidence-Policy Loop

25. Key Issues And Implications “ • The information produced through CER will be a “public good”, benefiting all decision makers. • Models are a valuable tool for synthesizing data. • Value-of-information analysis can help PCORI with research prioritization. “Experiment, observation, and mathematics, individually and collectively, have a crucial role in providing the evidential basis for modern therapeutics. Arguments about the relative importance of each are an unnecessary distraction. Hierarchies of evidence should be replaced by accepting—indeed embracing—a diversity of approaches.” Sir Michael Rawlins, NICE

26. Methodological Standards • Best practice for a given type of methodology • Calls for a “translation table” to link methods to research questions • ACA: “provide specific criteria for internal validity, generalizability, feasibility, and timeliness of research” There is a need for balance—and tradeoffs—among these objectives.

27. Conclusion • PCORI should take a balanced and flexible approach to the types of research it sponsors, being careful not to let rigorous scientific methods become a rigid evidentiary standard.

28. Designing Comparative Effectiveness Research On Prescription Drugs:Lessons From The Clinical Trial Literature Aaron S. Kesselheim, M.D., J.D., M.P.H. Co-authors: Dave A. Chokshi, M.D., M.Sc., Jerry Avorn, M.D. Division of Pharmacoepidemiology and Pharmacoeconomics Brigham and Women’s Hospital, Harvard Medical School, Boston MA

29. CER Not A New Phenomenon • CER existed before recent developments • Government and commercial funding • Review of some examples of drug CER • Goal: Identify methodological issues • 3 areas of special concern: • Choice of comparison treatments • Time frame • External validity Sources: Hochman & McCormick, JAMA, 2010; Clement et al., JAMA, 2009

30. Designing Better CER Trials • Choice of comparator • The “straw man” problem • Ex. #1: Comparator doesn’t represent “standard of care” in efficacy trial (ASCOT) • Ex. #2: Comparator with high side effect risk in safety trial (MEDAL) • Study time frame • The “surrogate endpoint” problem • Ex. #1: Surrogate endpoint not valid (DCCT) • Ex. #2: Surrogate endpoints may mask safety problems (RECORD, torcetrapib) Sources: Dahlof et al., Lancet, 2005; Cannon et al., JAMA, 2006; DCCT Research Group, NEJM, 1993; Home et al., Lancet, 2009; Psaty & Lumley, JAMA, 2008

31. Designing Better CER Trials (cont’d) • External validity • How are CER findings applied? (If at all!) • Ex. #1: Trial design doesn’t anticipate real-world use (RALES) • Ex. #2: Non-representative enrollment • Ex. #3: Inadequate effort to change prescribing practices (ALLHAT) • Ex. #4: Failure to account for critical pushback (CATIE) Sources: Juurlink et al., NEJM, 2004; Jagsi et al., Cancer, 2009; ALLHAT Research Group, JAMA, 2002; Lieberman et al., NEJM, 2005

32. Moving Forward • Options for the new era of CER in US • Innovative approaches to study methods • Ex. #1: Adaptive trial design (STAR-D) • Alternatives to randomized controlled trials • Observational studies • Meta-analyses and systematic reviews Sources: Rush et al., NEJM, 2006; Nelson, JNCI, 2010; Lumley, Stat Med, 2002; Hlatky, Lancet, 2009

33. Policy Recommendations • CER must compare the best available alternatives • Use validated surrogate endpoints, but follow with hard endpoints and safety studies • Emphasize “pragmatic” clinical trials with ongoing reassessment • Implement observational studies with methodological rigor • Link CER to effective strategies for dissemination (academic detailing) • Market findings to health care institutions

34. Why Observational Studies Should Be Among The Tools Used In Comparative Effectiveness Research Nancy Dreyer Chief of Scientific Affairs, Outcome

35. 19 Evidence Is Needed Goal: Informed decision making about real-world practices and outcomes that result from that behavior or treatment • Seeking Evidence • Relevant to Patients and Providers • Valid • Feasible 19

36. The Framework • Decision-makers need to rely on a full range of high-quality comparative effectiveness research • Methods and challenges • Guidance for determining which type of study to employ • Whatever methods are used, research should be designed and implemented using best practices • Bodies of evidence • Systematic reviews, meta-analysis, grading systems • Individual study types • RCT, observational studies, other applications

37. Choosing The Right Fit • How much certainty is required? • Time and budget? • What are the comparators of interest? • Is there enough variability in treatment without randomization? • Is there a reasonable overlap of patient characteristics between treatments? • Treatment complexity? Single, multiple, etc.? • Would it be ethical and/or feasible to randomize?

38. Use Observational CE Studies When RCTs Are Impractical Or Infeasible (Examples) • Patients and conditions not typically included in RCT (frail elderly, children, pregnancies) • Multiple treatment paradigms (hearing loss) • Treatment adherence differs (asthma) • Off-label uses result in different pop’ns being treated (stents) • Operator training (ICDs) • Operator or institutional experience (surgical procedures) • Large study sizes needed (small differences)

39. The Most Informed Decisions Will Benefit From Considering The Full Range Of High Quality Research Optimal study design depends on purpose, validity and feasibility within constraints of time and budget

40. Adding The Patient Perspective To Comparative Effectiveness Research Albert Wu Professor, Health Policy and Management, Bloomberg School of Public Research, Johns Hopkins University

41. Some Questions Cannot Be Answered Without Asking The Patient • The main objective of much of health care is improving how patient feels and functions • Reduction in pain (hip replacement) • Improved functioning (cataract extraction) • Patient is best judge • Patient best observer of some events and health outcomes (complications)

42. Types of Information Captured In Data Used For CER

43. Linking Data Sources for CER Administrative Data Patient Reported Outcome Data Clinical Research and EHR Data

44. Electronic Health Records (EHR) • How can EHR be structured to include PRO • As clinical indicator that can be tracked, profiled • Useable for CER?

45. Johns Hopkins Medical Institutions EPR

46. Clinician Website Clinician can schedule a patient a survey

47. Patient Website Patient is presented with survey(s) to complete

48. Patient Website Patient survey – sample question

49. Clinician Website Clinician can view patients results

50. “Listen to the patient: He is telling you the diagnosis” - William Osler