Comparative Effectiveness Research: Background, Priorities, Methods

1. Comparative Effectiveness Research: Background, Priorities, Methods Sean Tunis MD, MSc July 10, 2009

2. http://www.cbo.gov/ftpdocs/89xx/doc8972/02-15-GeogHealth.pdf

3. Quality of Evidence for Guideline Recommendations in CV disease Robert Califf, IOM Meeting on Evidence-based Medicine, December 2007

4. Great Expectations • Promoted from CBO director to OMB director • “better information about the costs and benefits of different treatment options, combined with new incentive structures reflecting the information….is essential to putting the country on a sounder long-term fiscal path.” 4

5. The nature of “C” evidence • The paradox • “Available evidence is limited or poor quality” • 18,000 RCTs are published every year • The gaps, as seen by decision makers • Patients are highly selected • Research settings are not typical of community • Missing or incorrect comparators • Physiologic or surrogate outcomes, not function • Results are not available when decisions made

7. Committee’s Definition of CER The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.

8. Recommendation 6 The CER Program should fully involve consumers, patients, and caregivers in key aspects of CER, including strategic planning, priority setting, research proposal development, peer review, and dissemination.

9. Engaging stakeholders Recommendation 1: Conduct a systematic assessment of best practices for effective engagement of decision makers Recommendation 2: As a condition of receiving federal funding for any CER study, the investigators must form a stakeholder advisory committee whose function is based on findings of #1

10. AJRR 9/19 Meeting Participants • Shami Feinglass (CMS) • Mike Rapp (CMS) • Penny Mohr (CMS) • Tom Gross (FDA) • Susan Gardner (FDA) • Danica Marinac-Dabic (FDA) • Mark Melkerson (FDA) • Art Sedrakyan (AHRQ) • Jean Slutsky (AHRQ) • Nancy Foster (AHA) • Teresa Lee (Advamed) • Jeff Kang (Cigna) • Bob McDonough (Aetna) • Joshua Benner (Brookings) • David Brogan (Brookings) • Michael Gluck (CMTP) • Randee Kastner (CMTP) • Sean Tunis (CMTP) • Mark McClellan (Brookings) • Tony Rankin (AAOS) • John Callaghan (AAOS/Iowa) • Kevin Bozic (AAOS/UCSF) • David Lewallen (AAOS/Mayo) • Jeannie Kennedy (AAOS) • Kathryn Sale (AAOS) 1

11. Categories of CER Methods Systematic reviews of existing research Decision modeling, with or without cost information Retrospective analysis of existing clinical or administrative data Prospective non-experimental studies, including registries Experimental studies, including randomized clinical trials (RCTs)

12. All methods have a role Inevitable tradeoff between internal validity and feasibility, generalizability, cost, time The nature of the research question, and the decision maker will influence best practices Experimental studies will have a crucial role in CER, and there is need for improving design and implementation Non-experimental methods hold great promise, particularly as methods are refined and data infrastructure is improved

13. Debate on CCTA Coverage Payers/researchers want RCT with death/AMI outcome 20k patients, 2+ years of follow-up Non-experimental options rejected Vendors / clinicians existing evidence adequate for coverage Medicare final decision (March 2008) No adequately designed studies show improved outcomes “We believe large, well-designed prospective trials needed” Broad coverage by local contractors retained NHLBI currently reviewing 3 large RCTs No progress on shared evidentiary framework for CER

14. Recommendation 7 The CER Program should devote sufficient resources to research and innovation in the methods of CER, including the development of methodological guidance for CER study design such as the appropriate use of observational data and more informative, practical, and efficient clinical trials.

15. Baucus-Conrad PCOR Legislation • “Establish and maintain methodological standards for comparative clinical effectiveness research on major categories of interventions to prevent, diagnose, or treat a clinical condition or improve the delivery of care” 15

16. Effectiveness Guidance Documents Analogous to FDA-guidance Targeted to product developers, clinical researchers Recommendations for design of clinical studies to generate evidence that will provide “reasonable confidence” of improved health outcomes Multi-stakeholder advisory group, iterative draft and comment process EGDs under development by CMTP Gene expression profiling for breast cancer Treatments for chronic wounds Cardiac imaging Integrative medicine Radiation oncology

17. Pragmatic Pharmaceutical Trials • Optimize design of phase III trials to be more informative for post-FDA decision makers • Clarify patient, clinician, payer evidence needs • Identify critical regulatory, methodological financial, operational barriers • Develop PPCT guidance document • Industry, FDA, CMS, NICE, PBAC, CDR, Consumer’s Union, Medco, BSBCA, others 17

18. Contact Info • sean.tunis@cmtpnet.org • www.cmtpnet.org • 443-759-3116 (D) • 410-963-8876 (M)

19. Final Portfolio: 100 CER Priority Topics * Although this category was described as “Safety and Quality of Health Care” in the web-based questionnaire, the category was re-labeled by the committee as “Health Care Delivery System” to be more accurate.

21. Recommendation 5 The HHS Secretary should establish a mechanism—such as a coordinating advisory body— with the mandate to strategize, organize, monitor, evaluate and report on the implementation and impact of the CER Program.

22. Recommendation 6 The CER Program should fully involve consumers, patients, and caregivers in key aspects of CER, including strategic planning, priority setting, research proposal development, peer review, and dissemination.

23. Recommendation 6 cont’d • The CER Program should develop strategies to reach out to, engage, support, educate, and, as necessary prepare consumers, patients, and caregivers for leadership roles in these activities. • The CER Program should also encourage participation in CER in order to create a representative evidence base that could help identify health disparities and inform decisions by patients in special population groups.

24. Recommendation 8 The CER Program should help to develop large-scale, clinical and administrative data networks to facilitate better use of data and more efficient ways to collect new data to inform CER. • The CER Program should ensure that CER researchers and institutions consistently adhere to best practices to protect privacy and maintain security.

25. Recommendation 8 cont’d • The CER Program should support the development of methodologies for linking patient-level data from multiple sources. • The CER Program should encourage data holders to participate in CER and provide incentives for cooperation and maintaining data quality.

26. AHRQ Systematic Review: Tx of Clinically Localized Prostate Cancer • Limited evidence on relative safety and effectiveness of major treatment options • prostatectomy, brachytherapy, radiation, active surveillance • New technologies rapidly spreading without data • robotic surgery, proton beam • Rigorous trials needed to compare treatment options, especially for side effects

27. CMS Efforts to Improve Evidence • Category B IDE regulation (1996) • Cover routine costs of clinical trials (2000) • Coverage with evidence development (2003) • Promote pragmatic clinical trials (2003) • Priorities for Sec 1013 of MMA (2004) • MCAC becomes MedCAC (2005) • Ad hoc efforts to work with NIH 27

28. Tools and Strategies for CER • Methodological framework • Pragmatic clinical trials • Coverage with evidence development

29. CMTP Mission • Collaborative projects to promote generation of new evidence that it is more informative to patients, consumer clinicians, and payers. • Fewer C’s, more B’s and A’s • We don’t do CER studies; we develop methods, policies, and collaborations to make CER happen 29

30. CMTP Basics • Started Jan 2006 within HealthTech in SF • CHCF and Blue Shield California Foundation • Incorporated as 501c3 in Jan 2008 (Maryland) • 5 member governing board; 14 on advisory board • Funding • Founding members: Blue Shield California, Kaiser, United, Aetna NPC, Pfizer, Amgen, JNJ • Additional funds from foundations, government, professional societies • Staffing: 12 FTEs

31. Project Categories • Priorities for evidence development • Facilitated research • Total joint replacement registry • Proton beam vs IMRT in prostate cancer • Methods development / guidance • Gene expression, chronic wounds, cardiac imaging • “Pragmatic” Phase III pharmaceutical trials • Applied policy • Commercial payer CED • MedPAC project on Medicare CER and CED 31

32. Coverage with Evidence Development • Links payment to requirement for prospective data collection • Intent is to guide clinical research to address questions of interest to Medicare • Medicare must approve study design • Goal to support evidence and rapid access • Lower evidence threshold with commitment to generate better info later

33. Examples of Medicare CED • Lung volume reduction surgery • FDG-PET for suspected dementia • Implantable defibrillators • Off-label use of drugs for colorectal cancer • FDG-PET for oncology • Home testing for sleep apnea • Daily dialysis • Artificial heart 33

34. CED Lesson Learned • Timing: when coverage under review, may be too late for CED • Methods • Difficult to design studies in coverage context • Registries provide broader access; ?? validity • RCTs viewed as equivalent of non-coverage • Large simple trials may help, but few examples • Payers view as benefit expansion; Vendors opposite • Unclear how best to fund clinical and research costs

35. Banff Workshop on CED • Presentations from Scotland, UK, Australia, US, Ontario, British Columbia, Alberta • Everyone trying to solve the same policy problem – balancing access and evidence • Encountering similar pressures and compromises, repeating the same mistakes • Some hints about viable approaches • Will begin work on CED Field Guide for Payers and Policy Makers

36. Medicare Review of CCTA • EPC report from Duke (April 2006) • Limited evidence of clinical utility in any population • MedCAC mtg (May 2006) • “Uncertain confidence about existing evidence” • Broad local coverage of CCTA • Medicare draft policy in 12/07 proposed CED for CCTA in “adequate” studies

37. Debate on CCTA Coverage • Payers/researchers • want RCT with death/AMI outcome • 20k patients, 2+ years of follow-up • Non-experimental options rejected • Vendors / clinicians • existing evidence adequate for coverage • Medicare final decision (March 2008) • No adequately designed studies show improved outcomes • “We believe large, well-designed prospective trials needed” • Broad coverage by local contractors retained • NHLBI currently reviewing 3 large RCTs

38. Cardiac Imaging Think Tank • Discussed alternative methods for evaluating cardiac imaging • Co-sponsors: ICER, ACC, ACR, SCCT, ASNC • private payers, CMS, vendors, clinical researchers, consumers, AHRQ, VA, etc. • Goal: balance validity with feasibility • Current status • Meeting summary completed • Effectiveness Guidance Document on clinical utility of non-invasive cardiac imaging

39. Effectiveness Guidance Documents • Analogous to FDA-guidance • Targeted to product developers, clinical researchers • Recommendations for design of clinical studies to generate evidence that is adequate for decision making • “reasonable confidence” of improved health outcomes • Started from insights from systematic reviews • Multi-stakeholder advisory group, iterative draft and comment process • Ongoing work • Gene expression profiling for breast cancer • Treatments for chronic wounds • Cardiac imaging • Radiation oncology

40. Tools and Strategies for CER • Coverage with evidence development • Methodological guidance • Pragmatic clinical trials

41. The blank pragmatic–explanatory continuum indicator summary (PRECIS) “wheel.”

42. PRECIS Domains Illustrating the Extremes of Explanatory and Pragmatic Approaches to Each Domain

43. PRECIS Domains Illustrating the Extremes of Explanatory and Pragmatic Approaches to Each Domain

46. Pragmatic Clinical Trials Initiative • Optimize design of phase III trials to be more informative for post-FDA decision makers • Clarify patient, clinician, payer evidence needs • Identify critical regulatory, methodological financial, operational barriers • Develop PPCT guidance document • Industry, FDA, CMS, NICE, PBAC, CDR, Consumer’s Union, Medco, BSBCA, others 46

48. Contact Info • sean.tunis@cmtpnet.org • www.cmtpnet.org • 443-759-3116 (D) • 410-963-8876 (M)