BioMedCom

1. EVIDEM BioMedCom A practical framework to support healthcare decisionmaking, assess quality of evidence, and define explicit data needs Evidence and Value: Impact on DEcisionMaking - A practical framework applying Multi-Criteria Decision Analysis to support healthcare decisionmaking Mireille M Goetghebeur PhD, Monika Wagner PhD, Hanane Khoury PhD, Randy Levitt PhD, Lonny J Erickson PhD, Donna Rindress PhD BioMedCom Consultants incMontreal, Canada

2. Background & Hypotheses • Healthcare decisionmaking is a complex process that requires simultaneous integration of numerous disparate types of evidence and value judgments • There is a need, nationally and internationally for a transparent access to evidence and values on which decisions are based for coverage/use of healthcare interventions. • Ad-hoc assessment of a healthcare intervention without breakdown into components may result in loss of information and possibly biased valuation • Healthcare decisionmaking could be facilitated by structuring evidence and value judgments on which it is based into a practical and transparent architecture • Transparency will enhance understanding of healthcare decisions

3. Objectives • Breakdown components of healthcare decisionmaking into practical instruments structuring and quantifying assessment of healthcare interventions to facilitate decisionmaking • Build an iceberg architecture to provide multiple layers of transparent access to components of decisionmaking Ad hoc assessment Burden of disease Budget impact Type of medical service Healthcare savings Improvement of efficacy Equity Cost-effectiveness Etc… • Ultimately, optimize health by best use of healthcare interventions

4. Decisionmaking process Scientific judgment Value judgment Intrinsic value of product Extrinsic value of product Quality of of evidence Improvement over existing treatment None Lower Major Higher Validity of efficacy trial Low Small High Large Type of population treated (equity) Low priority? High priority? Number of patients Etc… Efficacy Etc… Vulnerable vs productive patients • Based on international standards • Comprehensive analyses can be systematized • Not highly dependent on evaluator perspective • Similar value system • Requires value judgment • Dependent on evaluator perspective • No shared value system • Dependent on region/institution priorities BioMedCom BioMedCom Conceptual frameworkComponents of decisionmaking 1Tunis SR. Reflections on science, judgment and value in evidence-based decision making: a conversation with David Eddy. Health Affairs. 2007 (26): w500-w515; 2Eddy DM. Clinical decision-making: from theory to practice-Anatomy of a decision. JAMA 1990 (263):441-443. 1Tunis SR. Reflections on science, judgment and value in evidence-based decision making: a conversation with David Eddy. Health Affairs. 2007 (26): w500-w515; 2Eddy DM. Clinical decision-making: from theory to practice-Anatomy of a decision. JAMA 1990 (263):441-443.

5. Extrinsic value Intrinsic value of treatment Value Matrix Multi Criteria Decision Analysis (MCDA) Synthesized evidence Key info for each component of Value Matrix Fully referenced Quality of evidence Quality Matrix Criteria of quality International standards Decisionmaking body requirements BioMedCom BioMedCom EVIDEM – What?Practical iceberg architecture to support decisionmaking Decision Evidence Full text source documents (published, registries, proprietary)

6. T r a n s p a r e n t c o m m u n i c a t i o n t o o l BioMedCom EVIDEM – Who?Communication tool to connect stakeholders Decisionmakers EVIDEM investigators & experts Data producers

7. BioMedCom EVIDEM - Why?Sharing of value judgments and structured multilevel access to data More transparent and understandable Can be explored more easily Limit obstruction of thinking process by large amount of data Focus on value Sharing of weights & scores Transparent access to quality of data and methods Transparent multilayer access to evidence

8. Quality Matrix: quality of clinical evidence Synthesized clinical evidence References Evidence - Full text sources/links BioMedCom Proprietary data Meta- Analyses (Cochrane) Registries Published trials EVIDEM – How?Integrated process – example “Improvement of efficacy” Decision Extrinsic value Value Matrix (MCDA) Value Matrix component T3: Improvement of medical service - efficacy 2

9. EVIDEM tools – MCDA Value Matrix 15 components defining intrinsic value of treatment Quality Matrix Components identified from literature review and current decisionmaking processes, and selected to fulfill MCDA guidelines on clustering, completeness, redundancy, operationality & mutual independence (Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14.; National Economic Research Associated. Multi-criteria analysis manual 2005. http://www.communities.gov.uk/pub/252/MulticriteriaanalysismanualPDF1380Kb_id1142252.pdf Accessed Jan2007)

10. EVIDEM tools - Quality Matrix12 types of evidence, 3 criteria of quality • Structured based on criteria defining quality and on requirements from more than 20 decisionmaking bodies worldwide (INTERFACE database. 2000-2008. www.biomedcom.org); Assessment for each component based on instruments derived from quality standards (e.g., CONSORT, CHEC, STROBE, Siegel et al, • Mauskopf et al) as well as guidelines and processes from decisionmaking bodies

11. Pilot study in Canadian contextProof of concept using historical cases • Historical cases: 10 medicines (cardiovascular diseases, endocrinology, infectious diseases neurology, oncology, ophthalmology,) using data from literature review and manufacturer dossiers submitted to the Canadian Common Drug Review (CDR) and Québec Conseil du Médicament (CM) • QM scores: EVIDEM investigators; • VM weights and scores, and feedback on process: Canadian Value Panel composed of representative stakeholders across Canada (decisionmakers, specialists, generalists, nurses, pharmacists, health economists/epidemiologists) • Feasibility: Algorithm developed to operationalize each cell of the matrices; applicable to any therapeutic areas and jurisdiction • Practicality: 30 min on average to apply VM by stakeholders; about 250 hrs to build the structured package of fully traceable information (quality, synthesized format and full text access) and value scores • Feedback from panelist: Value of EVIDEMin structuring evidence, assessing strengths and weaknesses systematically, and sharing values and value scores • Limitations of pilot: limited access for panelists to underlying source data (electronic interface to be developed); extrinsic components not covered (to be explored)

12. Pilot study - Value MatrixValue assessment example for clusters 1 & 2 • Anchors defined from a societal perspective, i.e., optimizing health at the societal level • Score options translate to 0% (score of 0), 33% (score of 1), 66% (score of 2) and 100% (score of 3)

13. Pilot study - Value MatrixValue assessment example for cluster 3

14. Pilot study - Value MatrixValue assessment example for cluster 4

15. OTHER HEALTHCARE DECISIONS Healthcare intervention reimbursement decisions Collaboration axis Explicit data needs • Define explicit needs of decisionmakers • Establish reasonable requirements Application axis Retrospective Application axis Prospective EVIDEM framework • Validate process in various jurisdictions • Explore context of past decisions • Generate data on quality of evidence • Adapt framework to existing decisionmaking processes • Basis to develop extrinsic value components Collaboration axis Research planning • Planning tool for researchers and intervention developers to meet explicit needs • Develop methodology to generate data tailored to critical data needs BioMedCom BioMedCom Potential applicationsTransparent access to evidence and values

16. Future developments • Collaborative studies and iterative processes to explore the value of EVIDEM in context • Interactive web based architecture integrating evidence and value for various healthcare interventions • Expected outcome of a systematized and shareable approach for data access and value assessment is to optimize resources, decisions and health

17. Acknowledgments • EVIDEM Canadian Value panel: • Jean-Francois Bussières BPharm MSc MBA FCSHP, Director, Pharmaceutical Practice Research Unit, Dept of Pharmacy, CHU Sainte-Justine Research Center; Clinical Associate Professor, Faculty of Pharmacy, University of Montréal. • Benoit Cossette BPharm MSc, Pharmacist, Coordinator, Drug Disease Management Program, Fleurimont Hospital, Sherbrooke University Hospital Centre. • Doug Coyle PhD, Associate Professor, Faculty of Medicine, Epidemiology and Community Medicine, University of Ottawa. • Cheri Deal MD PhD FRCPC, Associate Professor and Program Director, Endocrine Service, CHU Sainte-Justine Research Center, University of Montréal. • RolandGrad MD CM MSc CCFP FCFP, Associate Professor, Department of Family Medicine, McGill University. • Christine Lee MD, FRCPC Assistant Professor, Director of the Microbiology Residency Program, Department of Pathology and Molecular Medicine, McMaster University. • Mitchell Levine MD FRCPC FISPE, Professor, Director of Centre for Evaluation of Medicines Department of Clinical Epidemiology & Biostatistics, Department of Medicine, McMaster University • Diane Lowden RN MSc, Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University. • John Mancini MD FRCP, Professor/Program Director Continuing Medical Education, Department of Cardiology, University of British Columbia. • Paul Oh MD FRCPC, Medical Director of the Cardiac Rehab and Secondary Prevention Program, Department of Medicine, Clinical Pharmacology and Toxicology, University of Toronto. • Genevieve Tousignant N MScN, Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University. • Wendy Ungar PhD, Senior Scientist, Child Health Evaluative Sciences, The Hospital for Sick Children; Associate Professor, Health Policy, Management, and Evaluation, University of Toronto. • Marie-Claude Vanier BPharm MSc, Clinical Associate Professor, Faculty of Pharmacy, University of Montréal; Pharmacist UMF-GMF Cité de la Santé de Laval. • Funded by an unrestricted research grant from Pfizer Canada

18. EVIDEM • Thank you EVIDEM tools are open access We welcome input and collaborations Please contact the EVIDEM Group @ www.evidem.org mireille_goetghebeur@biomedcom.org