Engaging Stakeholders, Improving Research: Stakeholder Engagement in the DEcIDE Program

1. Engaging Stakeholders, Improving Research: Stakeholder Engagement in the DEcIDE Program L. Ebony Boulware, MD MPH Associate Professor of Medicine Johns Hopkins University

2. Objectives • Identify potential challenges to Stakeholder engagement in CER protocol development • Discuss engagement strategies that involve Stakeholders in a meaningful way to enhance the quality and relevance of study outcomes

3. DEcIDE Network • Developing Evidence to Inform Decisions about Effectiveness • Collection of research centers that the Agency for Healthcare Research and Quality (AHRQ) created in 2005 • Centers gather new knowledge and information on specific treatments • Conducts studies on the outcomes, effectiveness, safety, and usefulness of medical treatments and services

4. From the perspective of the investigator: DEcIDENetwork • Research priorities established through previous Agency work • Engagement of Stakeholders and technical experts • Institute of Medicine CER priorities • Research opportunities competitively offered among centers • Proposal submission • Peer review • Contracted research • Aims, approach often pre-specified during contract negotiation

5. From the perspective of the investigator: DEcIDENetwork • Stakeholder engagement still required during research • Plan for engaging Stakeholders must be specified within research applications

6. Emphasis of Stakeholder engagement changes during DEcIDE projects • Identifying and prioritizing topics of CER • Framing the questions • Selecting the comparators, outcomes • Creating conceptual framework • Data collection • Analyze and interpret results • Translation • Dissemination Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA. 2012 Apr 18;307(15):1587-8. PubMed PMID: 22511684

7. What key challenges does DEcIDE research present? • Orient Stakeholders to established research • Established questions, comparators • Preliminarily established research methodology • Help Stakeholders understand their value • Identify ways Stakeholder activities can enhance existing research plans • Enhance relevance • Input on planned approaches with end user in mind

8. DEcIDE Patient Outcomes in End Stage Renal Disease (ESRD) Study • 3- year ARRA contract to compare effectiveness of three treatment strategies for patients with ESRD (hypertension treatments, dialysis timing, iron therapy) • First Stakeholder meeting held within first 4 months of funding (October 2010) • Identifying and contacting Stakeholders • Working with partner to strategize and plan (Center for Medical Technology Practice, CMTP) *Priorities established through IOM recommendations

9. Identifying Stakeholders (1)

10. Identifying Stakeholders (2)

11. Orienting Stakeholders to DEcIDE ESRD Patient Outcomes Project • Orient Stakeholders to project • Email brief executive summary • Brief telephone call • Prepare for meeting • Employed medical writer to distill 50 page research application into 10 page lay summary • Planned meeting agenda with help of CMTP

12. Preparing First Stakeholder Meeting • In-person event in Baltimore, MD • Mailed Stakeholders materials before meeting • Summary of project • Detailed research protocols • Expectations of stakeholder roles • Meeting agenda and structure (pre-assigned breakout group leaders) • Questions to be addressed during meeting • Began to prepare investigative team

13. Conducting Stakeholder Meeting • Professional moderator • Discussed stakeholder roles, importance • Project summaries from research team • Breakout group discussions (organized by Stakeholder type) • Key questions to frame research approach • Consider potential implications of positive or negative findings for decision-makers • Informal lunch withgroup discussion of breakout group findings • Review discussion points and promised follow up

14. Special Considerations • Room setup (Stakeholders in center, investigators out of the way) • Prior investigator preparation (i.e. more listening, less talking, less technical jargon) • Interactive breakout groups with investigators as scribes, gathering information but little input

15. Helping Stakeholders see Value of Input • Audio recorded and transcribed entire meeting • Created detailed summaries of key points discussed • Distributed summary of meeting to Stakeholders • Prepared for in-person follow up

16. Helping Stakeholders see Value of Input:Preparing for in-person follow up • Research team digested transcribed Stakeholder comments • Responded point-by-point to comments, modifying research protocol where appropriate • Enhancing outcomes to improve relevance to different decision-makers (e.g., quality of life) • Adjusting emphasis of approach (e.g., considering harms of therapies in addition to effectiveness)

17. Example Stakeholder Comments Project 1: Antihypertensive Medications • Theme 1: Examine relationship of blood pressure and volume control (fluid management) • Response: • Several analyses planned to examine relationships between volume management and blood pressure control • Studies to look at simultaneous changes in blood pressure, volume and blood pressure medications on outcomes

18. Example Stakeholder CommentsProject 2: Timing of Dialysis Initiation • Theme 1: Not enough information about pre-dialysis illness and events that led toward patients starting dialysis (e.g. sicker patients more likely to start earlier) • Response: Plan to use alternate datasets to assess pre-dialysis medical history • Pre-dialysis ESRD Medicare Claims (“Backcasted Data”) • Proposed use of other data from other cohorts with CKD (AASK/MDRD) to explore rates of progression toward ESRD • Exploring alternative datasets (e.g. Cleveland Clinic)

19. Example Stakeholder CommentsProject 3: Iron Management • Theme 1: Impossible to disentangle simultaneous use of erythropoiesis stimulating agents and iron. Both should be considered as primary exposures of interest • Response: • Will explore simultaneous use of the two medications longitudinally • Employ complex modeling strategies (marginal structural)

20. Year 2: In-Person Follow Up • Second in-person meeting • Reviewed Stakeholder feedback and our responses in document provided to them and during presentation • Sought additional feedback • Provided research updates (preliminary findings) • Obtained feedback on preliminary findings and • Asked Stakeholders for input on framing and dissemination strategies

21. Year 3: Extending Impact of Stakeholder Engagement • Planning third in-person meeting • Pre- meeting engagement to get ideas about extension of current research, other relevant research studies in ESRD • Review final preliminary contract research findings, obtain feedback • Focus on framing and dissemination of final projects

22. Final Planned Engagement • Plan to send Stakeholders summary of input over life of project • Provide Stakeholders with some final products • Obtain feedback on clarity of products • Additional avenues for dissemination • Thank Stakeholders for generous time investment • Encourage future engagement

23. Resource Investment • Administrative (emails, letters, meeting materials coordination of meetings, obtaining COI statements) (45 hours per meeting) • Medical writer (20 hours) • Investigator preparation (20-40 hours depends on size of project) • Investigator presence at meetings (8 hours for 6 investigators) • Securing venue for meetings (travel, refreshments) • Audio recording and transcribing meeting (40 hours)

24. Successful strategies • Early engagement of Stakeholders • Well-planned in-person meetings • Setting clear expectations of the Stakeholders • Demonstrating use of input from Stakeholders with report back • Continued engagement throughout project

25. Challenges we faced • Presenting a lot of scientific information to group of busy Stakeholders • Technical aspects difficult to convey • Sheer amount of information • Ambitious contract research schedule • Challenging to engage Stakeholders throughout year • Would have liked to have resources to engage more frequently • Much of the research happening in background while preparation for future meetings ongoing

26. Summary • Engagement of Stakeholders in contract research may require special consideration • Help Stakeholders see their relevance • Translate complex protocols for a broad range of audiences • Demonstrating utilization of Stakeholder input important • Advance planning critical to maximize output of Stakeholder groups • Partner with experts in Stakeholder engagement • Invest adequate resources and time to maximize yield