RTI International is a trade name of Research Triangle Institute

1. Overview of the AHRQ Project “Designing Consumer Reporting Systems for Patient Safety Events” Michael T. Halpern, M.D., Ph.D.Senior FellowDivision of Health Services and Social Policy ResearchSept. 14, 2009 RTI International is a trade name of Research Triangle Institute

2. Background • Since 1999, 25+ states have passed legislation or created regulations on hospital reporting of adverse events. • Many current event reporting systems are aimed at health care providers – don’t facilitate input from patients and their families on experiences with care. • Such input is needed - may facilitate detection of patterns of patient safety events, highlight system flaws, and reveal information not necessarily observed or reported by providers.

3. Consumer Reporting Systems for Patient Safety Events • Patients and family members are in a unique position to view continuum of care and identify gaps that may contribute to adverse events. • Systems including patient reports will complement information from health care providers collected through other reporting mechanisms such as PSOs. • Consumer reporting systems may improve understanding of medical errors and enhance methods for improving quality and preventing patient harm.

4. AHRQ Project: Designing Consumer Reporting Systems for Patient Safety Events • To develop recommendations for consumer reporting systems, AHRQ awarded a 2-year contract in Sept. 2008 to RTI International and Consumers Advancing Patient Safety (CAPS). • The project involves an iterative consensus-building process, including: • a technical expert panel; • consumer focus groups; • interviews with key stakeholders; and • an environment scan and literature review.

5. Project Framework – IDEALS Concept • Based on the “Ideal Design of Effective and Logical Systems” (IDEALS) design concept (Nadler, 1967). • In this concept, a recommended system evolves through three stages: • A theoretical system capturing a vision of an ideal system; • An ultimate ideal system built on the theoretical ideal system but contains achievable operational and practical goals; and • A technologically workable ideal system (TWIS).

6. Technical Expert Panel (TEP) • Purpose: to provide input on features of ideal systems, technically workable ideal systems, and recommended systems for consumer reporting systems for patient safety events. • Panel comprised of 18 individuals selected to include experts in patient safety, event reporting systems, health care delivery, quality improvement, patient-centered care, and patient advocacy. • Recommendations generated through Delphi questionnaires and TEP meetings.

7. IDEALS Concept

8. Delphi Questionnaires • Three rounds of questionnaires, with anonymous responses: • 1st round focused on 5 open-ended questions collecting information on functions, features, and characteristics of consumer reporting systems from an ideal perspective • Second and third round questionnaires asked participants to rate and rank suggestions provided from the round 1 questionnaire • Final round results described the ideal attributes of a consumer reporting system for patient safety events and formed the basis for the first TEP meeting.

9. First TEP Meeting • First meeting held at AHRQ headquarters June 24, to discuss questionnaire results, provide more detail on design features of ideal reporting systems, and form initial set of recommendations. • Meeting facilitated using Nominal Group Technique (NGT), which allows TEP members to generate items on priorities individually and then share and discuss these with other TEP members.

10. Preliminary Consensus Points from First TEP Meeting • Purpose of consumer reporting systems: improvement of health care and development of safer care • System ownership: independent, to avoid conflicts of interest • System governance: strong representation by consumers, but open to all stakeholders, using a commission model • System design: allowing submission of consumer reports by multiple modalities (e.g., phone, email, fax, etc.) and providing real-time (24/7) response capabilities • Level of operations: at multiple levels, from local to national or international

11. Second, Third, & Fourth TEP Meetings • Second TEP meeting (Dec. 2 & 3) will focus on parameters and consensus recommendations for TWIS, including potentially feasible approaches and barriers. • Third TEP meeting (webinar) and will focus on completion of TWIS specifications, final list of recommendations. • Recommendations from third meeting and the draft final report will undergo external review by at least three patient safety experts; recommendations will also be reviewed by consumer and caregiver focus groups. • Fourth TEP meeting (webinar) - TEP members will review feedback received from the external review process and develop the final list of recommendations.

12. Environmental Scan and Literature Review • Aim: explore technical feasibility and design elements for consumer reporting systems identified from Delphi questionnaires, first TEP meeting, and initial consumer focus groups. • Will focus on technically possible options for consumer reporting systems • Supporting elements • Strengths • Weaknesses • Potential challenges for implementation. • Includes interviews with 9 key informants

13. Focus Groups with Consumers • Six first-round focus groups will be conducted with patients and/or caregivers who have experienced near miss or adverse events. • Two focus groups held in Denver in June to provide ideas and input for developing consumer reporting systems. • Two more held in early September in Houston to provide additional ideas and inputs. Two additional focus group meetings scheduled for Boston.

14. Second Round Focus Groups with Consumers • Second-round focus groups will be held after second TEP meeting, to gather additional feedback on system design issues and recommendations from TEP. • Findings from focus groups will be shared with the TEP for incorporation in final recommendations.

15. Interviews with Key Stakeholders • 25 key stakeholders identified to participate in telephone interviews • Diverse group of individuals who can provide information on the design of consumer reporting systems for patient safety events. • 11 interviews completed to date. • Findings from interviews will be shared with TEP members without identifying sources.

16. Overview of Process Completion of Specifications of TWIS: 5. Interviews with Stakeholders 6. 2nd TEP Meeting (in-person) 7. 2nd-round Focus Groups with Consumers 8. 3rd TEP Meeting (webinar) Technologically Workable Ideal Systems (TWIS): 2. 1st TEP Meeting (in-person) 3. Focus Groups with Consumers 4. Environmental Scan and Literature Review Recommended Systems: 9. External Peer Review 10. 4th TEP Meeting (webinar) Theoretical Ideal and Ultimate Ideal Systems: 1. Three-round Delphi Questionnaires

17. Dissemination Plans • An Information Dissemination Plan Report will be prepared that will present outreach strategies to disseminate the project’s key findings. • Key findings will be disseminated in: • Peer-reviewed journal articles; • News briefs in newsletters; • On web sites of major health care delivery organizations, patient advocacy organizations; • In presentations at conferences; • In brief publications in trade journal; and • In other media outreach activities.

18. Project Personnel • AHRQ: Linda Greenberg, James Battles • RTI: Michael Halpern, Amy Roussel, Linda Lux, Patrick Nerz, Katherine Treiman, Asta Sorensen • CAPS: Marty Hatlie, Sue Sheridan

19. For More Information .. • Please contact: • AHRQ: Linda G. Greenberg, PhD at:  Linda.greenberg@ahrq.hhs.gov • RTI: Michael T. Halpern, MD, PhD at: mhalpern@rti.org • Thank you very much!