Networks’ Experiences

1. Networks’ Experiences • Connecticut Center for Primary Care: ProHealth Physicians Practice Based Research Network (PI: John Lynch, MPH) • The Oregon Rural Practice-based Research Network (ORPRN) (PI: LJ Fagnan, MD) • CenTexNet (PI: Samuel N. Forjuoh, M.D., M.P.H., Dr.P.H) • LA Net (Lyndee Knox, PHD)

2. Connecticut The Connecticut Center for Primary Care (CCPC) is an independent 501c3 non-profit research and education foundation. On incorporation in 2002, CCPC assumed management of the ProHealth Physicians Practice Based Research Network, a PBRN affiliated with the AAFP Federation of Practice Based Research Networks.

3. PBRN Setting: ProHealth Physicians • CT’s largest primary care group practice • 75 practice sites throughout Hartford, Middlesex, Litchfield, Tolland, and New Haven Counties • (9 practice sites participated in MEADERS) • 221 primary care providers • 166 MD/DO, 55 APRN/PA (72 Family Practice, 73 Pediatric, 75 Internal Medicine, and 3 specialists) • Patient population 349,000 (approx 10% of CT) • Patient Contacts: over 750,000 encounters/yr

4. CCPC MEADERS 9 Study Sites • Pediatric – 9 clinicians • Pediatric – 6 clinicians • Family Practice – 7 clinicians • Family Practice – 2 clinicians • Internal Medicine – 4 clinicians • Internal Medicine – 3 clinicians • Internal Medicine – 3 clinicians • Internal Medicine – 2 clinician • Internal Medicine – 1 clinician

5. CCPC Results continued • Responses by type of event • 61.6% (n=77) Adverse drug events • 20.0% (n=25) Med Error – provider • 8.8% (n=11) Med error patient • 4.0% (n=5) Pharmacy • 5.6% (n=7)other (MCO, manufacturer, multiple, …)

6. CCPC Results continued • When happened: • 9.6% Today • 44.8% Past week • 26.4% Past month • 18.4% Over month ago • 0.8% Don’t know

7. Discussion: Experience • Recruiting – easy • 20 practices for 10 spots • Extended wait – OMB • Lost one site – left ProHealth • Sites didn’t remember they had signed up • System – Surprisingly easy/straightforward • Retention – consistency by week

8. Discussion: Lessons • Pilot test quirks • Participants say they completed many more forms • Form utility • Comfort reporting adverse drug events • Patient/pharmacist/MCO failed to… • Blame culture – CQI opportunity • Need for better education/support • Legal concerns • Need for Patient Safety Organization

9. The Oregon Rural Practice-based Research Network (ORPRN)

10. Participating ORPRN Clinics Enterprise Scappoose Baker City The Dalles John Day

11. Participant Job Categories

12. Reporting Facilitators Adequate Training Reminders & Feedback Weekly “Chalk Talk” reports Access to online error report page Occasional “fun” reminders MEADERS Desktop Icon User-friendly Program Active engagement of lead clinician & staff Did you ask about patient medications?

13. Qualitative Findings in ORPRN 1) Rural Family Practice Clinicians and office staff will report to MEADERS as evidenced during a 10 week reporting trial “It was painless.” “It went well. All participated and seemed to enjoy it. It did not take an excessive amount of time.” “People in the practice picked up on the type of events to report quite quickly and required little direction. Melinda helped a lot too.”

14. Qualitative Findings in ORPRN 2) Barriers to reporting included: making the time to report, breadth of reporting options, and confusion regarding what to report. “It was more difficult for the staff it seemed because it was one more thing that they had to do.” “I am assuming that some errors did not get reported because the criteria were too broad and clinicians didn’t think that they were important.”

15. Qualitative Findings in ORPRN 3) Some practices used MEADERS data for internal Quality Improvement “Overall, I think that the MEADERS reporting system was helpful to staff and providers to remind us that errors occur and we should be mindful of them and work to prevent them in the future…my MA and I worked especially hard with one patient who kept taking the wrong dose of her Coumadin… Her care improved because of MEADERS.” “The biggest thing it changed was making us more attentive to documentation and keeping med lists up to date ‑ especially between institutions like LTC and hospitals and hospices.”

16. Qualitative Findings in ORPRN 4) Some clinics report a desire to continue using MEADERS while others do not. “It was easy to use, we would like to continue using it in our practice.” “I doubt the practice would be excited about reporting events on a regular basis…This will be a significant obstacle to over come if this or a similar system becomes required in clinical practice.”

17. ORPRN MEADERS Conclusion “The overall thing is that the direct access made the process much easier - most of our staff had never made any reports regarding adverse drug reactions so in my eyes - if the goal is to improve the process by which reports are made - this is a huge improvement.” Family Physician, Clinic 2

19. Waen Gatesville Temple Killeen Taylor CenTexNet Practices 28 Scott & White Regional Clinics in 22 Locations + Brazos Family Medicine Residency Clinic, Bryan + Family Practice Residency Clinic, Waco

20. Follow-up Phone Call about Status of Survey Response • “Clinic has been really busy and just hasn’t had time to complete survey. Will try to get to it ASAP.” • “Clinic is short of nurses this week. Will try to get to it TODAY.” • “Contact has been out sick and just got back to work today. Will try to get to it.” • No responses from two clinics

21. Facilitators Quality & Safety Officers Clinic staff & MD champions Adequacy of training Part of network mission Staff familiarity with IT Barriers Time constraints Flu season Staff turnover Workflow conflict Facilitators & Barriers

22. LA Net www.lanetpbrn.net Reducing Health Care Disparities in Southern California Through Participatory Research Partnerships With the Health Care Community

23. LA Net - 5 FQHCs Clinica Msr. Oscar A. Romero Alvarado St. Clinica Msr. Oscar A. Romero Marengo St. QueensCare, Echo Park Cleaver Family Wellness Center East Valley Community Health Center West Covina East Valley Community Health Center Pomona

24. LA Net- Sites

25. Weekly Data Submissions by Clinic Note: QueensCare dropped out week 5 and Cleaver joined at week 6

26. Reporting Facilitators • Agreed w/ concept reporting/tracking is important • Liked MEADERS system • Easy to use • Fast - 20 seconds to enter • Said project increased their awareness

27. Reporting Barriers: Not about time • Culture: Administrative significance not enough-if not clinically significant, why report? • Ergonomics/Work flow • Interrupted paper-based work flow for clinicians • Dispensary uses computer but also interrupted process • Translation to QA processes not easy • Feedback function came on-line toward end • When did, not used in QI • Duplicates existing paper based ME systems for County and State (logs) • Likely will be a key motivator in future

28. Learning & recommendations (LA Net) • Need training & implementation supports for program that aid: • Identifying administrative value of detecting & reporting non-harm ADEs and MEs • Translation of data into existing QI processes • On-demand reports that meet existing reporting requirements • Multiple options for entering data: • Electronic (Electronic ICON integrated into EHR - No site had EHR or eRxing capabilities at time of study but all are either considering or in process of transition at present) • Paper (Paper to electronic)

29. Total Submissions by Week

30. Total Submissions by Network/Site

31. Post-reporting survey of the experience of the participants A survey was sent to the 220 clinicians and staff members who agreed to participate in the project Purpose: to learn about participants experience with MEADERS 164 completed the survey, a 75% response rate

32. Practice role of the 164 respondents

33. Demographics of 164 respondents

34. Demographics of 164 respondents

35. Training for MEADERS

36. Difficulty using MEADERS

37. Experience using MEADERS-1

38. Experience using MEADERS-2

39. Did you have any concerns about. . .

40. Forwarding event data to MedWatch

41. Under what circumstance might you have reported more events?

42. Did the study affect you personally?

43. Conclusions The MEADERS tool was used by practices and felt to be of value FDA MedWatch is rarely used by clinicians Reporting on medication errors is was not considered “safe” by many participants. They expressed concerns about repercussions of reporting and a few people were uncomfortable reporting on the mistakes of others Comments of participants note that MEADERS improved the quality of their care; they learned from mistakes and changed how they did their job Time is money: the sustainability of using a reporting tool such as MEADERS is unclear. Health systems might consider implementing MEADERS across their organization.

44. DISCUSSION