Selected Data from the CAP Q-Probes studies national benchmarking data base

1. Strategies to Reduce Errors in the Delivery of Health Care ServicesLessons from the College of American Pathologists Q-PROBES Program see www.davidnovis.com

2. Selected Data from the CAP Q-Probes studies national benchmarking data base • Q-PROBES: Data collection tools that define benchmarks of quality • Studies of best practices in Anatomic Pathology and Laboratory Medicine • Data applicable to all areas of health care delivery

3. Today’s Objectives • Present the role of Q-PROBES studies in benchmarking quality and determining best practices • Discuss Q-PROBES studies related to reducing errors in clinical practice • Discuss concept of “redundancy” as a strategy by which to reduce errors

4. College of American Pathologists Q-PROBES program • Description Q-PROBES and Q-TRACKS studies • Illustrative example • Use of studies in clinical practice

5. Anatomy of a Q-PROBES study • Define a measurable quality indicator • Determine performance benchmark by collecting data on indicator from a large number of health care providers • Simultaneously collect data characterizing clinical practices of study participants • Determine which clinical practices are associated with best performance

6. Q-PROBE Example: Monitoring vital signs of blood recipients • Audit of 16,500 transfusions in two studies representing 660 hospitals • 15-20% of patients had incomplete vital sign monitoring • Bottom preforming 10% of institutions: 1/3 to ½ of patients had incomplete vital sign monitoring • Practices associated with better performance: Use of checklists Formal instruction of transfusionists Routine audits of transfusions

7. Q-Probes: snapshots of practices and quality in the United States • Since 1989, over 140 studies conducted • Short duration(usually less than 4 months) • Large sample sizes: 100’s of participants in each study; 1000’s over life of program Data points in 10’s of thousands to million • Heterogeneous populations and practice environments

8. College of American Pathologist Q-TRACKS Program • Monitor quality indicator over long periods of time (years) • Compare rates of improvement with peers • Evaluate effects of interventions within institutions

9. Where to find Q-PROBE studies benchmarking data • Archives of Pathology • JAMA abstracts, Reviews • Other journals (Access PubMed) • www.cap.org • www.davidnovis.com

10. Q-PROBES studies of benchmarks in errors occurring in Pathology and Laboratory Medicine • Sources of information • Obstacles to measuring errors • Strategies for reducing errors • Examples of practices shown to reduce errors

11. Sources of Information for this presentation on errors • Novis DA. Detecting errors in Laboratory Medicine and Anatomic Pathology: Q- PROBES and Q-TRACKS programs. Clin Lab Med. 2004; 24:965-978 • Novis D. Routine review of surgical pathology cases to reduce diagnostic errors. Pathology Case Reviews 2005; 10:63-67 • Summary table on website table • References provided • www.davidnovis.com

12. Obstacles to Measuring Errors • Perspective: Defining errors often a matter of viewpoint • Frequency: Serious outcomes of errors are rare events Most studies of errors deal with intermediary processes

13. SIMPLE STRATEGIES LEARNED FROM Q-PROBES DATA • Prevent errors from occurring • Catch the mistakes when they do

14. Practice characteristics of top performers • 1. Establish protocols with contingencies • 2. Reduce processes to as few steps as possible. • 3. Educate and motivate workers • 4. Identify patients/specimens meticulously • 5. Monitor the process • 6. Build redundancy into system

15. 1. Establish policies and protocols detailing responsibilities: provide contingencies when responsibilities not met Fewer errors in patient identification (wristband errors) when: hospital policies documented written orders for placing and removal of wrist bands Renner SW, Howanitz PJ, Bachner P. Wristband identification error reporting in 712 hospitals. Arch Pathol Lab Med 1993;117:573-577.

16. 2. Reduce steps in processes to as few as possible: reduce the opportunities to err • Fewer errors in matching transfusion patients with units of blood • Better compliance performance of required safety checks when:   • Blood handled by just one person en route • Route from blood bank to patient is direct Novis DA, Miller KA, Howanitz PJ, Renner SW, Walsh M. Audit of transfusion procedures in 660 hospitals. Arch Pathol Lab Med 2003:127:541-548.

17. 3. Educate and motivate workers in performing their tasks: create a culture of error free service • Fewer errors in matching transfusion patients with units of blood • Better compliance performance of required safety checks when: transfusionists received formal education and specialized training Novis DA, Miller KA, Howanitz PJ, Renner SW, Walsh M. Audit of transfusion procedures in 660 hospitals. Arch Pathol Lab Med 2003:127:541-548.

18. 4. Identify patients, specimens and tasks correctly from the outset: reconfirm those identities prior to engaging tasks • a. Require more than one type of identification • b. Attach patient identification to the patient • c. Require written documentation of orders • d. Include personalized instructions on orders and requisitions

19. a. Require more than one type of identification Greater number errors in misidentification of pathology specimens when: Polices allowed label to have only one, rather than two or more types of identifiers Nakhleh RE, Zarbo RJ. Surgical pathology specimen identification and accessioning: a CAP Q-Probes study of 1,004,115 cases from 417 institutions. Arch Pathol Lab Med 1996;120:227-233.

20. b. Attach patient identification to the patient Fewer errors in patient identification (wristband errors) when: policies allowed bracelet to be attached to something other than patient Renner SW, Howanitz PJ, Bachner P. Wristband identification error reporting in 712 hospitals. Arch Pathol Lab Med 1993;117:573-577.

21. c. Require written documentation of orders Greater number of errors in outpatient ordering of laboratory tests when: policiesdid not require written documentation of all orders Valenstein PN, Meier F. Outpatient order accuracy: a CAP Q-Probes study of requisition order entry accuracy in 660 institutions. Arch Pathol Lab Med 1999;123:1145-1150.

22. d. Personalize instructions on orders and requisitions Greater number of Digitalis doses given at incorrect intervals when: (17) policies did not require time of last Digoxin dose to be listed on Digitalis blood level requisition slips Howanitz PJ, Steindel SJ. Digoxin therapeutic drug monitoring practices: a CAP Q-Probes study of 666 institutions and 18 679 toxic levels. Arch Pathol Lab Med 1993;117:684-690.

23. 5. Monitor the process: maintain culture of excellence Routine continuous monitoring of performance associated with fewer errors in: •   Computerized ordering of laboratory tests • Patient identification (wristband errors)  • Matching transfusion patients with units of blood • Performance of required safety checks during blood transfusions Valenstein PN, Outpatient order accuracy: Arch Pathol Lab Med 1999;123:1145-1150. Renner SW. Wristband identification error. Arch Pathol Lab Med 1993;117:573-577. Novis DA. Audit of transfusion procedures. Arch Pathol Lab Med 2003:127:54154

24. Q-TRACKS: Long term benefits of continuous quality monitoring Routine continuous monitoring of performance associated with fewer errors in: • Frozen section diagnoses • Patient misidentification (wristband errors) Howanitz PJ, Renner SW, Walsh MK. Continuous wristband monitoring over 2 Years decreases identification errors: a CAP Q-Tracks study. Arch Pathol Lab Med 2002;126:809-815.28. Zarbo RJ. Continuous assessment lowers error rate in intraoperative consultation; the CAP Q-TRACKS experience 1999-2000. (abstract) Mod Pathol January 2001.

25. 6. Build redundancy into system : short circuit the inevitability of errors • a. Document the successful completion of required, requisite tasks • b. Make it Impossible to proceed to subsequent tasks until requisite tasks have been completed • c. Shut down systems when completion of previous steps not verified.

26. 6. Redundancy: Q-Probes studies Fewer errors in computerized ordering of laboratory tests when policies require personnel to check accuracy of orders Fewer errors in matching transfusion patients with units of blood and better compliance performance of required safety checks when: • Orders checked by second individual • Identification info. read aloud between 2 people • Transfusionists use checklists Valenstein. Ordering accuracy. Arch Pathol Lab Med 1995;119:117-122. Valenstein Outpatient order accuracy. Arch Pathol Lab Med 1999;123:1145-1150. Novis DA. Audit of transfusion procedures. Arch Pathol Lab Med 2003:127:541548.

27. Building redundancy into systems of health care delivery • Strategic objectives: Process and Product focused • Studies supporting reduced errors (fewer product defects) with product focused redundancy • Applications beyond the laboratory

28. Redundancy: strategic objectives • Process Focus Prevent errors by improving process and worker • Product Focus Last chance to catch inevitable mistakes and minimize damage

29. Product focused redundancy: Examples from Anatomic Pathology • Retrospective Case Review • Amended Case Review

30. Measuring errors by Retrospective Case Review Advantage: “True” measure: captures greatest number of errors Disadvantages: • Usually reveals errors after therapy instituted • Can not benchmark

31. Rates of errors in Anatomic Pathology upon retrospective review of case material Ranges 1-9% with outliers as high as 43% Lack of comparability--non uniformity of: • Definitions • Study conditions • Practice settings • Counting methods

32. Measuring errors by amended report rate Advantages • Focuses on outcome (product) • Able to benchmark easily Disadvantages • Underestimates magnitude of errors • Small sample sizes preclude stratifying by practices

33. Rates of errors in Anatomic Pathology based upon amended report rates: Q-PROBES study • 1.67M reports • 359 anatomic pathology departments • Aggregate rate: 1.9 errors per 1000 cases • Rates lower in departments in which cases reviewed routine prior to release of reports Nakhleh RE, Zarbo RJ. Amended reports in surgical pathology and implications for diagnostic error detection and avoidance: a CAP Q-PROBES study of 1,667,547 accessioned cases in 359 laboratories. Arch Pathol Lab Med 1998;122:303-309.

34. Effects of pre release review of case material on amended report rate: experience in a community hospital Year prior to instituting policy of routine pre-release review: diagnostic error rate = 1.3/1000 (7909 cases) Year following instituting policy of routine pre-release review: diagnostic error rate = 0.6/1000 (8469 cases) Novis D. routine review of surgical pathology cases as a method by which to reduce diagnostic errors in a community hospital. Pathology Case Reviews 2005; 10:63-67

35. Product focused redundancy • Implementation • Projected outcomes • Advantages and Disadvantages • Consumer Expectations • Global Applications

36. Implementation of product focused redundancy • Prospective review before release • Monitor failure rate

37. Outcomes of Strategy • Prevent release of defects • Shuts down production system until defects corrected • Antiquate monitor

38. Disadvantages of product focused redundancy Rare event: “The low clinically significant diagnostic error rate (-1.2% or less) probably does not justify prospective review of all diagnoses.” • Increased turnaround times. • Workday prolongation • No Reimbursement Troxel D. Error in surgical pathology. Am J Surg Pathol. 2004; 28:1092-1095.

39. Who decides what is and what is not rare? • Error rate or 1% practice of 15,000 cases per 250 days/year: > one every other day • Industry standards 6 sigma: 3.4 defects per million • Practical examples in industry: • Automotive • Airline

40. Advantages of product focused redundancy-- Addresses consumer needs • Prevents Errors • Increases efficiency • Increases uniformity of product (e.g., diagnostic consensus )

41. Global applications of lessons learned in Pathology and Laboratory medicine • Pharmacy (medication errors) • Disc imaging • Limb surgery

42. Future Directions Industrial techniques of redundancy applied to all services of medicine

43. Summary • QPROBES studies are effective tools in benchmarking and determining best practices • 6 practice characteristics identified as being effective in reducing errors • Systems employing redundancy to reduce errors meet the needs of health care consumers