CheckPoint Error Prevention Audit and Methodology Update Fall 2006

1. 1 CheckPoint Error Prevention Audit and Methodology UpdateFall 2006 Welcome to today�s presentation on CP Error Prevention Audit and Methodology Updates Purpose of this type of audit: assess degree of hospital compliance to CP Error Prevention Methodology Why? No other audit being done of this data where medical and SIP measures have audits Challenge? Methodology has to be specific enough to set a criteria for others to follow. Welcome to today�s presentation on CP Error Prevention Audit and Methodology Updates Purpose of this type of audit: assess degree of hospital compliance to CP Error Prevention Methodology Why? No other audit being done of this data where medical and SIP measures have audits Challenge? Methodology has to be specific enough to set a criteria for others to follow.

2. 2 Audit BackgroundError Prevention Data � July 1 to December 31, 2005 16 hospitals randomly selected for audit, 111 hospitals reporting data Mixture of large and small hospitals All hospitals passed with scores from 81.4% to 99.7% (80%=passing score) Audit reports mailed August 2, 2006 Hospital with the lowest score opted to not submit Error Prevention measures in the future because they operate on a more ambulatory basis 2nd lowest scorer didn�t keep a record of the medication orders so no points assigned Selection process: random (big metal bowl) No hospital excluded Concerns raised? BAA�s, time, didn�t know about requirements to keep documentsHospital with the lowest score opted to not submit Error Prevention measures in the future because they operate on a more ambulatory basis 2nd lowest scorer didn�t keep a record of the medication orders so no points assigned Selection process: random (big metal bowl) No hospital excluded Concerns raised? BAA�s, time, didn�t know about requirements to keep documents

3. 3 Summary of Scores  Most variability in Measure #3, Dangerous Abbreviations. Most challenging process measure to implement. Communication between quality and pharmacy departments to keep documentation needed.Most variability in Measure #3, Dangerous Abbreviations. Most challenging process measure to implement. Communication between quality and pharmacy departments to keep documentation needed.

4. 4 Key Findings Good compliance to site marking, time-out, and concentrated electrolyte measures (measures 1, 2 & 4) Medication abbreviations (measure 3) showed most variance with scores from 50 to 100 points (total of 100 points) 2 hospitals were unable to submit med orders for audit but still passed Site marking, time-out/final verification: usually only a problem if non-surgical area (exceptions). Use of forms varied from very structured forms to free-form notes. 2 hospitals didn�t keep list of med orders used. Did well on other measures so passed. Can�t audit without documentation. Site marking, time-out/final verification: usually only a problem if non-surgical area (exceptions). Use of forms varied from very structured forms to free-form notes. 2 hospitals didn�t keep list of med orders used. Did well on other measures so passed. Can�t audit without documentation.

5. 5 Key FindingsDangerous Abbreviations Hospitals didn�t always count numerical abbreviations properly -- didn�t count at all or counted numbers such as 3.25 Sometimes counted other �Q� abbreviations such as Q AM Use of computerized systems and pre-printed forms reduced error rates A few hospitals don�t count numbers (dosages etc.) used properly or improperly (trailing 0 or lack of a leading 0). Need to count all instances of whole numbers with trailing 0�s as incorrect (0 in numerator, 1 in denominator) and without trailing 0�s as correct (1 in numerator and denominator). The same applies to decimal fractions. Count the lack of a leading 0 as incorrect and the use of a leading 0 as correct. If numbers not counted, generally had lower demonstrated success rates because more errors occur with word (alphabetic) abbreviations than numerical abbreviations. Resulted in understated rates, look worse than they were in CheckPoint. Q still a source of problems. We�d prefer it not be used but recognize some abbreviations as okay example: Q daily. Automation in whatever form generally reduces error rates (could be forms or EPR). Some hospitals still have dangerous abbreviations on pre-printed forms. A few hospitals don�t count numbers (dosages etc.) used properly or improperly (trailing 0 or lack of a leading 0). Need to count all instances of whole numbers with trailing 0�s as incorrect (0 in numerator, 1 in denominator) and without trailing 0�s as correct (1 in numerator and denominator). The same applies to decimal fractions. Count the lack of a leading 0 as incorrect and the use of a leading 0 as correct. If numbers not counted, generally had lower demonstrated success rates because more errors occur with word (alphabetic) abbreviations than numerical abbreviations. Resulted in understated rates, look worse than they were in CheckPoint. Q still a source of problems. We�d prefer it not be used but recognize some abbreviations as okay example: Q daily. Automation in whatever form generally reduces error rates (could be forms or EPR). Some hospitals still have dangerous abbreviations on pre-printed forms.

6. 6 Key Post-Audit Survey Findings 13/16 surveys received- 81% response rate) Question: Audit helped me to improve reporting of Error Prevention measures 69% of responses � Strongly Agree/Agree Question: Total time spent on audit 31% indicated 5-10 hours 23% indicated 10-15 hours 46% indicated 15-20 hours Most agreed that the audit provided value Hours spent include hours for all departments involved in audit. Smaller hospitals are sometimes more challenged to participate in audit because quality person may not be full-time. Flexibility in deadlines extended.Most agreed that the audit provided value Hours spent include hours for all departments involved in audit. Smaller hospitals are sometimes more challenged to participate in audit because quality person may not be full-time. Flexibility in deadlines extended.

7. 7 Follow-up to Audit Updated Error Prevention Methodology to provide more counting examples for med abbreviations and address other questions raised during audit Scheduled teleconference with next set of hospitals to be audited. Reformatted audit procedures to be more user-friendly. To develop audit failure policy Agreement on what to count and what not to count will add to more consistency in counting dangerous abbreviations. Still a work a progress and judgment still required. WHA Board to work on an audit failure policy. Currently the only policies are if you fail, you�ll be subject to re-audit. Agreement on what to count and what not to count will add to more consistency in counting dangerous abbreviations. Still a work a progress and judgment still required. WHA Board to work on an audit failure policy. Currently the only policies are if you fail, you�ll be subject to re-audit.

8. 8 Follow-up to Audit Schedule a regular (quarterly?) CheckPoint user group conference (in person or web conference) to train new users and to provide updates on Error Prevention methodology Would like to schedule quarterly updates in the off months, maybe 1 month before a data submission date Would like to schedule quarterly updates in the off months, maybe 1 month before a data submission date

9. 9 Questions on CheckPoint Error Prevention Audit? Next audit has started with results expected out in mid-December Med Reconciliation audit procedures are being tested but have no impact on audit scores In 2007, expect that only demonstrated success rates will be reported and audited reducing audit work burden It�s an evolving process as new measures added Next audit has started with results expected out in mid-December Med Reconciliation audit procedures are being tested but have no impact on audit scores In 2007, expect that only demonstrated success rates will be reported and audited reducing audit work burden It�s an evolving process as new measures added

10. 10 CheckPoint Error Prevention Methodology Version 7 Updates No major changes Clarifications: What to keep for audit and minor audit changes CPT codes added/reformatted current code list When to exclude a surgical case from sampling How to handle laparoscopic cases Definition of a med order Examples of dangerous abbreviations counting No major changes, no new measures Changes based on questions received or on audit findings No major changes, no new measures Changes based on questions received or on audit findings

11. 11 Audit Documentation Keep list of cases or medication orders/list of med orders for 1 year for audit purposes Hospitals that can�t produce a list of cases for audit will get 0 marks for that audit section Short version � keep an audit trail such as list of cases or med orders used. 32 hospitals audited per year, you have a good chance of being selected for audit in any given year. Make your life easier. Short version � keep an audit trail such as list of cases or med orders used. 32 hospitals audited per year, you have a good chance of being selected for audit in any given year. Make your life easier.

12. 12 Site Marking/Final VerificationICD-9 and CPT Code lists ICD-9 Code list reformatted to be in a similar format to National Hospital Quality Measures� ICD-9 list CPT code list has been added -- new requirement to use these for ambulatory surgery cases effective 1/1/2007 for submitting data to WHA Information Center ICD-9 code list written out instead of abbreviated. Consistent with national standards but makes the Error Prevention methodology long. CPT codes added due to new requirement to report for ambulatory surgery cases ICD-9 code list written out instead of abbreviated. Consistent with national standards but makes the Error Prevention methodology long. CPT codes added due to new requirement to report for ambulatory surgery cases

13. 13 Surgical Cases As long as the outpatient area is within your hospital�s control and management structure, include the case Exclude cases in clinics or outpatient areas owned by someone else. For instance, you allow another organization to use your physical space but have no control or accountability for quality of service When to count a surgical case? If you manage or control the department, count it If you just rent out the space and have no control, don�t count it. When to count a surgical case? If you manage or control the department, count it If you just rent out the space and have no control, don�t count it.

14. 14 Surgical Cases Primary or secondary ICD-9 or CPT code on case? Surgical cases can have a code as primary or secondary to qualify for the site marking or time-out/verification measures Question was raised on whether a code had to be primary on the case to include it As long as a code is primary or secondary, site marking or time-out/final verification would be required. The case can be included. Question was raised on whether a code had to be primary on the case to include it As long as a code is primary or secondary, site marking or time-out/final verification would be required. The case can be included.

15. 15 Laparoscopic/Interoperative Cases Mark site for all laparoscopic cases that have laterality (near proposed site or insertion point) Interoperative imaging cases � mark for those cases involving organs or structures with laterality If the site has laterality (bilateral organs/structures), site marking should be done. If the site has laterality (bilateral organs/structures), site marking should be done.

16. 16 Medication Order Definition Medication orders include herbals, nutritional supplements, homeopathic remedies etc. if your hospital has a policy to count these as medication orders or includes them in the medication reconciliation process If your hospital considers it a medication, include it in dangerous abbreviation counting or medication reconciliation measures. If your hospital considers it a medication, include it in dangerous abbreviation counting or medication reconciliation measures.

17. 17 Medication Order Counting Primary problem is that some hospitals aren�t counting numbers that are correctly or incorrectly written. This usually results in a lower demonstrated success rates (more errors occur in word abbreviations) Other hospitals count more than 9 dangerous abbreviations. Count dosages/other numbers used correctly or incorrectly Don�t count other abbreviations your hospital has added to the list for CheckPoint. Count dosages/other numbers used correctly or incorrectly Don�t count other abbreviations your hospital has added to the list for CheckPoint.

18. 18 Medication Order Counting (Cont�d) Examples have been provided to standardize counting among hospitals and reduce reporting variation Some questions remain on counting different Q abbreviations. The preference is to avoid Q and write it out in full. Examples given as a starting point. Can�t provide examples for all instances. Still judgment required. Q still causes problems. Write out in full and avoiding use of Q is still preferred. Examples given as a starting point. Can�t provide examples for all instances. Still judgment required. Q still causes problems. Write out in full and avoiding use of Q is still preferred.

19. 19 Questions on CheckPoint Error Prevention Methodology? Any questions, please contact: Kathleen Caron: Kcaron@wha.org Thanks for joining us for this presentation. Any questions, please call.Thanks for joining us for this presentation. Any questions, please call.