1 / 32

Refinement and Validation of the AHRQ Patient Safety Indicators

Refinement and Validation of the AHRQ Patient Safety Indicators. EPC Team (PSI Development) PI: Kathryn McDonald, M.M., Stanford Patrick Romano, M.D., M.P.H, UC Davis Jeffrey Geppert, J.D., Ed.M., Stanford Sheryl Davies, M.A., Stanford

regis
Télécharger la présentation

Refinement and Validation of the AHRQ Patient Safety Indicators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Refinement and Validation of the AHRQ Patient Safety Indicators EPC Team (PSI Development) PI: Kathryn McDonald, M.M., Stanford Patrick Romano, M.D., M.P.H, UC Davis Jeffrey Geppert, J.D., Ed.M., Stanford Sheryl Davies, M.A., Stanford Bradford Duncan, M.D., M.A., Stanford Kaveh G. Shojania, M.D., UCSF Support of Quality Indicators PI: Kathryn McDonald, M.M., Stanford Patrick Romano, M.D., M.P.H, UC Davis Jeffrey Geppert, J.D. Ed.M., Stanford Sheryl Davies, M.A., Stanford Mark Gritz, PhD, Battelle Greg Hubert, Battelle Denise Remus, Ph.D., RN, AHRQ Developed by UC-Stanford Evidence Based Practice Center Funded by the Agency for Healthcare Research and Quality

  2. Acknowledgments Funded by AHRQ Contract No. 290-97-0013 Support of Quality Indicators Contract No. 290-02-0007 Data used for analyses: Nationwide Inpatient Sample (NIS), 1995-2000. Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality State Inpatient Databases (SID), 1997 (19 states). Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality For more information: http://www.qualityindicators.ahrq.gov

  3. Acknowledgments • We gratefully acknowledge the data organizations in participating states that contributed data to HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health and Development; Colorado Health and Hospital Association; CHIME, Inc. (Connecticut); Florida Agency for Health Care Administration; Georgia Hospital Association; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Maryland Health Services Cost Review Commission; Massachusetts Division of Health Care Finance and Policy; Missouri Hospital Industry Data Institute; New Jersey Department of Health and Senior Services; New York State Department of Health; Oregon Association of Hospitals and Health Systems; Pennsylvania Health Care Cost Containment Council; South Carolina State Budget and Control Board; Tennessee Hospital Association; Utah Department of Health; Washington State Department of Health; and Wisconsin Department of Health and Family Service.

  4. Rationale for the PSIs Background: Perceived need for an inexpensive patient safety surveillance system based on readily available data UC-Stanford EPC charge: To review and improve the evidence base related to potential patient safety indicators (PSIs) that can be ascertained from data elements in a standardized, multi-state health data system, the Healthcare Cost and Utilization Project (HCUP).

  5. Literature review to find candidate indicators • MEDLINE/EMBASE search guided by medical librarians at Stanford and NCPCRD (UK) • Few examples described in peer reviewed journals • Iezzoni et al.’s Complications Screening Program (CSP) • Miller et al.’s Patient Safety Indicators • Review of ICD-9-CM code book • Codes from above sources were grouped into clinically coherent indicators with appropriate denominators

  6. Structure of indicators • All definitions were created using ICD-9-CM diagnosis and procedure codes (along with DRG, MDC, sex, age and procedure dates) • Numerator of each indicator is the number of cases with the complication of interest (e.g., Postop DVT/PE) • Denominator of each indicator is the number of hospitalizations (or patients) considered to be at risk (e.g. elective surgical patients) • Exclusions were defined to restrict the denominator to patients for whom the complication was less likely to have been present at admission, and more likely to have been preventable • The indicator “rate” is the numerator/denominator

  7. PSI assessment methods • Literature review to gather data on coding and construct validity • ICD-9-CM coding consultant review (face validity) • Clinical panel review (face validity) • Empirical analyses of nationwide rates, hospital variation, impact of risk adjustment, and relationships among indicators

  8. Clinical panel review • Intended to establish consensual validity • Modified RAND/UCLA Appropriateness Method • Physicians of various specialties/subspecialties, nurses, other specialized professionals (e.g., midwife, pharmacist) • Potential indicators were rated by 8 multispecialty panels; surgical indicators were also rated by 3 surgical panels • All panelists rated all assigned indicators (1-9) on: • Overall usefulness • Likelihood of identifying the occurrence of an adverse event or complication (i.e., not present at admission) • Likelihood of being preventable (i.e., not an expected result of underlying conditions) • Likelihood of being due to medical error or negligence (i.e., not just lack of ideal or perfect care) • Likelihood of being clearly charted in the medical record • Extent to which indicator is subject to bias due to case mix

  9. Evaluation framework Medical error and complications continuum Medical error Nonpreventable Complications • Pre-conference ratings and comments/suggestions • Individual ratings returned to panelists with distribution of ratings and other panelists’ comments/suggestions • Telephone conference call moderated by PI and attended by note-taker, focusing on high-variability items and panelists’ suggestions (90-120 mins) • Suggestions adopted only by consensus • Post-conference ratings and comments/ suggestions

  10. Final selection of indicators • Retained indicators for which “overall usefulness” rating was “Acceptable” or “Acceptable-” : • Median score 7-9 • Definite or indeterminate agreement • Excluded indicators rated “Unclear,” “Unclear-,” or “Unacceptable”: • Median score <7, OR • At least 2 panelists rated the indicator in each of the extreme 3-point ranges

  11. PSIs reviewed • 48 indicators reviewed in total • 37 reviewed by multispecialty panel • 15 of those reviewed by surgical panel • 20 “accepted” based on face validity • 2 dropped due to operational concerns • 17 “experimental” or promising indicators • 11 rejected

  12. Selected postoperative complications Postoperative thromboembolism Postoperative respiratory failure Postoperative sepsis Postoperative physiologic and metabolic derangements Postoperative abdominopelvic wound dehiscence Postoperative hip fracture Postoperative hemorrhage or hematoma Selected technical adverse events Decubitus ulcer Selected infections due to medical care Technical difficulty with procedures Iatrogenic pneumothorax Accidental puncture or laceration Foreign body left in during procedure Other Complications of anesthesia Death in low mortality DRGs Failure to rescue Transfusion reaction Obstetric trauma and birth trauma Birth trauma – injury to neonate Obstetric trauma – vaginal delivery with instrument Obstetric trauma – vaginal delivery without instrument Obstetric trauma – cesarean section delivery “Accepted” PSIs

  13. National trends in PSI rates • Nationwide Inpatient Sample (NIS), 1995-2000 • 7.5 million discharges/1,000 hospitals/28 States • Approximates 20% sample of nonfederal acute care hospitals • Discharge level weights applied to generate national estimates for each year • Adjusted for age, gender, age-gender inter-actions, comorbidities, and DRG clusters • 1,121,000 potential safety-related events affecting 1,070,000 hospitalizations Romano et al., Health Affairs 2003; 22(2):154-166

  14. Estimated cases in 2000

  15. Impact of patient safety events in 2000 (Zhan and Miller, JAMA 2003)

  16. Estimated cases in 2000

  17. Impact of patient safety events in 2000 (Zhan and Miller, JAMA 2003)

  18. National trends 1995-2000 Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

  19. National trends 1995-2000 Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

  20. National trends 1995-2000 Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

  21. National trends 1995-2000 Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

  22. National trends 1995-2000 Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

  23. Research/Policy Question Why are some PSIs increasing in incidence over time while others are decreasing? • Selective changes in coding practice • Changes in severity of illness or underlying risk of potential safety-related events • True changes in quality due to technical improvements in surgical or nursing technique, counterbalanced by inadequate staffing to prevent some complications

  24. Standard deviation of hospital effects: 1997 SID PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

  25. Ratio of hospital-level signal to total hospital variation: 1997 SID PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

  26. Year-to-year correlation of hospital effects: 1996-97 Florida SID PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

  27. Risk adjustment methods • Must use only administrative data • APR-DRGs and other canned packages may adjust for complications • Final model • DRGs (complication DRGs aggregated) • Modified Comorbidity Index based on list developed by Elixhauser et al. • Age, Sex, Age-Sex interactions

  28. Hospital level variation:Impact of bias, 1997 SID (summary)

  29. PSIs loading on “catheter-related and technical complications” (factor 1) PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

  30. PSIs loading on “post/intraoperative complications” (factor 2) PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

  31. PSIs loading on neither factor (<1% variance explained) PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

  32. Conclusions • Administrative data are appealing, but the development of indicators is time-consuming • Variations across hospitals and over time merit further exploration • Potentially useful screening tool for providers, provider associations, and health data agencies to identify possible safety problems • Ongoing support and validation work expected to offer many more insights into opportunities and obstacles in using administrative data for patient safety surveillance

More Related