SIR 101 : Interpretation and public reporting

1. SIR 101: Interpretation and public reporting Dana Burshell, MPH, CPH, CIC HAI Epidemiologist Virginia Department of Health

2. NHSN’s Guide to the Standardized Infection Ratio (SIR) http://www.cdc.gov/nhsn/PDFs/Newsletters/NHSN_NL_OCT_2010SE_final.pdf

3. What is a standardized infection ratio? The standardized infection ratio (SIR) is a summary measure used to track healthcare-associated infections (HAIs) at a national, state, or local level over time. The SIR adjusts for patients of varying risk within each facility. - The National Healthcare Safety Network (NHSN)

4. Benefits of using the SIR • Single metric • One number that can be used to make comparisons • Scalable • National, regional, facility-wide, location-specific, by surgeon for SSIs, etc. • Can combine the SIR values at any level of aggregation • Can perform more detailed comparisons within any individual risk group • Risk-adjusted • Adjusts for factors known to be associated with differences in HAI rates • Risk-adjustment differs between types of HAIs and types of surgical procedures - HHS HAI Action Plan - http://www.hhs.gov/ash/initiatives/hai/appendices.html#appendix_g_comparison

5. The SIR calculation • In HAI data analysis, the SIR compares the actual number of HAIs reported (observed) with the baseline U.S. experience (predicted) adjusting for several risk factors that have been found to be significantly associated with differences in infection incidence. • SIR is a ratio that is a comparison of two values SIR = number of observedHAIs number of predictedHAIs

6. What is the “baseline U.S. experience”? • 2006-2008 NHSN aggregate data are used as the standard population and considered to be the baseline U.S. experience for the SIR calculations. • NHSN baseline data used in an SIR are used to calculate the predicted number of HAIs adjusting for the identified risk factors.

7. What does the SIR number mean? • An SIR greater than 1.0 indicates that more HAIs were observed than predicted. • An SIR of 1.0 indicates that the number of HAIs observed was equal to the number predicted. • An SIR less than 1.0 indicates that fewer HAIs were observed than predicted. • However, the SIR alone does not imply statistical significance. • The SIR is only a point estimate and needs additional information to indicate if the finding is significant and not likely due to chance (that is, statistically significantly different from 1).

8. Statistical significance • A p-value and 95% confidence interval (CI) are calculated by NHSN for each SIR. • The p-value identifies if the information is statistically significant. • If the p-value is < 0.05, the SIR is “statistically significant". • The 95% CI can sometimes be used to approximate statistical significance. • A 95% CI assesses the SIR’s magnitude and stability. • If the SIR 95% CI does not contain the value 1, the SIR is considered "statistically significant".

9. What does a significant SIR mean? • While in many cases, significantly high SIRs may reflect a need for stronger CLABSI prevention efforts and significantly low SIRs may support already existing strong CLABSI prevention efforts, several other factors such as validation of reported data may play a role.  • The real measure of success is following the SIRs over time to indicate if positive progress occurs and is sustained. • Because the ultimate goal is zero HAIs, prevention efforts are never complete.

10. Explaining and interpreting the SIR: Virginia data During 2011, there were 233CLABSIs identified and 194,483central line days observed in Virginia adult intensive care units. Based on the NHSN 2006-2008 baseline data and the composition of locations in Virginia facilities, 366 CLABSIs were predicted. This results in an SIR of 0.64 (O/P= 233/366), signifying that during this time period, Virginia facilities identified 36% fewer CLABSIs than predicted. The p-value (<0.001) and 95% confidence interval (0.56, 0.72) indicate that the number of observed CLABSIs is statistically significantly lower than the number of predicted CLABSIs. (Reminder: If the p-value is less than 0.05 and the 95% CI does not cross 1, the SIR is statistically significantly different than 1.)

11. Why publicly report HAI data? • Infection data can give healthcare facilities, patients, and public health agencies the knowledge needed to design and implement prevention strategies that protect patients and save lives. • Research shows that when healthcare facilities are aware of their infection issues and implement concrete strategies to prevent them, rates of certain hospital infections can be decreased by more than 70 percent.  - CDC NHSN 2009 Report Q&A: http://www.cdc.gov/hai/QA_stateSummary.html

12. Intra-facility data sharing benefits: • Top benefits identified by SSI pilot study IPs (2011) • Increased awareness of HAIs within the facility • Presented data to those who can make a difference • Provided benchmark data to support improvement initiatives • Kept HAIs in the spotlight

13. How is the SIR being used currently? • NHSN SIR reports • CLABSI, SSI • CMS Hospital Compare website • CLABSI • Updated VDH HAI report • CLABSI • Other states • Within hospitals

14. NHSN SIR report: CLABSI July-Dec 2009 Virginia US - all Note: Includes PICU, but not NICU Note: Data only for states using NHSN to comply with a legislative mandate* to report HAIs to the state health department (reported as of September 2010) http://www.cdc.gov/HAI/pdfs/stateplans/state-specific-hai-sir-july-dec2009r.pdf

15. Note: Reference Period = 2006-2008 http://www.cdc.gov/HAI/pdfs/stateplans/SIR-2010_JunDec2009.pdf

16. Hospital Compare • Go to the Hospital Compare website • http://www.hospitalcompare.hhs.gov/ • Find hospitals near your location • Choose up to 3 hospitals to compare • Select “Patient Safety Measures” • Scroll down to “Healthcare • Associated Infections (HAIs)” • Click “View Graphs”

17. Hospital Compare displays SIR • Only includes SIR point estimate (no p-value or 95% CI) • Language • Better than the US National Benchmark • Same as the US National Benchmark • Worse than the US National Benchmark • Not available • Lower numbers are better. A score of zero – meaning no CLABSIs – is best.

18. New VDH CLABSI report: Table 1 shows annual SIR of all hospitals

19. New VDH CLABSI report: Color legend

20. In Hospital X’s adult intensive care units during 2011, there were 5CLABSIs identified and 2,611central line days observed. • Based on the NHSN 2006-2008 baseline data and the composition of locations in Hospital X, 5.3 CLABSIs were predicted. • This results in an SIR of 0.9 (O/P= 5/5.3), signifying that during this time period, Hospital X identified 10% fewer CLABSIs than predicted. • The 95% confidence interval (0.30, 2.21) indicates that the number of observed CLABSIs is not statistically different than the number of predicted CLABSIs.

21. SIR comparisons • Hospital X’s SIR of 0.9 is higher than both the overall SIR for its bedsize category (0.6) and the SIR of all Virginia hospitals with adult ICUs (0.6). • Interpretation: Hospital X identified more CLABSIs than the average for its bedsize and in the state overall; however, it is not statistically significant.

22. New VDH CLABSI report: Hospital-specific graph shows annual CLABSI SIR over time

23. Interpreting VDH CLABSI report graph SIRpointestimate SIR95%CI SIR= 1 observed = predicted

24. New VDH CLABSI report: Table 2 has hospital-specific and comparison data

25. Central line-associated bloodstream infections (CLABSIs) in adult intensive care units standardized infection ratio by quarter, Virginia, 2009-2011 Trendline added using Excel to visually show overall increases or decreases Line graph of SIR point estimates • Since 2009, Virginia facilities have identified fewer CLABSIs than predicted in adult ICUs, • reflecting already existing strong infection prevention efforts. • Although there is not a consistent decrease from quarter to quarter, the overall trend • since 2009 has been decreasing. • Because the ultimate goal is sustaining zero CLABSIs, individual hospital effective • prevention efforts should continue in addition to identifying and acting to address gaps.

26. Resources • CDC's National Healthcare Safety Network (NHSN) HAI Summary Data Reports Q and A • http://www.cdc.gov/HAI/surveillance/QA_stateSummary. html • NHSN e-News: SIRs Special Edition • http://www.cdc.gov/nhsn/PDFs/Newsletters/NHSN_NL_OCT_2010SE_final.pdf • VDH HAI website – surveillance • http://www.vdh.virginia.gov/Epidemiology/Surveillance/HAI/SurveillanceReporting.htm

27. Thank you! Dana.Burshell@vdh.virginia.gov 804-864-7550

28. Henrico Doctors’ Hospital IP Team Shared SIR with Leadership of Committee • Why you decided to educate your team • Hospital Compare website (1st Quarter Data) • VDH planning to use SIR • Corporate 2011 report using SIR (red, yellow, green) • Who you thought it was important to educate • Leadership in IP (Chairman, Chief Nursing personnel, Quality Director) • How did it go? • Explanation of SIR focused on > < 1 compared with NHSN national data • Example calculation • Examples of CLABSI corporate SIRs compared with CLABSI rates • Hospital Compare screenshot • Tables from VDH newsletter comparing CLABSI rates 2009-11 beside SIRs. • Nice display of confidence intervals. • Tips • Bring NHSN data summary reports to show where comparative data comes from • Next steps

29. - http://www.vdh.virginia.gov/epidemiology/surveillance/hai/communication.htm

30. Questions?