Ohio’s hospital birth certificate data quality dashboard

1. Ohio’s hospital birth certificate data quality dashboard John Paulson, Judy Nagy, Ike Mgbatogu, Jingyu Hu, Srivalsan Karuvada Ohio Department of Health, Office of Vital Statistics

2. Why we created the dashboard • NCHS documents many Ohio problems • Utility of some variables compromised by inadequate quality • VS desired an automated tool to • report data quality issues to hospitals • point VS staff to most important areas for improvement

3. Main contents of the Dashboard --tabular

4. Dashboard data sourcing IPHIS (BC application) Analysis data (nightly copy) Dashboard Hospital corrections occur in the source

5. Data quality indicator sources • NCHS validation and verification errors • NCHS unknowns • Other data quality items of special interest • Included a few easy ones with few errors • No independent data were used to evaluate BC data (e.g., Medicaid, hospital discharge data)

6. Statistics shown in reports • Birth count • Error count • Error rate (%) • Error share (% errors within Ohio, within peer group) • Font color denotes rates above a threshhold

7. Thresholds • Indicator thresholds define problem cases and font color • One cut-point per indicator (not peer group specific) • In general we used the 75th percentile in error rates among hospitals as the cut-point

8. Hospital and VS user types • Hospital users see (for top 12 indicators) • own hospital’s statistics • peer group statistics • statewide statistics • listing of certificates with errors • VS users see • All of above (for all hospitals and peer groups) • 12 more indicators

9. Ohio statewide report for April/May 2014 births

10. Ohio statewide graph for April/May 2014 births

11. When user clicks on the indictor row, the peer group statistics are shown Peer groups based on birth count in 2013: 10 large hospitals, 22 medium, 85 small Statewide Large Medium Small

12. Click on peer group to get hospital-level statistics

13. Click on the hospital to see the line listing of certificates with problems

14. Report details customized to indicator

15. User can sort the rows by a columns’ values In this case bringing cases with most errors to top

16. Sorting by error share to direct intervention

17. Hospital user display features • Show error rate and how it compares to other hospitals • Show proportion of errors the hospital is responsible for statewide and within a peer group • Use color to direct attention to problems • Provide a line listing of certificates with problems

18. Hospital is known from the local login process Date range defaults to the current year Hospital can choose a specific date range for the report as needed

19. Tabular report shown in bar chart, hospital with a handful of problems

20. Hospital with two handfuls of problemsl

21. Observations • Good • Development was intensive iterative process • We are happy with the product and its functionality • We built a hospital usage report to track activity • Not so good • We did not build a summary report that shows us the biggest problem facilities across all indicators • Adoption by hospitals has been slow • Requires individual log-in assignment outside the electronic birth system • We have not identified the correct staff within hospitals and hospital systems who are most likely to use the dashboard

22. John Paulson john.paulson@odh.ohio.gov 614-644-8507