The Challenges of Working With Clinical Data

1. The Challenges of Working With Clinical Data The Promise and Pain of Electronic Health Records Robert W. Grundmeier, MD October 17, 2008

2. Disclosures • Employment • The Children’s Hospital of Philadelphia • Financial Interests • None • Research Interests • Funded by Agency for Healthcare Research and Quality, Pew Charitable Trusts, NIH • Organizational Interests • American Academy of Pediatrics • Gifts • Nothing to Disclose

3. Overview • The Promise • Richness and freshness of data • Large volume of data • The Pain • Data access and tools • Misclassification • The Path Ahead

4. Paper-Based “Poetry”

5. “Semi-Structured” EHR Template

6. The Promise • Large number of potential subjects with unprecedented “richness” and “freshness” of data • One organization actually funded their EHR implementation with clinical research* * Miller JL. The EHR solution to clinical trial recruitment in physician groups. Health Management Technology. 2006;27(12):22-5

7. The Promise ofRichness and Freshness • Thousands of repeated observations recorded for each potential subject over time • Longitudinal health problem diagnoses • Billing diagnoses • Vital signs and measurements • Prescriptions • Immunizations • Structured preventive health visits • Laboratory and radiology data • Procedures • And many more!

8. The Promise:Large Volume of Data • 7 Years of data • 9 Subspecialty centers • 9 Subspecialty divisions • 32 Primary care sites • 300,000+ Patients • 2,500,000+ Visits • 65,000,000+ Observations

9. The Pain: Data Access • The promise of self-service reporting is rarely delivered • Data model is overwhelmingly complex • 2,800+ Tables • 29,000+ Fields • “Haphazard” normalization • Extensive training and certification required to be granted the “keys to the kingdom” • You need a Svetlana • And even that doesn’t make you “mistake proof”

10. The Pain: Tools • Vendor or health system supplied or recommended reporting tools are expensive and pretty, but inadequate • Adequate tools may be free but ugly and complex to use effectively • Transfer of large datasets is non-trivial • MS Excel limitation always exceeded (65,000 rows) • MS Access limitation often exceeded (2 Gb) • We use XML as our interchange format • Question: How will you get a >2 Gb file to your collaborator? E-mail? I think not!

11. The Pain: Misclassification • EHR marketing departments say: • In the bad old days, a professional coder would choose the best ICD-9 code for reimbursement purposes • In the new era of the EHR, a clinician chooses the most clinically meaningful ICD-9 code • Svetlana says: • In the bad old days, you knew the limitations of your data • In the new era, you discover new limitations of your data every day… if you’re lucky

12. The Pain: Vignettes • Asthma • Attention Deficit Hyperactivity Disorder • Varicella • Otitis Media

13. Asthma “Misclassification across time and space” • Common conditions are coded commonly, and reasonably well • 57,820 Patients billed for asthma care • 53,824 Patients with asthma on problem list • 54,993 Patients with at least 2 albuterol prescriptions • This is EXCELLENT correlation

14. Persistent Asthma • What about persistent asthma? • 16,949 Patients billed for “persistent asthma” • 11,943 With “persistent asthma” as a problem • But… • 23,673 Patients with at least 2 inhaled corticosteroid prescriptions • And… • Only 3,553 With persistent symptoms based on questionnaire • Huh?

15. Non-Random Misclassification By Care Location • “It is OK to compare organizations using their electronic data because everyone has the same problems with their data… the playing field is level” • Svetlana: Oh, really?

16. Non-Random Misclassification Over Time • And, the playing field changes over time • In 2004 one could have been lulled into a false sense of security over the reliability of encounter or problem list data… Actually, WE WERE!

17. Another Vignette: ADHD“Read The Chart”

18. Availability of ADHD “Phenotype” (N=57 Subjects)

19. ADHD: “Prospective ≠ Retrospective” • Initial explorations of data available for exposures of interest regarding ADHD were promising • Cohort of 12-23 month olds were examined in 2007

20. Retrospective “Exposures” • “Exposures” of interest often occur years before disease (e.g. ADHD patients are 6+ years old) • Data are that are well captured prospectively may not be available retrospectively

21. The Varicella Story: “What we have here is a failure to negate” • In our environment, routine data extracts are sent to public health officials for outbreak monitoring • Before universal immunization, Varicella was a common clinical diagnosis • Now classic presentations are rare and corroborating lab evidence is usually obtained by clinicians

22. Varicella: “h/o” ICD9 052 • Now that Varicella is rare, and coding is completed by clinicians seeking clinical relevance there are some “oopsies” • ICD9 codes beginning with 052 all imply “Varicella at this moment” • No ICD9 codes exist to say, “This is not Varicella,” or “Patient had Varicella in the past” • For rare conditions, deliberate use of a diagnosis code to mean the absence of a condition will exceed the use to mean the presence of a condition

23. Varicella: How to Make Your Health Department Unhappy • Health Department officials began chasing down cases of NON Varicella • In 2005: 39 non-cases vs. 34 actual cases • Families were confused, scarce public health money wasted, providers yelled at “Dr. Bob, why did you send that patient to school with Chicken Pox?” –Philadelphia Department of Public Health Official • Instead… Look for the labs!

24. Otitis Media: “There is hope… you can change the future” • Key process outcomes in otitis media include: • Documented laterality of disease (which ear) • Documented presence or absence of middle ear effusion and inflammation • Judicious use of antibiotics and watchful waiting • And a few others that we actually can measure today

25. Otitis Media • ICD9 codes and medication data could help, but rely on free text clarification • “ACUTE OTITIS MEDIA on RIGHT” • “WATCHFUL WAITING: begin in 2 days if fever/pain: Amoxicillin 400 mg PO BID” • “Floxin Otic to RIGHT ear…”

26. Otitis Media • Desperation brought us to inspect free text ear exam information available • 427,831 Visits for an ear related complaint inspected • 58,905 Unique ear exam descriptions were found

27. Otitis Media: Change the Data Collection Form • Must think about how to make the clinician want to use the new data capture tool • We are doing a comprehensive decision support intervention for this reason

28. The Path Ahead:The Misclassification Story • Once you have a research variable “well coded,” guard it with your life • Re-check validity if it has been idle for a year • Do not make colleagues re-invent measures • Once you have validated a measure, SHARE IT! (At least within your organization) • We use a wiki within our group for this purpose • But… your variable definition may not apply at another organization, even with the same EHR!

29. The Path Ahead:Read The Charts! • Resist the temptation of ICD9 and CPT… Or even SNOMED, LOINC, and RxNorm if you are so lucky • Start with the clinical workflow for a disease of interest, and consider all the clues that may exist in the electronic artifacts • You don’t know what you don’t know, until you read the charts • Find cohorts enriched in the disease, and read! • If still unsure, pound the pavement • Visit the locations of primary data collection

30. The Path Ahead:Prospective ≠ Retrospective • Just because data are available in a recent convenience sample does not mean it will be available for your real cohort • As much as possible, look at the actual cohort for preliminary data purposes • Avoid over re-use of preliminary data from one study for subsequent studies

31. The Path Ahead:Negation and ICD9 • Stating a negative with ICD9 nomenclature is not possible, yet clinicians want to (and will) express negatives using ICD9 codes • This is especially a problem for rare conditions • Look for corroborating evidence that arises from the workflow, especially lab tests or other observations that may suggest concern for the disease

32. The Path Ahead: Data Collection Nirvana • Changing data collection strategies in the EHR is possible • However, these modifications must be part of a comprehensive approach designed to improve the clinical workflow • You can’t get data for nothing

33. Thank You