The Challenges of Working With Clinical Data The Promise and Pain of Electronic Health Records Robert W. Grundmeier, MD October 17, 2008
Disclosures • Employment • The Children’s Hospital of Philadelphia • Financial Interests • None • Research Interests • Funded by Agency for Healthcare Research and Quality, Pew Charitable Trusts, NIH • Organizational Interests • American Academy of Pediatrics • Gifts • Nothing to Disclose
Overview • The Promise • Richness and freshness of data • Large volume of data • The Pain • Data access and tools • Misclassification • The Path Ahead
Paper-Based “Poetry”
“Semi-Structured” EHR Template
The Promise • Large number of potential subjects with unprecedented “richness” and “freshness” of data • One organization actually funded their EHR implementation with clinical research* * Miller JL. The EHR solution to clinical trial recruitment in physician groups. Health Management Technology. 2006;27(12):22-5
The Promise ofRichness and Freshness • Thousands of repeated observations recorded for each potential subject over time • Longitudinal health problem diagnoses • Billing diagnoses • Vital signs and measurements • Prescriptions • Immunizations • Structured preventive health visits • Laboratory and radiology data • Procedures • And many more!
The Promise:Large Volume of Data • 7 Years of data • 9 Subspecialty centers • 9 Subspecialty divisions • 32 Primary care sites • 300,000+ Patients • 2,500,000+ Visits • 65,000,000+ Observations
The Pain: Data Access • The promise of self-service reporting is rarely delivered • Data model is overwhelmingly complex • 2,800+ Tables • 29,000+ Fields • “Haphazard” normalization • Extensive training and certification required to be granted the “keys to the kingdom” • You need a Svetlana • And even that doesn’t make you “mistake proof”
The Pain: Tools • Vendor or health system supplied or recommended reporting tools are expensive and pretty, but inadequate • Adequate tools may be free but ugly and complex to use effectively • Transfer of large datasets is non-trivial • MS Excel limitation always exceeded (65,000 rows) • MS Access limitation often exceeded (2 Gb) • We use XML as our interchange format • Question: How will you get a >2 Gb file to your collaborator? E-mail? I think not!
The Pain: Misclassification • EHR marketing departments say: • In the bad old days, a professional coder would choose the best ICD-9 code for reimbursement purposes • In the new era of the EHR, a clinician chooses the most clinically meaningful ICD-9 code • Svetlana says: • In the bad old days, you knew the limitations of your data • In the new era, you discover new limitations of your data every day… if you’re lucky
The Pain: Vignettes • Asthma • Attention Deficit Hyperactivity Disorder • Varicella • Otitis Media
Asthma “Misclassification across time and space” • Common conditions are coded commonly, and reasonably well • 57,820 Patients billed for asthma care • 53,824 Patients with asthma on problem list • 54,993 Patients with at least 2 albuterol prescriptions • This is EXCELLENT correlation
Persistent Asthma • What about persistent asthma? • 16,949 Patients billed for “persistent asthma” • 11,943 With “persistent asthma” as a problem • But… • 23,673 Patients with at least 2 inhaled corticosteroid prescriptions • And… • Only 3,553 With persistent symptoms based on questionnaire • Huh?
Non-Random Misclassification By Care Location • “It is OK to compare organizations using their electronic data because everyone has the same problems with their data… the playing field is level” • Svetlana: Oh, really?
Non-Random Misclassification Over Time • And, the playing field changes over time • In 2004 one could have been lulled into a false sense of security over the reliability of encounter or problem list data… Actually, WE WERE!
Another Vignette: ADHD“Read The Chart”
Availability of ADHD “Phenotype” (N=57 Subjects)
ADHD: “Prospective ≠ Retrospective” • Initial explorations of data available for exposures of interest regarding ADHD were promising • Cohort of 12-23 month olds were examined in 2007
Retrospective “Exposures” • “Exposures” of interest often occur years before disease (e.g. ADHD patients are 6+ years old) • Data are that are well captured prospectively may not be available retrospectively
The Varicella Story: “What we have here is a failure to negate” • In our environment, routine data extracts are sent to public health officials for outbreak monitoring • Before universal immunization, Varicella was a common clinical diagnosis • Now classic presentations are rare and corroborating lab evidence is usually obtained by clinicians
Varicella: “h/o” ICD9 052 • Now that Varicella is rare, and coding is completed by clinicians seeking clinical relevance there are some “oopsies” • ICD9 codes beginning with 052 all imply “Varicella at this moment” • No ICD9 codes exist to say, “This is not Varicella,” or “Patient had Varicella in the past” • For rare conditions, deliberate use of a diagnosis code to mean the absence of a condition will exceed the use to mean the presence of a condition
Varicella: How to Make Your Health Department Unhappy • Health Department officials began chasing down cases of NON Varicella • In 2005: 39 non-cases vs. 34 actual cases • Families were confused, scarce public health money wasted, providers yelled at “Dr. Bob, why did you send that patient to school with Chicken Pox?” –Philadelphia Department of Public Health Official • Instead… Look for the labs!
Otitis Media: “There is hope… you can change the future” • Key process outcomes in otitis media include: • Documented laterality of disease (which ear) • Documented presence or absence of middle ear effusion and inflammation • Judicious use of antibiotics and watchful waiting • And a few others that we actually can measure today
Otitis Media • ICD9 codes and medication data could help, but rely on free text clarification • “ACUTE OTITIS MEDIA on RIGHT” • “WATCHFUL WAITING: begin in 2 days if fever/pain: Amoxicillin 400 mg PO BID” • “Floxin Otic to RIGHT ear…”
Otitis Media • Desperation brought us to inspect free text ear exam information available • 427,831 Visits for an ear related complaint inspected • 58,905 Unique ear exam descriptions were found
Otitis Media: Change the Data Collection Form • Must think about how to make the clinician want to use the new data capture tool • We are doing a comprehensive decision support intervention for this reason
The Path Ahead:The Misclassification Story • Once you have a research variable “well coded,” guard it with your life • Re-check validity if it has been idle for a year • Do not make colleagues re-invent measures • Once you have validated a measure, SHARE IT! (At least within your organization) • We use a wiki within our group for this purpose • But… your variable definition may not apply at another organization, even with the same EHR!
The Path Ahead:Read The Charts! • Resist the temptation of ICD9 and CPT… Or even SNOMED, LOINC, and RxNorm if you are so lucky • Start with the clinical workflow for a disease of interest, and consider all the clues that may exist in the electronic artifacts • You don’t know what you don’t know, until you read the charts • Find cohorts enriched in the disease, and read! • If still unsure, pound the pavement • Visit the locations of primary data collection
The Path Ahead:Prospective ≠ Retrospective • Just because data are available in a recent convenience sample does not mean it will be available for your real cohort • As much as possible, look at the actual cohort for preliminary data purposes • Avoid over re-use of preliminary data from one study for subsequent studies
The Path Ahead:Negation and ICD9 • Stating a negative with ICD9 nomenclature is not possible, yet clinicians want to (and will) express negatives using ICD9 codes • This is especially a problem for rare conditions • Look for corroborating evidence that arises from the workflow, especially lab tests or other observations that may suggest concern for the disease
The Path Ahead: Data Collection Nirvana • Changing data collection strategies in the EHR is possible • However, these modifications must be part of a comprehensive approach designed to improve the clinical workflow • You can’t get data for nothing
Thank You
