Richard F Davies Professor of Medicine University of Ottawa Heart Institute

1. Advanced Syndromic Surveillance and Emergency Triage (ASSET) Establishing Syndromic Surveillance in OttawaCRTI 06-0234TA Richard F Davies Professor of Medicine University of Ottawa Heart Institute

2. CRTI Norm Yanofsky NRC -IIT Charles-Antoine Gauthier, Janice Singer , Norm Vinson Joel Martin, Berry De Bruijn Ottawa Public Health Amira Ali Isra Levy Ottawa Heart Institute Team Susan McClinton Jason Morin Ottawa Hospitals SilvaCorp Stephen D’Silva CHEO eHealth research program Khaled El Emam Grey Bruce Public Health AMITA Corp Sonny Lundahl Monica Preston Anu Pinnamaneni PHAC CNPHI Project US Partners Michigan Dept of Community Health – Melinda Wilkins and Jim Collins Carnegie Mellon University Auton Lab –Daniel Neill Consultants Gini Bethel John Boufford Collaborators Kieran Moore – KFLA Health Unit Altarum Corp – Rick Keller ASSET Partners and Collaborators

3. The need for health care data • Health care is information rich and data poor. • 80/20 rule applies • 80% of time spent finding data • 20% of time spent working with it • Clinical Care • Also applies to population based care • Improving Quality of Care • Disease management initiatives • Health care reform • Population based surveillance

4. Syndromic Surveillance • Syndrome in Clinical Medicine • Def: “The sum of symptoms and signs of any morbid state.” • Classifies a patient based on the information at hand. • Guides triage, investigation and initial management in absence of a confirmed diagnosis. • Syndromic Surveillance • Routine monitoring of existing data streams for findings consistent with an event of interest. • Epidemiological syndrome – a cluster of findings potentially associated with a disease outbreak. • Medical records an important potential data source.

5. Rationale for Syndromic Surveillance: Earlier Detection Syndrome Diagnosed Illness EXPOSURE Number of Cases Notifiable Disease Reporting Syndromic Surveillance

6. Sample Chief Complaints • Tried to jump off two story house • Fish hook in ear • Fell and laid in yard for a couple of hours • Computer fell on head at school • Fell onto cactus • Hit face on furniture while playing hide and go seek in the dark • Rt foot pain, dropped turkey (2 days before Thanksgiving) • Run over by golf cart • “My kidneys are all off” • “I have fleas” • Blue plate special • Ingestion of windshield fluid • Candy stuck in his ear • Stuck bead in nose • Peanut in nose • Popcorn stuck in ear • Dog bite to eye lid • Squirrel bite • Hit top of head with hammer • Drank Vodka to prevent seizure • Talking to dead people • Odd behavior after eating fresh peaches

7. Text Mining • Using IT to guess information from text using statistics • Types of text mining: • Finding the right documents – e.g. Google • Finding facts in documents • This 75 year old male with NIDDM, known CAD and mild CRF presents with acute pulmonary edema.

8. Multiple Uses of Surveillance Systems for Detecting Disease Outbreaks • Possibility of Bioterrorism. • 100 kg anthrax, released over D.C., could kill 1-3 million and hospitalize millions more. • Emerging infectious diseases Potential impact of pandemic like H1N1, particularly if it became more virulent.

9. Emergency Room CBRN Attack Detection by Medical Records Surveillance (ECADS)

10. Symptom onset for 1335/1346 Cases Boil Water Advisory 160 140 120 100 Culture confirmed 80 Culture negative 60 Visits to ER by WalkertonResidents classified as GI Not tested 40 20 0 2 5 8 1 4 7 26 29 11 14 17 20 23 26 29 10 14 17 20 23 13 16 19 22 25 29 Visits to Walkerton ER classified as GI DAY

11. ASSET Project Inception • Enthusiastic response to ECADS by Public Health • Dave Salisbury, Isra Levy • Amira Ali • Partnership • NRC – IIT (Federal Lead) • ECADS Partners • 5 Ottawa area hospitals • PHAC CNPHI Team • AMITA

12. Project Champion Mr. Charles-Antoine Gauthier NRC-IIT Portfolio Manager Mr. Norman Yanofsky CRTI Project Manager Dr. Janice Singer NRC-IIT Deputy Project Manager Dr. Richard Davies UOHI Deputy Project Manager Mr. Sonny Lundahl AMITA Corporation Health Care/Public Health Team Technical Team ASSET Organization

13. ASSET Objectives • Establish a successful Syndromic Surveillance system deployment in Ottawa • Provide methods to improve the adoption, usability and ongoing operations of Syndromic Surveillance in Canada • Deliver a system that is ready for deployment anywhere in Canada • Interface locally collected Syndromic Surveillance data with Canadian Network for Public Health Intelligence (CNPHI) • Provide response protocols suitable for Canadian cities

14. ASSET Non-Technical Development • Establishment of Data Sharing Agreements • Project • Ottawa Public Health • Participating Hospitals • Researchers • Dealing with Privacy and Confidentiality • Approach to accessing and using health data for surveillance • Contribution of Dr Khaled El Emam re: reidentification risk • Development of Response Protocols • Partnership

15. ASSET Technical Development • Deployment • The Ottawa Hospital (Civic and General Campuses) – established • CHEO and Queensway Carleton – established • Montfort (French Language) - September 2009 • Development • Better data • Better categorization • Better output and utility • Deploy-ability • Use of EMPI data feeds • Collects ADT data from the 18 hospitals in the Champlain LHIN • CHEO, QCH and Montfort to add presenting complaint data to the EMPI data feed mechanism

16. ASSET Development: Contribution of NRC-IIT • Human-Computer Interaction (HCI) • Increase efficiency and effectiveness by improving the interface between the user and computer • Determine public health’s syndromic surveillance needs • Develop & test interface designs that meets those needs • Improve understanding of epidemiologists’ decision processes • Develop evaluation methods for S2 systems.

17. ASSET Development: Contribution of NRC-IIT • Better text categorization • Support French and English input text • Allow users to define new output syndromes • Allow multiple syndromes per case • Handle various reporting styles • Adapt to gradual drift in styles • Allow new document sources • Ongoing measurement of text categorization accuracy

18. ASSET Project Time Lines Key Dates Deployment of Influenza-Like Illness (ILI) Watch to Ottawa Public Health and Grey Bruce Health Unit Sep, 2009 Oct, 2009 Oct, 2009 Study/Stakeholder Meeting 4 Feb, 2009 ASSET Version 1 Evaluation completed Mar, 2010 Mar, 2010 Response Protocols completed Mar, 2010 ASSET Project completed

19. ASSET ILI Watch

20. Project Status at H1N1 Onset • ECADS running in Grey Bruce • ASSET Project • ASSET V1 in Development • TOH on line • CHEO Pending • QCH Pending • Montfort awaiting ASSET V1 with French Language Classifier • Data Fusion • Award announced, Charter under development

21. Initiation of ASSET ILI Watch • Week of April 21 • first reports of “Swine Flu” from Mexico • Discussion with Grey Bruce and Ottawa Public Health • Confirm additional data to monitor ILI would be very useful • April 27 Conference Call • ASSET/Data Fusion Partners • Agree to focus on adapting S2 to monitoring ILI. • May 5 – meeting with CRTI • Endorse ASSET ILI-Watch initiative • Provide resources • Request ASSET charter modification

22. ASSET ILI Watch: Challenges Data accessibility Data Quality Case Classification System output and end user support

23. ASSET ILI Watch Data Accessibility • Status • Data Availability Owen Sound Hospital and TOH • Rights to access • Ottawa and Grey Bruce Public Health • Privacy and confidentiality framework in place • Technical Solution: AMITA • Results • Outbreak hits media – week of April 20 • First ASSET ILI Watch conference call – April 27 • Request made to AMITA for data extraction • First manual extraction for OPH – April 28 • First manual extraction for GBPH – April 29 • Automated extraction of Q6H cumulative data files • Deployed for OPH – April 30 • Deployed for GBPH – May 1

24. ASSET ILI Watch - Data Quality Status April 27 2009

25. CEDIS Presenting Complaints Respiratory Codes Allergic reaction Stridor Wheezing – no other complaints Apnic spells in infants Other and unspecified abnormalities of breathing • Shortness of Breath • Respiratory arrest • Cough / congestion • Hyperventilation • Hemoptysis • Respiratory foreign body • Problems: • Poor specificity • In patients with multiple complaints only one is captured

26. ASSET ILI Watch: Grey Bruce Data Quality

27. ASSET ILI WatchImproving data Quality at TOH • Solution • Free text final diagnosis is available in OACIS • Advantage – very sensitive and specific • Disadvantage – 2-4 hour delay • CTAS • Each patient categorized I – V • Provides indication of degree of illness • Results • Request to TOH and AMITA – April 30 • First conference call between TOH and AMITA – April 30 • Design, build and testing of modification to feeds – May 1-13 • Testing on TOH Staging Server – May 14-15 • Deployment to production – May 19

28. ASSET ILI WatchImproving Data Quality • Tasks pending • Evaluate quality of free text CC at Queensway • Access and evaluate free text CC at CHEO • Available on separate IT system • Needs to be accessed and linked to EMPI data feed • Evaluate quality of free text CC at Montfort

29. ASSET ILI Watch Case Classification Status as of April 27 2009 • Grey Bruce and TOH on line using RODS Complaint Coder. • Separate Respiratory, Constitutional and GI syndromes. • No Multiple Syndromes • Misclassification (Nausea + resp. distress = GI) • Competition (ILI may be Constitutional or Respiratory) • No distinct ILI syndrome • NRC Classifier developed for ASSET, not yet integrated with RODS application. • Solution • Apply NRC classifier directly to raw data feed • Integrate processing with automated reports

30. NRC ILI Watch Classification Results Comparing SVM, Naive Bayes and CoCo Rules

31. SVM vs. Naïve Bayes • Naïve Bayes (NB) assumes independence of features, Support Vector Machine (SVM) does not. • NB will classify “Urinary Tract Infection (UTI)” as an ILI • UTI often occurs with fever, weakness etc. • NB sees the association but can’t interpret the connection. • SVM can learn that UTI (or even “Fever plus UTI”) is not an ILI • SVM should improve sensitivity and specificity, particularly when complaints have multiple features.

32. Advantages of having a Specific ILI Syndrome • Non-ILI Respiratory Syndromes • Hockey injury – trouble breathing • COPD • Cough • Wheeze • Asthma • Sore ears • Sinus Infection • “Croupy” • Non-ILI Constitutional Syndromes • Shaking • Not feeling well • Lightheaded • Dizziness • Weak and Shaky • Syncopal Episode

33. Hypotheses SVM will perform better than NB A system trained on a specific ILI Syndrome will perform better than one using a combination of Respiratory, Constitutional and GI syndromes.

34. Methods • 52,840 records from 12 Grey Bruce Region hospitals from Dec 30 2008 – April 30 2009 • 10,000 Chief Complaints classified by expert. • Any case that could be ILI was counted as ILI. • Naive Bayes and SVM evaluated • NRC text classification software • 10-fold stratified cross validation. • Results compared with ILI rules based on CoCo classifications.

35. CoCo vs. NB - ILI vs. SVM - ILI

36. Results *ILI Rule based on RODs classifiers ** Similar to CoCo re-trained on ILI syndrome

37. Caveats and Conclusions • Grey Bruce data extremely high quality • Very consistent • Few misspellings. • Often contain multiple symptoms and precise information. • Cases with any component of ILI counted as positive. • System not asked to make complicated distinctions • The high PPV is potentially very valuable. • Reduces false positives • Increase confidence in system

39. Workflow and End User Support Status as of April 27 2009 • Public Health Epidemiologists coping with dramatic increase in workload because of H1N1 • Data access via RODS Interface difficult • RODS syndromes possibly related to ILI (Resp, Constitutional, GI < 5 yrs old) must be assessed separately • “Black Box” quality of existing systems • No ILI syndrome • Limited statistical repertoire • Inability to easily do follow up and ad hoc analyses

40. ASSET STAT • ASSET development aimed at making SS data more useful to epidemiologists • Concept: Provide categorized data for COTS Statistical package (STATA, SPSS, SAS). • Advantages • COTS packages statistical capability >>> S2 applications. • End users already familiar with these packages. • Training transferrable to their other work. • Support resources readily available. • End user community already established. • Highly developed scripting and integration tools.

41. ASSET ILI Watch Reports • Automated analysis of S2 data using COTS Statistical and Graphing Packages • ASSET ILI Watch Report generated automatically at pre-defined intervals. • Automated data feed. • Categorized using NRC classifier. • Processed automatically in COTS Statistical/Mapping Package • Automated follow up report • Public health identifies high risk FSAs from ILI Watch Report • Automated follow up reports focusing on these areas. • Ad hoc analyses • S2 data easily accessible to Public Health for analysis using • Data set loaded and “ready to go” • Template scripts • Central support

42. Data Fusion for Monitoring CBRNE Threats (DF-Surveillance) • CRTI R&D project • Objective • Create reusable framework to apply S2 technology to new problems. • Technological • Non-technological • Adaptive Process framework • Stakeholder engagement • Response protocols • Data source identification • Feature extraction • Data access and use (privacy, security, jurisdictional data sharing) • Prototype using specific new applications of S2 technology • In hospital outbreak of serious infection (e.g. SARS) • Track events related to illicit drug use • Monitoring of high risk event

43. DF-Surveillance: New Partners • DRDC Valcartier Decision Support Systems for Command and Control Section (Patrick Maupin) • Advanced Research Project 13qd "Sensor Networks for Critical Assets Protection and Surveillance in Support of Air Expeditionary Wing Operations (CAPS)" PM : Patrick Maupin, DRDC-Valcartier • Health Canada Office of Controlled Substances (Suzanne Desjardins) • Ontario Agency for Health Protection and Promotion (Natasha Crowcroft, Rachel Savage)

44. Keys to Long Term Success • Teamwork and collaboration • Better solutions • Maximal efficiency • Mutual Benefit • Stakeholder Engagement • Broader deployment • Sustainable funding • Broader expertise • Visibility and public confidence • Better Data, and Improved Analytic Capability • Improve accuracy and quality of information • Broader capabilities • Improved situational awareness and response

45. Thank you Questions and Comments