Artificial Intelligence in Medicine HCA 590 (Topics in Health Sciences)

1. Artificial Intelligence in MedicineHCA 590 (Topics in Health Sciences) Rohit Kate 10. Clinical Decision Support Systems

2. Source and Reading • Chapter 20, Biomedical Informatics: Computer Applications in Health Care and Biomedicine by Edward H. Shortliffe (Editor) and James J. Cimino (Editor), Springer, 2006.

3. Clinical Decision Support • Medical practice is essentially medical decision-making • Expectations from computers: • What can computers do to support clinical decision-making? • How soon will be such tools be generally available? • How good will they be?

4. Types of Medical Decisions • Diagnosis: Analyze available data to determine pathophysiologic explanation for a patient’s symptoms • Diagnostic Process: What tests, procedures etc. to perform for diagnosis taking into account the financial costs and value of the results • Management: Treat the patient or allow the process on its own? What treatment to be used?

5. Role of Computers in Clinical-Decision Support • A clinical decision-support system is any computer program designed to help healthcare professionals to make clinical decisions • Programs must have • Access to accurate data • Extensive clinical background knowledge • Intelligent problem-solving capabilities which takes into account cost-benefit trade-offs and efficiency

6. Types of Computer Tools • Tools for Information Management: Information retrieval systems that can store and retrieve clinical knowledge • Do not help in applying that information to a task • Interpretation is left to the clinician • Tools for Focusing Attention: Flag abnormal values, alert about drug interactions, remind the user of overlooked diagnoses • Tools for Providing Patient-Specific Recommendations: Provide custom-tailored advice based on patient-specific data • We will focus on this here

7. Historical Perspectives: Leeds Abdominal Pain System • In late 1960s F. T. deDombal and his associates at University of Leeds developed computer-based decision aids using Bayesian probability theory • Their system used Bayes’ theorem to calculate probability of seven possible explanations for acute abdominal pain • Conditional independence assumption of the findings of various diagnoses • Mutual exclusivity of the seven diagnoses

8. Leeds Abdominal Pain System • Evaluation (1972) • Using the clinical and laboratory findings for 304 patients who came to emergency room with abdominal pain onset • Clinicians’ diagnoses were correct 65-80 % of 304 cases • The program’s diagnoses were correct 91.8 % of 304 cases • The system achieved widespread use in emergency department • Although never achieved the same degree of accuracy on other settings • Perhaps because of different probabilistic relationships between findings and diagnoses in different patient populations

9. Historical Perspectives: MYCIN • MYCIN program (mid-late 1970s) concentrated on management of patients who have infections • The developers doubted statistical methods for the purpose and so used rule-based methods • Knowledge of infectious diseases was represented as production rules • A production rule is a conditional statement that relates observations to inferences

10. A MYCIN Production Rule • Rule507 (English translation): If: • The infection that requires therapy is meningitis • Organisms were on seen on the stain of the culture • The type of infection is bacterial • The patient does not have a head injury defect • The age of the patient is between 15-55 years Then: • The organisms that might be causing the infections are diplococcus-pneumoniae and neisseria-meningitidis

11. MYCIN • MYCIN determined which rules to use and how to chain them to make decisions • Rules were in machine-readable format, English translation could be displayed • Could display its explanations to the user • System developers could modify the program’s knowledge structure by removing, adding or altering rules • May lead to unintended side effects

12. Evaluation of MYCIN • Evaluated for blood-borne bacterial infections and meningitis • MYCIN gave advice that compared favorably with that offered by experts • It was never deployed clinically, but paved way for research and development in 1980s • Helped in the surge of rule-based approaches in AI in the early 1980s

13. Historical Perspectives: HELP • An integrated hospital system developed at LDS Hospital in Salt Lake City, from 1970s • HELP has ability to generate alerts when abnormalities in the patient record are noted • Adds to a conventional medical-record system a monitoring program • Created and adopted a standard formalism for encoding decision rules known as Arden syntax • Arden syntax is a programming language that provides a canonical means for writing rules that relate patient situations to appropriate actions to be taken • Each decision rule is called medical logic module (MLM)

14. A Medical Logic Module in Arden Syntax Penicillin_order :=event {medication_order where class=penicillin}; /* find allergies*/ Penicillin_allergy := read last {allergy where agent_class = penicillin}; ;; evoke: penicillin_order;; logic: If exist (penicillin_allergy) then conclude true; endif; ;; action: write “Caution, the patient has the following allergy to penicillin documented:” || penicillin_allergy

15. HELP System • Whenever new data for a patient becomes available, the HELP system checks whether the data matches the criteria for invoking an MLM • The logic of MLMs has been developed by clinical experts • The output of MLMS include: alerts regarding drug reactions, interpretation of lab tests, calculations of the likelihood of diseases

16. Evaluation of HELP • Several studies in the 1980s demonstrated the beneficial effect of HELP at LDS Hospital • Demonstrated how integration of decision support with other system functions can increase its acceptance and use • Hospital systems have evolved towards more distributed architectures (several computers involved), HELP served as a model for decision support over an integrated data monitoring architecture

17. Lessons Learned from Early Decision-Support Systems • Clinical value of Bayesian diagnostic system demonstrated by Leeds abdominal pain system, subsequently more Bayesian systems were built, e.g. Pathfinder system for lymph-node pathology • MYCIN and HELP demonstrated the use of knowledge representation in encoding medical knowledge • Most early decision-support systems were rarely used in real practice and were viewed with skepticism but the attitudes are now changing

18. Changing Attitudes Towards Decision-Support Systems • Emergence of personal workstations and WWW along with easy-to-use interfaces • Recognition by the developers that their system must meet work practices of those who will use it • Growing amount of medical information • Fiscal pressure to practice cost-effective evidence-based medicine All this is leading to increasing acceptance of the ideas of computer-based decision tools This could change if the products won’t meet the logistical requirements of the clinical practice settings

19. Characterizing Clinical Decision-Support Systems Excellent decision-making capabilities alone do not guarantee system acceptance, there are a range of issues • System Function: Two categories • Determining what is true about the patient (usually diagnosis, e.g. Leeds abdominal-pain system) • Assisting in decisions about what to do (what tests to order, what therapy, whether to treat or not, e.g. MYCIN system) Majority of consultation physicians seek is about what to do rather than what is true

20. Characterizing Clinical Decision-Support Systems • Mode of Giving Advice: • Most assume a passive role, the system waits for the user to come to it, enter the data and request assessment, e.g. Leeds abdominal-pain system, MYCIN • Some play a more active role, e.g. HELP. This is possible because the system’s decision logic is integrated with patient information that is already being gathered from diverse healthcare sources A challenge: Avoid excessive number of warnings and alerts

21. Characterizing Clinical Decision-Support Systems • Style of Communication: • Consulting Model: The program serves as an advisor, accepting patient data, possibly asking questions and generating advice, e.g. MYCIN • Critiquing Model: Acts as sounding board for user’s own ideas, expressing agreement or suggesting alternatives • ATTENDING system: A standalone program that critiqued plan for anesthetic selection, induction and administration [Miller, 1986] • Hypercritic system [van der Lei & Musen, 1991]: Offered suggestions regarding management of patients with hypertension by doing behind-the-scene analysis of patient’s computer-based record • Such critiquing style meets physicians’ desires to formulate plan on their own but have them double-checked

22. Characterizing Clinical Decision-Support Systems • Underlying Decision-Making Process • Bayesian diagnosis programs: Have been shown to be accurate in selecting among competing explanations of a patient’s disease state • Earlier work assumed no conditional dependences • More recent work use general Bayesian networks which model conditional dependences as well • Decision Analysis: Adds to Bayesian reasoning the idea utilities associated with various outcomes

23. Characterizing Clinical Decision-Support Systems • Automated Medical Diagnosis: Use of statistical machine learning models like Artificial Neural Networks • Although accurate, there are issues about understanding the reasoning because the learned weights are opaque to human understanding • May be use rule-based machine learning methods in such situations, e.g. decision trees • Knowledge-based Systems: Symbolically encodes concepts derived from experts in a field in a knowledge base, e.g. MYCIN • Influenced by psychological studies of how medical experts perform problem-solving • The knowledge may also include probabilistic relations between symptoms and diseases, and qualitative relations such as causality and temporal relations

24. Characterizing Clinical Decision-Support Systems • Human-Computer Interaction • Logistical, mechanical and psychological aspects of system use • Besides good accuracy a system should • Have uncluttered and intuitive interface • Embedded within larger computer that is already in use by the users, making decision-support a by-product of the ordinary work practice instead of requiring an overhaul • Cumbersome to assess, slow, difficult to run, requiring moving somewhere else, lengthy interactions, manual reentry if information etc. dampen use of even a good decision-support system • Wireless networks, tablets, speech and gesture recognition offer good solutions

25. Construction of Decision-Support Tools • Acquisition and Validation of Patient Data • Keyboard entry, speech input, scannable forms, data monitoring systems • Healthcare workers state that their use of computers will be limited unless they are freed of the task of entering data • Lack of standardized ways of expressing most clinical situations in a form that computer can interpret • Although some controlled terminologies have been developed, e.g. ICD and SNOMED-CT, no coding system reflects all the nuances of physicians’ and nurses’ progress notes, a clear limitation on data • Even electronic medical records may not include all the patient-relevant data that the physician will see (e.g. job problems)

26. Construction of Decision-Support Tools • Modeling of Medical Knowledge • It is complex task to translate knowledge written in text to a structure appropriate for processing by a computer • Deciding what clinical distinctions and patient data are relevant • Identifying the concepts and relations between then that are relevant for decision-making • Model of problem-solving as well as clinical knowledge

27. Construction of Decision-Support Tools • Elicitation of Medical Knowledge • Rapid evolution of medical knowledge is making it difficult to develop and maintain medical knowledge bases • Computer programs have been developed that interact with the experts to acquire knowledge • Representation and Reasoning in Medicine • Although rules, frames etc. are useful, several challenges remain • Encoding three-dimensional anatomical knowledge • Performing spatial reasoning • Assessing temporal trends, models of disease progression • Modeling human skill of a “good clinical judgment”, understanding the psychology of human problem-solving is helpful

28. Construction of Decision-Support Tools • Validation of System Performance • How best to evaluate the performance of the decision-support tools • When gold-standard of performance exists, formal studies can be done but often it is difficult to define gold-standard, even experts disagree among themselves • Integration of Decision-Support Tools • There should be an effective integration with routine clinical tasks • Increasingly multiple small machines are used which are optimized for different tasks, decision-tools should be integrated with them through proper networking and interfaces

29. Illustrative Example: Internist-1 and QMR Project • Internist-1 was a large diagnostic program developed at the University of Pittsburgh in the 1970s • Contained knowledge of almost 600 diseases and 4500 findings (signs, symptoms, patient characteristics) • Decided against estimating conditional probabilities because some diseases are rare and not so well described in literature, instead used an ad hoc scoring scheme to relate findings and diseases

30. Internist-1 • One senior physician (50+ years of experience) and other physicians, medical students worked together considering each disease • Through literature review and case discussions determined a list of pertinent findings associated with each disease and scored the following • Frequency weight (1-5): How frequently the finding occurs with the disease • Evoking strength (0-5): How likely is the finding because of the disease • Import number (1-5) with each finding: The need to explain the finding

31. Internist-1 • The physician-user would enter an initial set of findings, and then the program would determine an initial differential diagnosis • The program would select appropriate questions to ask • Would recommend lab tests and diagnostic procedures after doing cost and benefits analysis • Could diagnose multiple diseases and did not make mutual exclusivity assumption as in Bayesian programs

32. Internist-1 Evaluation and QMR • 19 patients had a total of 43 diagnoses • Internist-1 identified 25 • Physicians identified 28 • Experts who presented the case identified 35 • In the 1980s, the program was adapted to run on personal computers as QMR (Quick Medical Reference) • QMR also served as: • Electronic textbook: Listing patient characteristics for a disease etc. • Medical spreadsheet: Obtain suggestions about coexisting diseases • Developers argue that electronic reference is more important than the consultation program

33. Illustrative Example: DXplain System • DXplain system developed at Laboratory of Computer Science at the Massachusetts General Hospital in the late 1980s • Given a set of clinical findings (signs, symptoms, laboratory data), DXplain produces a ranked list of diagnoses that might explain (or be associated with) the clinical manifestations • DXplain provides • justification for why each of these diseases might be considered • suggests what further clinical information would be useful to collect for each disease • lists what clinical manifestations, if any, would be unusual or atypical for each of the specific diseases. • Not intended to be used as a substitute for human clinician

34. DXplain • Has a large database of crude probabilities of over 4500 findings associated with 2000 different diseases • Adopts a modified form of Bayesian reasoning • Standalone version and also available over the Internet http://dxplain.org/dxp/dxp.pl • Used by a number of hospitals and medical schools, mostly for educational purposes but also for clinical consultation • The most extensively used decision-support tool today • Could be used as electronic medical textbook and a medical reference system

35. Illustrative Example: EON Architecture • EON constitutes a set of software components that must be embedded within some clinical information system • The components in EON are designed such that they can be mixed and matched to create different decision-support functionalities • Plugging in a knowledge-base of AIDS and HIV related disease, it becomes a decision-support system for AIDS (THERAPY-HELPER) • Plugging in a knowledge-base of breast cancer it becomes a corresponding decision-support system • The knowledge-bases are provided as an ontology, typically developed using a tool like Protege

36. Legal Issues • Formal legal precedents for dealing with clinical decision-support systems are lacking at present • Should the systems be viewed under negligence law or product liability law • It is unrealistic to expect systems to perform flawless, even physicians don’t perform flawless • May be potential liability borne by physicians who could have accessed such a program, and who chose not to do so, and who made an incorrect decision when the system would have suggested the correct one

37. Legal Issues • Several guidelines have been suggested for assigning legal liability to builders of knowledge-based medical decision-support systems or to the physicians using them • Validation of system before their release, evaluation is challenging and difficult to determine acceptable levels • Current policy of the Food and Drug Administration (FDA) in the United States indicates that such tools will not be subject to federal regulation if a trained practitioner is assessing the program’s advice and making the final determination of care • However, programs that make decisions directly controlling the patient’s treatment (e.g. closed-loop systems that administer insulin or that adjust intravenous infusion rates or respirator settings) are viewed as medical devices subject to FDA regulation

38. Future of Clinical Decision-Support • Concerns about cost and quality of patient care globally have altered the practice of medicine • Clinical practice guidelines based on empirical medical evidence are now ubiquitous • Decision-support systems will play a central role • Internet has greatly simplified information access • New pressures to learn best practices coupled with the ubiquity of information technology, have greatly encouraged the use of computer-based decision aids in health-professional schools around the world and this trend is likely to continue

39. Future of Clinical Decision-Support • Combining different reasoning methods to meet the specific requirements of increasingly complex decision-making tasks • Bayesian reasoners for performing probabilistic classification, rule-based methods for tasks such as planning or constraint satisfaction, and mathematical models for solving problems that can be best understood in terms of systems of equations • Heightened understanding of organizational behavior and of clinical workflow will stimulate a new generation of clinical information systems that will integrate smoothly into the practices of healthcare workers of all kinds • The very concept of a decision-support system itself will fade away as they blend into the infrastructure of healthcare delivery