Simulation For Emergency Medicine

1. Simulation For Emergency Medicine CORD Academic AssemblyMarch 4th, 2006

2. Steve McLaughlin, MDEM Program DirectorMedical Director ‘BATCAVE’ Simulation CenterUniversity of New MexicoMary Jo Wagner, MDEM Program DirectorSynergy Medical Education AllianceMichigan State University

3. Objectives Describe the current state of education and research in simulation. List the various simulators, mannequins and models available for emergency medicine training. Discuss the strengths and weaknesses of each simulation modality. List some of the best practice examples for using simulation in EM residencies.

4. Outline Introduction Spectrum of Simulation Equipment Best Practice Examples Hands-on Practice

5. Introduction “Simulation is the act of mimicking a real object, event or process.” “Simulation is a person, device or set of conditions which present evaluation problems authentically. The student responds to the problems as they would under natural circumstances.”

6. Introduction Characteristics Cues and consequences are like reality Situations can be complex Fidelity (exactness of duplication) is not perfect Feedback to users questions, decisions, and actions.

7. Introduction History 1928 Edwin Link develops first flight simulator – “Link Trainer” 1960 Laerdal introduces first “Resusci-Annie” 1968 “Harvey” cardiology simulator 1970 First power plant simulators 1973 First computer aided modeling of physiology 1975 Standardized patients and OSCE’s introduced 1988 First full body, computerized mannequin at Stanford 1989 ACRM – Anesthesia focused on patient safety and education movement at this time 1990 Term Virtual Reality was introduced, Screen Based Simulators Introduced

8. Link Trainer

9. Harvey

10. Introduction History 1990’s US IOM “To Err Is Human” report. 1994 Boston Center for Medical Simulation 1993 First national/international simulation meetings MMVR Late 1990’s Introduction of simulation into specialties like EM 1997 MIST VR Task Trainer 1998 AAMC MSOP 1991-1993 a total of 30 articles on High Fidelity Simulation 2000-1 Current generation of full body mannequins introduced by METI and Laerdal 2000-2003 a total of 385 articles on High Fidelity Simulation 2005 Society for Medical Simulation 2006 Simulation in Healthcare Journal

11. Introduction Why is this a valuable tool? Or is it? Learners can learn without risk to patient. Learning can be focused without regard to patient care needs/safety/etc. Opportunity to repeat lesson/skill to mastery. Specific learning opportunities? guaranteed. Learning can be done at convenient times. Performance can be observed/recorded.

12. Introduction Why is simulation important in medical education? Problems with clinical teaching New technologies for diagnosis/treatment Assessing professional competence Medical errors and patient safety Deliberate practice

13. Introduction CORD Consensus Conference Simulation is a useful tool for assess competence. Especially patient care, IP skills and SBP. There is a lack of evidence to support the use of simulation for high stakes assessment. Definitions of competence and tools to evaluate performance must be developed and tested. Scenarios and evaluation tools should be standardized.

14. Introduction ACGME ‘Toolbox of Assessment Methods’ Simulation is ‘the best’, ‘second best’ tool for assessing: Medical procedures Ability to develop and carry out patient management plans Investigative/analytical thinking Knowledge/application of basic sciences Ethically sound practice http://www.acgme.org/Outcome/assess/ToolTable.pdfhttp://www.acgme.org/Outcome/assess/ToolTable.pdf

15. Introduction LCME Requirements Allows simulated patients to count for student exposure to particular cases. RRC Requirements Allows simulated procedures to count for program/individual totals. Very helpful for rare procedures.

16. Introduction Simulation is one tool (new, expensive and exciting) in our educational repertoire. (Similar to lecture, case discussion, skill lab, MCQ, SP, etc.)

17. Outline Introduction Spectrum of Simulation Equipment Best Practice Examples Hands-on Practice

18. Available Simulation Equipment Standardized Patients Improvised Technology Screen Based Simulation Task Trainers Low/Mid/High Fidelity Mannequins Virtual Reality BEME articleBEME article

19. Evaluating Simulators Usability Validity Face, Content, Construct, Concurrent, Predictive Transfer Efficiency Cost Evidence

20. Standardized Patients Individuals trained to portray specific illness or behavior in a realistic and consistent manner for the purposes of teaching or assessment. Used in classroom setting, or without knowledge in clinical setting Especially useful to teach and assess communications and professionalism competencies in a standardized method.

21. Standardized Patients Initially started in the 1980’s Now - Association of Standardized Patient Educators http://www.aspeducators.org/sp_info.htm Required Clinical Skills testing for all students USMLE Part II CS exam

22. Standardized Patients Strengths Can consistently reproduce clinical scenario for standardized testing of learners Ability to assess rare conditions not otherwise reliably seen Patients trained to provide objective & accurate feedback Can use in real settings (arrive at office/ED as ‘real’ patient for realistic environment)

23. Standardized Patients Weaknesses Little research on effectiveness Most studies are from preclinical medical school education Few studies done with residents or practitioners and nearly all have small numbers (15-50) Cost to pay & time to teach standardized patients Quality of experience heavily dependent upon training of the patient and scenarios developed

24. Standardized Patients

25. Improvised Technology Models made of easily available items Closely mimic human tissue Allow for near replica of actual procedural steps Generally used for instruction of procedures Commonly used examples Slab of ribs to teach insertion of chest tubes Pigs feet or head for suturing practice Other examples in the literature Jello for vascular model Lasagna for split skin graft harvesting

26. Animal Models

27. Improvised Technology – Educational Theory Cognitive process for learning a procedure Understanding of indications, contraindications & complications Knowledge of equipment used for procedure Step-by-step knowledge of technical procedure Identifying anatomical landmarks and ‘tissue clues’ E.g. “pop” when entering dura or peritoneal cavity

28. Improvised Technology – Educational Theory Improvised Technology Useful to teach Knowledge of equipment Step-by-step knowledge of procedure Some ‘tissue clues’ Less useful for Anatomical landmarks “Greatest predictor of procedural competency … was the ability to sequentially order procedural steps” Chapman DM et al Open Thoracotomy Procedural…Ann Emerg Med 1996; 28:641.

29. Improvised Technology Strengths Cheap!!! Made easily available at all sites Easy to duplicate for repetitive use or numerous users Minimal instructor education needed Ability to create models otherwise not available Resuscitative Thoractomy

30. Improvised Technology Weaknesses Almost no research on effectiveness Less ‘real-life’ experience, therefore stress factor removed Often does not duplicate most difficult aspect of procedure (E.g. obese patient) Static devices , therefore useful for specific procedures only, not actively changing clinical scenarios

31. Examples – Vascular model

32. Examples – DPL model

33. Examples – Lumbar puncture model

34. Examples – Thoracotomy model

35. Examples – Thoracotomy model

36. Improvised Technology

37. Screen Based Simulation Desktop Computer Strengths – low cost, distance learning, variety of cases, improving realism, self guided Weaknesses – procedural skills, teamwork skills

38. Screen Based Simulation Laerdal Microsim www.Anesoft.com ACLS Critical Care Anesthesia Sedation Neonatal

39. Screen Based Simulation

40. Task Trainers Devices designed to simulate a specific task or procedure. Examples: Lap simulator Bronch simulator “Traumaman” Artificial knee

41. Task Trainers

42. Task Trainers Strengths High fidelity, good research on efficacy, may have self guided teaching, metrics available Weaknesses Poor haptics on most machines, expensive, focus on single task, not integrated into complete patient care

43. Task Trainers

44. Low Fidelity Mannequins Features: Static airways +/- rhythm generation No/minimal programmed responses. Strengths: Low cost, reliable, easy to use, portable Weaknesses: Limited features, less interactive, instructor required

45. Low Fidelity Mannequins Examples

46. Mid Fidelity Mannequins Relatively new class of mannequins, often used for ACLS training. Features: Active airways – ETT, LMA, Combitube Breathing/pulses, rhythms Basic procedures – pacing, defibrillation Some automated response and programmed scenarios

47. Mid Fidelity Mannequins Strengths: Active airways, somewhat interactive, moderate cost, moderate portability Weaknesses: Semiskilled instructor, limited advanced procedures (lines, chest tubes)

48. High Fidelity Mannequins Mannequin with electrical, pneumatic functions driven by a computer. Adult, child and newborn models Features: Dynamic airways, reactive pupils Heart sounds, lung sounds, chest movement Pulses, rhythms, vital signs Abdominal sounds, voice CO2 exhalation, cardiac output, invasive pressures Bleeding, salivation, lacrimation

49. High Fidelity Mannequins Procedures O2, BVM, Oral/nasal airway, ETT, LMA, Cric Pericardiocentesis, PIV Defibrillation, Pacing, CPR Needle or open thoracentesis TOF, Internal gas analysis Foley placement Reacts to medications

50. Features

51. Laerdal vs. METI Laerdal Instructor programmed physiology changes Windows Terrific Airway Reliability Ease of Use Cost: 35-45K METI Physiology modeled to respond to interventions Macintosh Drug Recognition Gas Analyzer Two Cost Levels ECS: 45K HPS: >150K

55. High Fidelity Mannequins Strengths Many dynamic responses, preprogrammed scenarios, widest variety of procedures, most immersive. Weaknesses Cost, procedures are not very realistic, reliability, lack of portability, significant instructor training required.

56. Mannequins

57. Virtual Reality Advanced form of human-computer interaction Allow humans to work in the computer’s world Environment understandable to us Four necessary components Software Hardware Input devices Output devises

58. Input and Output devices

59. Virtual Reality Types of VR applicable to medicine Immersive VR Desktop VR Pseudo-VR Augmented reality

60. Immersive VR

61. Desktop VR

62. Pseudo-VR

63. Augmented Reality

64. Virtual Reality

65. Outline Introduction Spectrum of Simulation Equipment Best Practice Examples Hands-on Practice

66. Research Rapidly expanding body of literature since 2000. First issue of ‘Simulation in Healthcare’ Jan 2006. Many articles on ‘look at what we did’ level and data that says ‘everyone thought it was nifty.’ Focus on best practices in teaching/learning and assessment using simulation.

67. Best Teaching Practices Screen based teaching with feedback is better than self study. Schwid, H. A., G. A. Rooke, et al. (2001). "Screen-based anesthesia simulation with debriefing improves performance in a mannequin-based anesthesia simulator." Teaching & Learning in Medicine 13(2): 92-6. We measured the effectiveness of screen-based simulator training with debriefing on the response to simulated anesthetic critical incidents. The intervention group handled 10 anesthetic emergencies using the screen-based anesthesia simulator program and received written feedback on their management, whereas the traditional (control) group was asked to study a handout covering the same 10 emergencies. All residents then were evaluated on their management of 4 standardized scenarios in a mannequin-based simulator using a quantitative scoring system. Residents who managed anesthetic problems using a screen-based anesthesia simulator handled the emergencies in a mannequin-based anesthesia simulator better than residents who were asked to study a handout covering the same problems.

68. Best Teaching Practices Comparing simulation to other teaching modalities demonstrates some slight advantages. Lee, S. K., M. Pardo, et al. "Trauma assessment training with a patient simulator: a prospective, randomized study." Journal of Trauma-Injury Infection & Critical Care. 55(4):651-7, 2003 Oct. Interns (n = 60) attended a basic trauma course, and were then randomized to trauma assessment practice sessions with either the patient simulator (n = 30) or a moulage patient (n = 30). After practice sessions, interns were randomized a second time to an individual trauma assessment test on either the simulator or the moulage patient. Within randomized groups, mean trauma assessment test scores for all simulator-trained interns were higher when compared with all moulage-trained interns. Use of a patient simulator to introduce trauma assessment training is feasible and compares favorably to training in a moulage setting.

69. Best Teaching Practices Simulation can be an effective replacement for live practice for some skills. Hall, R. E., J. R. Plant, et al. (2005). "Human Patient Simulation Is Effective for Teaching Paramedic Students Endotracheal Intubation." Acad Emerg Med 12(9): 850-855. Paramedic students (n = 36) with no prior ETI training received identical didactic and mannequin teaching. After randomization, students were trained for ten hours on a patient simulator (SIM) or with 15 intubations on human subjects in the OR. All students then underwent a formalized test of 15 intubations in the OR. When tested in the OR, paramedic students who were trained in ETI on a simulator are as effective as students who trained on human subjects.

70. Best Teaching Practices Learner centered teaching with simulation. Gordon, J. A. and J. Pawlowski (2002). "Education on-demand: the development of a simulator-based medical education service." Academic Medicine. 77(7): 751-2. Using the simulator, we wanted to create a medical education service-like any other clinical teaching service, but designed exclusively to help students fill in the gaps in their own education, on demand. We hoped to mitigate the inherent variability of standard clinical teaching, and to augment areas of deficiency. Upon arriving at the skills lab for their appointments, students would proceed to interview, evaluate, and treat the mannequin-simulator as if it were a real patient, using the instructor for assistance as needed. All students participated in an educational debriefing after each session. Customized, realistic clinical correlates are now readily available for students and teachers, allowing reliable access to "the good teaching case."

71. Best Teaching Practices Cheap may be as good as expensive. Keyser, E. J., A. M. Derossis, et al. (2000). "A simplified simulator for the training and evaluation of laparoscopic skills." Surg Endosc 14(2): 149-53. The purpose of this study was to compare a simplified mirrored-box simulator to the video- laparoscopic cart system. 22 surgical residents performed seven structured tasks in both simulators in random order. Scores reflected precision and speed. There were no significant differences in mean raw scores between the simulators for six of the seven tasks. A mirrored-box simulator was shown to provide a reasonable reflection of relative performance of laparoscopic skills.

72. Best Teaching Practices Team behavior can be effected by focused simulation experiences. Shapiro, M. J., J. C. Morey, et al. (2004). "Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum?" Quality & Safety in Health Care 13(6): 417-21. ED staff who had recently received didactic training in the Emergency Team Coordination Course (ETCC) also received an 8 hour intensive experience in an ED simulator in which three scenarios of graduated difficulty were encountered. A comparison group, also ETCC trained, was assigned to work together in the ED for one 8 hour shift. Experimental and comparison teams were observed in the ED before and after the intervention. The experimental team showed a trend towards improvement in the quality of team behavior (p = 0.07); the comparison group showed no change in team behavior during the two observation periods (p = 0.55).

73. Best Teaching Practices Innovative use of two new technologies helps to engage learners in a large group setting. Vozenilek, J., E. Wang, et al. (2005). "Simulation-based Morbidity and Mortality Conference: New Technologies Augmenting Traditional Case-based Presentations." Acad Emerg Med: j.aem.2005.08.015. The use of two separate technologies were enlisted: a METI high-fidelity patient simulator to re-create the case in a more lifelike fashion, and an audience response system to collect clinical impressions throughout the case presentation and survey data at the end of the presentation. The re-creation of the patient encounter with all relevant physical findings displayed in high fidelity, with relevant laboratory data, nursing notes, and imaging as it occurred in the actual case, provides a more engaging format for the resident-learner.

74. Best Teaching Practices Orientation Introduction to session Expectations What is real/what is not Self assessment Debriefing Evaluation

75. How To Best Use Simulation Provide feedback Give opportunities for repetitive practice Integrate simulation into overall curriculum Provide increasing levels of difficulty Provide clinical variation in scenarios Control environment Provide individual and team learning Define outcomes and benchmarks

76. Best Teaching Practices

77. Best Assessment Practices Simulation has some data to support its use as an assessment modality. Schwid, H. A., G. A. Rooke, et al. (2002). "Evaluation of anesthesia residents using mannequin-based simulation: a multiinstitutional study." Anesthesiology 97(6): 1434-44. 99 anesthesia residents consented to be videotaped during their management of four simulated scenarios on MedSim or METI mannequin-based anesthesia simulators Construct-related validity of mannequin-based simulator assessment was supported by an overall improvement in simulator scores from CB and CA-1 to CA-2 and CA-3 levels of training. Criterion-related validity was supported by moderate correlation of simulator scores with departmental faculty evaluations (0.37-0.41, P < 0.01), ABA written in-training scores (0.44-0.49, < 0.01), and departmental mock oral board scores (0.44-0.47, P < 0.01). Reliability of the simulator assessment was demonstrated by very good internal consistency (alpha = 0.71-0.76) and excellent interrater reliability (correlation = 0.94-0.96; P < 0.01; kappa = 0.81-0.90).

78. Best Assessment Practices Task trainers appear to be a valid method for assessing procedural competence. Adrales, G. L., A. E. Park, et al. (2003). "A valid method of laparoscopic simulation training and competence assessment." Journal of Surgical Research 114(2): 156-62. Subjects (N = 27) of varying levels of surgical experience performed three laparoscopic simulations, representing appendectomy (LA), cholecystectomy (LC), and inguinal hemiorrhaphy (LH). Years of experience directly correlated with the skills ratings (all P < 0.001) and with the competence ratings across the three procedures (P < 0.01). Experience inversely correlated with the time for each procedure (P < 0.01) and the technical error total across the three models (P < 0.05).

79. Best Assessment Practices Multiple simulated encounters are needed to accurately assess resident abilities. Boulet, J. R., D. Murray, et al. (2003). "Reliability and validity of a simulation-based acute care skills assessment for medical students and residents." Anesthesiology 99(6): 1270-80. The authors developed and tested 10 simulated acute care situations that clinical faculty at a major medical school expects graduating physicians to be able to recognize and treat at the conclusion of training. Forty medical students and residents participated in the evaluation of the exercises. The reliability of the simulation scores was moderate and was most strongly influenced by the choice and number of simulated encounters. However, multiple simulated encounters, covering a broad domain, are needed to effectively and accurately estimate student/resident abilities in acute care settings.

80. Best Assessment Practices Checklists scoring of videotaped performance can have a high degree of inter-rater reliability. Devitt, J. H., M. M. Kurrek, et al. (1997). "Testing the raters: inter-rater reliability of standardized anaesthesia simulator performance." Can J Anaesth 44(9): 924-8. We sought to determine if observers witnessing the same event in an anaesthesia simulator would agree on their rating of anaesthetist performance. Two one-hour clinical scenarios were developed, each containing five anaesthetic problems. Video tape recordings were generated through role-playing with recording of the two scenarios three times each resulting in a total of 30 events to be evaluated. Two clinical anaesthetists, reviewed and scored each of the 30 problems independently. The raters were in complete agreement on 29 of the 30 items. There was excellent inter- rater reliability (= 0.96, P 0.001).

81. Best Assessment Practices Validation that simulator performance correlates with real practice. Fried, G. M., A. M. Derossis, et al. (1999). "Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator." Surg Endosc 13(11): 1077-81; discussion 1082. Twelve PGY3 residents were given a baseline evaluation in the simulator and in the animal model. They were then randomized to either five practice sessions in the simulator (group A) or no practice (group B). Each group was retested in the simulator and in the animal (final test). Performance in an in vitro laparoscopic simulator correlated significantly with performance in an in vivo animal model. Practice in the simulator resulted in improved performance in vivo.

82. Best Assessment Practices ABEM type assessment tools measure performance equally well in oral or simulation environments. Gordon, J. A., D. N. Tancredi, et al. (2003). "Assessment of a clinical performance evaluation tool for use in a simulator-based testing environment: a pilot study." Academic Medicine 78(10 Suppl). Twenty-three subjects were evaluated during five standardized encounters using a patient simulator. Performance in each 15-minute session was compared with performance on an identical number of oral objective-structured clinical examination (OSCE) sessions used as controls. In this pilot, a standardized oral OSCE scoring system performed equally well in a simulator-based testing environment.

83. Best Assessment Practices There are many aspects of human knowledge/skills/attitudes to assess and the correct tool must be used for each one. Kahn, M. J., W. W. Merrill, et al. (2001). "Residency program director evaluations do not correlate with performance on a required 4th-year objective structured clinical examination." Teaching & Learning in Medicine 13(1): 9-12. We surveyed program directors about the performance of 50 graduates from our medical school chosen to represent the highest (OSCEHI) and lowest (OSCELO) 25 performers on our required 4th-year OSCE. OSCE scores did not correlate with Likert scores for any survey parameter studied (r < .23, p > .13 for all comparisons). Similarly, program director evaluations did not correlate with class rank or USMLE scores (r < .26, p > .09 for all comparisons). We concluded that program director evaluations of resident performance do not appear to correlate with objective tests of either clinical skills or knowledge taken during medical school.

84. Best Assessment Practices ”Softer” competencies like professionalism can be assessed with the aid of simulation technology. Gisondi, M. A., R. Smith-Coggins, et al. (2004). "Assessment of Resident Professionalism Using High-fidelity Simulation of Ethical Dilemmas." Acad Emerg Med 11(9): 931-937. Each resident subject participated in a simulated critical patient encounter during an Emergency Medicine Crisis Resource Management course. An ethical dilemma was introduced before the end of each simulated encounter. Resident responses to that dilemma were compared with a It was observed that senior residents (second and third year) performed more checklist items than did first-year residents (p < 0.028 for each senior class). Residents performed a critical action with 100% uniformity across training years in only one ethical scenario ("Practicing Procedures on the Recently Dead"). Residents performed the fewest critical actions and overall checklist items for the "Patient Confidentiality" case. Although limited by small sample size, the application of this performance-assessment tool showed the ability to discriminate between experienced and inexperienced EMRs with respect to a variety of aspects of professional competency.

85. Best Assessment Practices The scoring/evaluation system chosen to assess simulated performance is critical. Regehr, G., R. Freeman, et al. (1999). "Assessing the generalizability of OSCE measures across content domains." Academic Medicine 74(12): 1320-2. Students' scores from three OSCEs at one institution were correlated to determine the generalizability of the scoring systems across course domains. Analysis revealed that while checklist scores showed quite low correlations across examinations from different domains (ranging from 0.14 to 0.25), global process scores showed quite reasonable correlations (ranging from 0.30 to 0.44). These data would seem to confirm the intuitions about each of these measures: the checklist scores are highly content-specific, while the global scores are evaluating a more broadly based set of skills.

86. Best Assessment Practices Learners are smart and will figure out the game. Jones, J. S., S. J. Hunt, et al. (1997). "Assessing bedside cardiologic examination skills using "Harvey," a cardiology patient simulator ." Acad Emerg Med 4(10): 980-5. To assess the cardiovascular physical examination skills of emergency medicine (EM) housestaff and attending physicians. Prospective, cohort assessment of EM housestaff and faculty performance on 3 valvular abnormality simulations conducted on the cardiology patient simulator, "Harvey." Forty-six EM housestaff (PGY1-3) and attending physicians were tested over a 2-month study period. Physician responses did not differ significantly among the different levels of postgraduate training. Housestaff and faculty had difficulty establishing a correct diagnosis for simulations of 3 common valvular heart diseases. However, accurate recognition of a few critical signs was associated with a correct diagnosis in each simulation.

87. Best Assessment Practices Determine what you want to assess. Design a simulation that provokes this performance. Observe/record the performance. Analyze the performance using some type of rubric: checklist, GAS, etc. Debriefing, feedback and teaching.

88. Best Assessment Practices

89. Outline Introduction Spectrum of Simulation Equipment Best Practice Examples Hands-on Practice

90. Summary Simulation is one tool (new, expensive and exciting) in our educational repertoire. (Similar to lecture, case discussion, skill lab, MCQ, SP, etc.)

91. Summary Provide feedback Give opportunities for repetitive practice Integrate simulation into overall curriculum Provide increasing levels of difficulty Provide clinical variation in scenarios Control environment Provide individual and team learning Define outcomes and benchmarks Determine what you want to assess. Design a simulation that provokes this performance. Observe/record the performance. Analyze the performance using some type of rubric: checklist, GAS, etc. Debriefing, feedback and teaching.

92. Outline Introduction Spectrum of Simulation Equipment Best Practice Examples Hands-on Practice

93. References Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Issenberg, McGaghie, Petrusa, Gordon and Scalese. Medical Teacher, vol 27, 2005, p 10-28. Loyd GE, Lake CL, Greenberg RB. Practical Health Care Simulations. Philadelphia, PA. Elsevier-Mosby. 2004. Bond WF, Spillane L, for the CORD Core Competencies Simulation Group: The use of simulation for emergency medicine resident assessment. Acad Emerg Med 2002;9:1295-1299. ACGME Resources www.acgme.org/Outcome/assess/Toolbox.pdf www.acgme.org/Outcome/assess/ToolTable.pdf Accessed on Feb 2nd 2006.

94. Additional References 1. Glassman PA, Luck J, O'Gara EM, Peabody JW. Using standardized patients to measure quality: evidence from the literature and a prospective study. Joint Commission Journal on Quality Improvement. 2000; 26:644-653. 2. Owen H, Plummer JL. Improving learning of a clinical skill: the first year's experience of teaching endotracheal intubation in a clinical simulation facility. Medical Education. 2002; 36:635-642. 3. Pittini R, Oepkes D, Macrury K, Reznick R, Beyene J, Windrim R. Teaching invasive perinatal procedures: assessment of a high fidelity simulator-based curriculum. Ultrasound in Obstetrics & Gynecology. 2002; 19:478-483. 4. Kurrek MM, Devitt JH, Cohen M. Cardiac arrest in the OR: how are our ACLS skills? Can J Anaesth. 1998; 45:130-2. 5. Schwid HA, Rooke GA, Ross BK, Sivarajan M. Use of a computerized advanced cardiac life support simulator improves retention of advanced cardiac life support guidelines better than a textbook review. Critical Care Medicine. 1999; 27:821-824. 6. Rosenblatt MA, Abrams KJ, New York State Society of Anesthesiologists I, Committee on Continuing Medical E, Remediation, Remediation S-C. The use of a human patient simulator in the evaluation of and development of a remedial prescription for an anesthesiologist with lapsed medical skills. Anesthesia & Analgesia. 2002; 94:149-153, table of contents. 7. Gisondi MA, Smith-Coggins R, Harter PM, Soltysik RC, Yarnold PR. Assessment of Resident Professionalism Using High-fidelity Simulation of Ethical Dilemmas. Acad Emerg Med. 2004; 11:931-937. 8. Schwid HA, Rooke GA, Michalowski P, Ross BK. Screen-based anesthesia simulation with debriefing improves performance in a mannequin-based anesthesia simulator. Teaching & Learning in Medicine. 2001; 13:92-96. 9. Gaba DM, Fish K.J., Howard S.K. Crisis Management in Anesthesiology. New York: Churchill Livingstone; 1994. 10. Reznek M, Smith-Coggins R, Howard S, Kiran K, Harter P, Sowb Y, Gaba D, et al. Emergency Medicine Crisis Resource Management (EMCRM): Pilot Study of a Simulation-based Crisis Management Course for Emergency Medicine. Acad Emerg Med. 2003; 10:386-389. 11. Small SD, Wuerz RC, Simon R, Shapiro N, Conn A, Setnik G. Demonstration of high-fidelity simulation team training for emergency medicine. Academic Emergency Medicine. 1999; 6:312-323.

95. Additional References 12. Shapiro MJ, Morey JC, Small SD, Langford V, Kaylor CJ, Jagminas L, Suner S, et al. Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum?[see comment]. Quality & Safety in Health Care. 2004; 13:417-421. 13. Berkenstadt H, Ziv A, Barsuk D, Levine I, Cohen A, Vardi A. The use of advanced simulation in the training of anesthesiologists to treat chemical warfare casualties. Anesthesia & Analgesia. 2003; 96:1739-1742, table of contents. 14. Kyle RR, Via DK, Lowy RJ, Madsen JM, Marty AM, Mongan PD. A multidisciplinary approach to teach responses to weapons of mass destruction and terrorism using combined simulation modalities.[see comment]. Journal of Clinical Anesthesia. 2004; 16:152-158. 15. Kobayashi L, Shapiro MJ, Suner S, Williams KA. Disaster medicine: the potential role of high fidelity medical simulation for mass casualty incident training. Medicine & Health, Rhode Island. 2003; 86:196-200. 16. Kassirer JPaK, R. I. Learning Clinical Reasoning. First ed. Baltimore, MD: Williams and Wilkins; 1991. 17. Bond WF DL, Kostenbader M, Worrilow CC. "Using Human Patient Simulation to Instruct Emergency Medicine Residents in Cognitive Forcing Strategies". Paper presented at: Innovation in Emergency Medical Education Exhibit, SAEM Annual Meeting, . 2003; Boston. 18. Croskerry P. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Academic Emergency Medicine. 2002; 9:1184-1204. 19. Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them.[comment]. Academic Medicine. 2003; 78:775-780. 20. Boulet JR, Murray D, Kras J, Woodhouse J, McAllister J, Ziv A. Reliability and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology. 2003; 99:1270-1280. 21. Bond WF, Spillane L. The use of simulation for emergency medicine resident assessment. Academic Emergency Medicine. 2002; 9:1295-1299. 22. Byrne AJ, Greaves JD. Assessment instruments used during anaesthetic simulation: review of published studies. British Journal of Anaesthesia. 2001; 86:445-450. 23. Gordon JA, Tancredi DN, Binder WD, Wilkerson WM, Shaffer DW. Assessment of a clinical performance evaluation tool for use in a simulator-based testing environment: a pilot study. Academic Medicine. 2003; 78.

96. Additional References 24. LaMantia J, Rennie W, Risucci DA, Cydulka R, Spillane L, Graff L, Becher J, et al. Interobserver variability among faculty in evaluations of residents' clinical skills. Academic Emergency Medicine. 1999; 6:38-44. 25. Morgan PJ, Cleave-Hogg D, McIlroy J, Devitt JH. Simulation technology: a comparison of experiential and visual learning for undergraduate medical students. Anesthesiology. 2002; 96:10-16. 26. Schwid HA, Rooke GA, Carline J, Steadman RH, Murray WB, Olympio M, Tarver S, et al. Evaluation of anesthesia residents using mannequin-based simulation: a multiinstitutional study. Anesthesiology. 2002; 97:1434-1444. 27. Beaulieu MD, R. M., Hudon E, Saucier D, Remondin M Favreau R (2003). "Using standardized patients to measure professional performance of physicians." International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua 15(3): 251-9 28. Chapman DM. Rhee KJ. Marx JA. Honigman B. Panacek EA. Martinez D. Brofeldt BT. Cavanaugh SH. Open thoracotomy procedural competency: validity study of teaching and assessment modalities. Annals of Emergency Medicine. 28(6):641-7, 1996 Dec. 29. Cubison, T. C. S. and T. Clare (2002). "Lasagne: a simple model to assess the practical skills of split-skin graft harvesting and meshing." British Journal of Plastic Surgery 55(8): 703-4. 30. Davidson R, D. M., Rathe R, Pauly R, Watson RT (2001). "Using standardized patients as teachers: a concurrent controlled trial." Academic medicine : journal of the Association of American Medical Colleges 76(8): 840-3 31. Hance, J., R. Aggarwal, et al. (2005). "Objective assessment of technical skills in cardiac surgery." European Journal of Cardio-Thoracic Surgery 28(1): 157-62. 32. Maran, N. J. and R. J. Glavin (2003). "Low- to high-fidelity simulation - a continuum of medical education?[see comment]." Medical Education 37 Suppl 1: 22-8. 33. Nikendei, C., A. Zeuch, et al. (2005). "Role-playing for more realistic technical skills training." Medical Teacher 27(2): 122-6Clauser, B. E., S. G. Clyman, et al. (1996). "Are fully compensatory models appropriate for setting standards on performance assessments of clinical skills?" Academic Medicine 71(1 Suppl): S90-2. 34. Williams, R. G. (2004). "Have standardized patient examinations stood the test of time and experience?" Teaching & Learning in Medicine 16(2): 215-22