Difficult to Assess Milestones

1. Difficult to Assess Milestones

2. . . . An Open Collaboration In Teaching and Assessment SaadiaAkhtar, MD, FACEP Program Director Beth Israel Medical Center Albert Einstein College of Medicine
 Rodney Omron, MD, FACEP Assistant Program Director John Hopkins University Lynn Roppolo, MD, FACEP Associate Program Director University of Texas Southwestern Parkland Health and Hospital System

3. Conflict of Interest • None to report

4. Objectives • Identify the challenges and obstacles to assessing these milestones • Create a more effective educational program to effectively teach these milestones • Develop assessment tools to appropriately evaluate residents in these milestones • Understand how to navigate the EM milestones wiki site

5. The Problem . . .

7. A simple but less defined assessment tool . . . . . .

8. . . . are now expanded into 23 sub- competencies!!!

9. Most faculty are convinced that the RRC-EM Milestones are a solution for which there is no known problem. -anonymous

15. Developing a roadmap . . .

16. Difficult Milestones Survey • 15-question survey, validated by 7 former PDs • Sent to CORDEM listserverin the Fall of 2013 • 62% (99/160) of EM residency programs responded • IRB approved study

19. Objectives (revised)

21. Begin with the end in mind . . .

22. Sub-competency 20: Problem Based Learning and Improvement (PBLI)

23. Sub-competency 20 (PBLI): Participates in performance improvement to optimize ED function, self-learning, and patient care.

24. EBM Practices and Expectations from EM Physician Educators • 98/157 CORD attendees surveyed • 71% do not use formal critical appraisal instrument • 75% do not have established EBM curriculum • 78% do not use any extramural sources (ACP, Best Evidence, EM Abstracts, or Annals of EM) Carpenter CR . Incorporating evidence-based medicine into resident education: a CORD survey of faculty and resident expectations. AcadEmerg Med 2010

26. Sub-competency 20: Most Common EBM Assessments Methods

27. Learning Basic EBM Principles • Journals: ACP Journal Club, EM Abstracts, Annals • Books • Emergency Medicine Decision Making by S. Weingart, 2006 • User’s Guide to the Medical Literature by G Guyatt, 2008 • Evidence-Based Diagnosis (Cambridge Medicine), TB Newman, 2009 • EBM: How to Practice and Teach EBM, 4th Edition, SE Straus, 2010 • Evidence-Based Physical Diagnosis: Expert Consult, S McGee, 2012 • Evidence-Based Emergency Care: Diagnostic Testing and Clinical Decision Rules by JM Pines and CR Carpenter, 2013

28. Learning Basic EBM Principles • Websites and podcasts • JAMAevidence: Using Evidence to Improve Care • http://www.dartmouth.edu/~library/biomed/guides/research/ebm-resources-materials.html • http://guides.mclibrary.duke.edu/ebm • https://www.med.emory.edu/EMAC/curriculum/diagnosis/maindiagnostic.html • http://www.annemergmed.com/content/ebemresources • http://adc.bmj.com/content/90/8/845.full.pdf • http://www.cebm.net/?o=1023 • http://bestbets.org/teaching/current.php • http://ktclearinghouse.ca/ • http://emjclub.com

29. Critical Appraisal Worksheetshttp://www.cebm.net/index.aspx?o=1157

32. CAT Maker and Critically Appraised Topics

33. Flipped Classroom: Teaching and Learning More Efficiently • Need to be information managers, not encyclopedias of knowledge. • Need to be life long learners… to ask questions and go out and find answers for themselves. • Need to search and filter vast quantities of information on the internet - “Free Open Access MedEd”(FOAMed) by way of blogs, podcasts, and videos, apps and mobile websites

34. Flipped Classroom Example • One hour of ‘individualized learning’ at home • Provide a clinical scenario and 3-5 questions for residents to answer in advance • Provide some electronic resources • Interactive discussion in classroom • Evaluate residents by written submission of their findings or conference participation

35. Fresno Test • 2 clinical scenarios with open ended questions • Must complete 4 key steps of EBM practice • 7 short answer questions, 2 mathematical calculations, and three fill-in-the-blank questions • The only validated, standardized, and objective measure of EBM competence currently available • 30 minute test • 13 page rubric

36. Fresno Test

37. Knowledge Translation Shift • Residents and an EBM faculty leader spend one shift per month in the ED answering EBM queries for all of the residents/faculty seeing patients.

39. Sub-competency 20: Self Assessment • 62% do a self-assessment tool or questionnaire • 61% discuss with residency leadership • 55% discuss with their advisor or mentor • 42% talk about it during clinical shifts • Most review semi-annually

40. Sub-competency 20: Process Improvement (PI) • 55% have PI project in the ED • 32% do a presentation at weekly conference • 28% have resident participate in a committee • 24% do not assess this

41. Sub-competency 21: Patient Safety (System Based Based Practice 1 or SBP1)

42. Sub-competency 21 (SBP1): Participates in performance improvement to optimize patient SAFETY.

44. Sub-competency 21- Tools to assess the Patient Safety Sub-competency

45. Sub-competency 21: Do you use any checklists or tools to assist with "Transitions of Care" such as the SBAR or Team Huddle?

50. After implementation of multiple medical team training programs: • Improved observed team behaviors. • Enhanced staff attitudes toward teamwork. • Reduced observedclinical errors. Morey, JC, Simon, R, Jay GD, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: Evaluation results of the MedTeams project. Health Serv Res. 37:1553-1581, 2002