USMLE Step 2 Clinical Skills

1. USMLE Step 2 Clinical Skills ASPE Webinar December 10, 2003 Richard Hawkins, Ann King National Board of Medical Examiners

2. Introduction: USMLE Step 2 CS Step 2 CS design (content, structure, scoring) General policies Exam logistics (dates, exam sites) Effects on medical education Overview

3. Serves the licensure process Independent verification of competence Educational experiences vary Co-sponsored by FSMB and NBME Step 2 CS  collaboration with ECFMG Includes essential attributes for practice Cognitive and clinical skills required for safe and effective patient care USMLE Purpose

4. “CSE” – enhancement of Step 2 USMLE Step 2 – components: Clinical knowledge (CK) Clinical skills (CS) Steps 1 & 2 prerequisite for Step 3 USMLE Step 2 CS

5. Step 2 Clinical Skills assesses whether an examinee has the clinical skills essential to the safe and effective practice of medicine, with a focus on those clinical presentations that are common to patient care under supervision… Purpose: USMLE Step 2 CS

6. Defines content categories Individual test form: Adequate sampling of content domain Comparable content between test forms Step 2 CS Blueprint

7. Common and important medical problems / patient presentations Acuity Age Gender Race / ethnicity CSE Blueprint: Content Categories

8. Case Content Cardiovascular Respiratory Gastrointestinal Musculoskeletal Constitutional Neurological Psychiatric Genitourinary Women’s health Other Case Acuity Acute Subacute / Chronic Test Form Patient age Age less than 18 Age 18 – 44 Age 45 – 64 Age 65 + Patient Gender Male Female

9. 12 patient encounters 15 min. per encounter / 10 min. for patient note Each encounter: Elicit pertinent history Perform appropriate physical examination Communicate effectively Document: Findings from the history and physical Diagnostic impression / Further work-up Step 2 CS Structure

10. Other station formats: Third party interviews Telephone encounters Physical examination stations Future formats: Difficult or sensitive communication issues Synthetic models and mannequins Step 2 CS Structure

11. Integrated Clinical Encounter (ICE) Data gathering: History and physical exam Patient Note Communication / Interpersonal Skills (CIS) Gathering/sharing information, manner, rapport Spoken English Proficiency (SEP) Listener effort, examinee pronunciation / word choice Step 2 CS Structure: Score Components

12. In order to Pass Step 2 CS, examinees required to pass all three components: ICE, CIS, SEP Those who fail and then retake: Reassessed in all three components Must pass all three Scoring - 2

13. Step 2 CS system will be comparable to that used for the rest of USMLE Decision ultimately made by the Step 2 Committee Will establish minimum passing point for ICE, CIS, and SEP separately Will consider data from multiple sources Survey of stakeholders Independent review of content/encounters Performance/reliability data Closely monitor Standard Setting

14. Standard Setting  summer – fall 2004 Adequate numbers of examinees required Initial score reports  fall 2004 Score Reporting

15. To Examinees Overall pass/fail outcomes Failing examinees: graphical representation of relative strengths and weaknesses (similar to current USMLE reports) Score Reporting - 2

16. Sample PerformanceReport

17. Performance Profile Lower Performance Borderline Performance Higher Performance Integrated Clinical Encounter Data-gathering Patient note Communication / Interpersonal Skiills Spoken English Proficiency XXXXXXXXXXXXXXXXXX xxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxx XXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXX

18. To Schools Individual student pass/fail outcomes If student permits Group summary reported yearly Including CS components Score Reporting - 3

19. Who must take Step 2 CS? Those who graduate in 2005 or later Earlier grads if Step 2 CK not passed by June 30, 2005 What are the eligibility requirements? (no change) Student/grad of LCME- or AOA-accredited school Is there a required sequence? (no change) Step 1, Step 2 CK, Step2 CS in any order Rules for repeaters (no change) Wait 60 days between attempts, no more than three times in 12 month period, no USMLE imposed limit on number of takes (but the licensing authority may have limits) General Policies (for US/Canadian Students)

20. First exam administered in June 2004 Collaboration with ECFMG Step 2 CS replaces ECFMG CSA for IMGs Delivered at 5 regional test centers Exam Logistics

21. Current ECFMG CSA test sites Philadelphia Atlanta Under Construction Chicago Los Angeles Houston Test Centers

22. Centers open in sequence First Center – June 2004 All Centers open – September 2004 Exact dates and sequence TBA Capacity 3 examinations/day (33 examinees) Up to 7 days/week Equal access to all Centers by US, Canadian and international examinees Exam Administration Schedule

23. Purpose Realistic experience for SPs Test AV and IT systems Train support staff Scope 6-8 weeks prior to implementation at each site Approximately 400 examinees at each site Pre-Implementation Pilot Exams

24. Open to all registered examinees Mechanisms for allocating slots under development Mock examinations very similar to actual CS Experience case design, SP interaction, examination logistics, Center Limited feedback, not meant to be predictive Mix of US, Canadian and International Examinees Scheduling details TBA Participation in Pilots

25. Registration and scheduling will become available concurrently in early January Students and graduates of US and Canadian medical schools will register using the NBME Interactive Website for Applicants and Examinees (telephone registration available) Detailed information on registration, scheduling and fee structure on USMLE Website Registration & Scheduling

26. Most significant impact at UME level Consequence of: Effect of assessment on education Placement of CSE in USMLE series CSE: Effect on Medical Education

27. Accountability for clinical skills acquisition (public and students) Identifies need for national consensus on objectives for clinical skills teaching and assessment CSE: Effect on Medical Education

28. Central role of the patient in medical education Clinical skills – curricular reform / emphasis Implications for resource allocation Faculty participation Enhanced faculty interest and involvement Support for faculty development CSE: Effect on Medical Education

29. Challenge for Medical Educators: Ensure minimal standard (“teach to the test”), or… Think beyond minimum standards Don’t miss the opportunity to: Fully explore clinical skills teaching / assessment Inspire curiosity and enthusiasm for learning around patient encounters Facilitate development of lifelong learning skills related to patient encounters… CSE: Effect on Medical Education

30. Will assume a more significant role Valued as experts: SP, clinical skills Role as a consultant More influence of educational programs With recognition, comes responsibility Standardized Patient Educators

31. For Residency Program Directors: Improved applicant selection Decreased time with problem residents Better foundation on which to build Continued exploration and application of SPs and other simulation methods: Advantages: faculty sparing, patient (and student) safety, ensure broad exposure… Applications across continuum of education and practice CSE: Other Implications

32. For the FSMB, NBME and ECFMG: Values statement Generation of doctors will understand the relevance and importance of clinical skills Relationship between NBME and ECFMG and “Medical Education” will continue to grow Implications for the public are significant Implications for the Profession