Objective Structured Clinical Examination (OSCE)

1. Objective Structured Clinical Examination(OSCE) Nadine J. Kaslow, PhD, ABPP nkaslow@emory.edu

2. Context ACGME Outcome Project – Toolbox of Assessment Methods Royal College of Physicians and Surgeons of Canada – CAN MEDS Assessment Tools Handbook

3. Examples of OSCE Stations1. resuscitation of a child2. consultation skills3. physical examination skills 1 3 2

4. Context Competency Assessment Toolkit for Professional Psychology High fidelity approach for summative assessment of myriad foundational and functional competencies

5. Context Competency Assessment Toolkit for Professional Psychology OSCEs are useful for assessing Readiness for practicum? Readiness for internship Entry level to practice Advanced credentialing

6. What is an OSCE? Objective – all candidates are presented with the same stimuli

7. What is an OSCE? Objective Structured – Specific foundational and functional competencies are tested at each station and the marking schema for each station is structured

8. What is an OSCE? Objective Structured Clinical Examination – Test of performance of clinical competencies, with an emphasis on skills and attitudes

9. What is an OSCE? Evaluation tool that allows people to be observed performing in many different clinical situations

10. What is an OSCE? OSCE combines Multiple observations Standardization of content Range of difficulty

11. What is an OSCE? Effective method to assess foundational competencies Professionalism (e) Scientific knowledge & methods (e) Reflective practice (e) Individual & cultural diversity (e) Relationships (o) Ethical & legal standards & policy (o)

12. What is an OSCE? Effective method to assess functional competencies Assessment (o) Intervention (o) Consultation (e) Research and evaluation (e)

13. Methodology Standardized patients (SP) Stations Marking scheme

14. Standardized Patients (SP) SPs are lay persons who are trained to portray scripted patient presentations accurately and consistently across many encounters SP encounters are credible and reliable

15. Standardized Patients (SP) SPs are used to teach Foundational competencies Interpersonal and communication skills (CIS) Clinical reasoning Functional competencies

16. Standardized Patients (SP) SP ratings of CIS provide a good proxy for actual patient satisfaction SP Communication Increases patient satisfaction Facilitates doctor-patient relationship Increases compliance Decreases malpractice claims

17. Standardized Patients (SP) Several encounters are required to obtain a reliable estimate of a person’s competence SP assessments are frequently organized as a set of cases or stations This series of encounters is known as OSCE

18. Implementation While there are variants on the original OSCE format, key implementation steps are Train SPs in the issues of encounter and ensure they don’t provide any information unless the person being assessed requests it

19. Implementation SP encounters or tasks requiring interpretation of clinical information must be provided at separate stations These must last 5-10 minutes and be observed by the assessor Topics – relationship development, risk assessment, diagnosis, therapeutic technique, consultation strategy

20. Implementation At each station, the person being assessed completes notes about the SP

21. Implementation The person being assessed moves between stations when an announcer indicates in accord with a specified time sequence Separate performance scores are generated for the task at each station based on input from the SP, the person being assessed, and the assessor

22. Implementation The scores from multiple informants across the stations or tasks are combined by the assessors to reach a final outcome determination

23. Role of Examiners - YES Observe the performance of the trainee at a particular task Score according to the marking scheme (get practice at marking) Contribute to the good conduct of the examination

24. Role of Examiners - NO Re-write the station Interfere with the SP’s role Design their own marking scheme Teach

25. Psychometrics With appropriate attention to design, OSCEs have acceptable psychometric properties including Good inter-rater, inter-station, and split-half reliability Good generalizability Strong content, construct and concurrent validity

26. Psychometrics OSCE scores correlate moderately with other performance indicators and other forms of evaluations

27. Psychometrics OSCE has increased reliability and content validity with greater number of stations and similarity between tasks at different stations

28. Strengths Provides for a high fidelity assessment

29. DOES Skills & Attitudes SHOWS HOW Professional Authenticity KNOWS HOW Knowledge KNOWS Miller et al., 1990

30. DOES SHOWS HOW OSCE KNOWS HOW KNOWS Miller et al., 1990

31. Strengths Measures clinical competence cross-sectionally using standarized means Focuses on observable behaviors Enables fairer peer comparison

32. Strengths Allows for assessing complex competencies without endangering patients’ well-being Encourages a collaborative assessment approach Has the potential for peer feedback and assessment

33. Strengths Has been extensively researched and found to be valuable for summative assessment Provides valuable information for curriculum review

34. Challenges Is challenging to create and administer Is labor intensive Is costly Only cost-effective when many individuals are examined at one administration

35. Challenges Requires high demand of other resources SPs Assessors Time commitment Physical resources

36. Challenges Doesn’t provide longitudinal assessment Doesn’t adequately tap complex skills requiring integrated professional judgment

37. Example: SP Information I am a 25 year old who comes to the Emergency Room for treatment of a 2-3 week old puncture wound that I can see but no one else can I am having a first episode of psychosis and have numerous alterations in my thinking and function; however it is not immediately obvious that my thoughts are disordered when the student begins taking my history

38. Example: SP Information Chief complaint - Stepped on a nail Opening statement - I stepped on a nail and the wound won’t heal – If pressed to say more… “It’s making me sick and I need help

39. Example: SP Information Appearance - alert and oriented, calm, cooperative, poor hygiene (dirty unmatched clothes, hair uncombed, no make up or grooming) Look like you haven’t showered in days Have some eye contact but less than normal Your affect is flat You talk very little and you have no emotion in your voice when you are talking

40. Example: SP Information Thought processes - you are focused on “saving America” and you mostly stay on topic when speaking but sometimes you start talking about one thing and it turns into a related topic but not on point An example might be “Coke advertisements transmit messages to me. The first coke slogan was ‘Drink Coca-cola’ released in 1886.” Refer to three Coke slogans during your interview. If possible say these when you can tangentially connect them to something else you are saying.

41. Examples

42. Conclusion OSCEs are a valuable and well-developed assessment approach in other health professions It behooves us as psychology to develop and share OSCE scenarios for the essential components of foundational and functional competencies

43. Conclusion OSCEs are advantageous to incorporate in our assessment armamentarium for summative evaluations particularly related to key transition phases Attention needs to be paid to increasing the feasibility of this high fidelity assessment methodology

44. Questions