Assessment in Year 4 Manchester Medical School 2009 - 2010

1. Assessment in Year 4 Manchester Medical School2009 - 2010 Dr Chris Harrison

2. What I hope to cover today What are the Year 4 assessments? How are we trying to make them fairer? What are the examiners looking for? How to maximise your chances of passing How can we improve feedback? Your questions

3. Your assessments this year Progress Test January May OSCE November April SSCs Project Option

4. Progress by Year

5. Progress by Module

6. Progress by Focus of Question

7. OSCE Format 16 stations, 8 on each day 1 minute to read instructions 8 minutes inside the station Plus rest stations as usual

8. M + M OSCE Approximately equal balance of: Neurology Orthopaedics & Rheumatology Psychiatry All subjects occur on both days

9. M + M OSCE Types of questions: Histories Explanations Examinations Data interpretation / viva (few)

10. Focussed history-taking Focussed relevant history Reach an appropriate diagnosis How good are you at problem-solving? How well do you communicate?

11. Focussed history-taking Common errors: Decide on the diagnosis very early on, fail to consider other possibilities Fail to consider relevant risk factors or associated symptoms which will affect diagnosis or management Fail to respond to cues appropriately

12. Examination of patients Examine real patient with physical signs or healthy volunteer Present findings to examiner Reach diagnosis Answer any supplementary questions

13. Examination of patients with stable chronic disease Common errors: Find signs which aren’t there Miss signs which are there Draw wrong conclusions Try to force patients to fit into a textbook description You can’t learn this from books or the skills lab

14. Explanation and counselling Can you explain a disease, treatment or investigation? Can you handle questions from patients? Can you help a patient to reach difficult management decisions? As well as communicating sensitively, are you giving accurate information?

15. Explanation and counselling Common errors: Long monologues Trying to bluff it with lack of knowledge Excessive reliance on stock phrases instead of exploring patient’s concerns “I will ask my senior colleagues” “I will give you information leaflets” Excessive jargon

16. Viva stations Discussion with examiner: Data interpretation Eg X-rays, videos of psychiatric patients, complex psychiatric scenarios

17. Viva stations Common errors: Lack of knowledge Guessing incorrectly rather than admitting uncertainty Lack of systematic approach

18. How do we design our OSCE? We try to assess important topics How essential is it that an average Year 4 student is competent in this station? Essential Important Supplementary We aim for at least half the questions to assess essential material We aim to have few questions assessing supplementary material

19. How do we design our OSCE? We try to make it moderately challenging How challenging would an average Year 4 student find this station? Easy Moderate Difficult We aim for at least half the questions to be moderately challenging

20. How do we design our OSCE? We use rigorous quality control Internal exam design group We have been writing lots of new questions At least 3 external examiners review all questions to ensure the standard is the same as other medical schools We use assessment blueprinting To ensure fairness

21. OSCE Blueprint example

22. OSCE Blueprint example

23. How do we mark the OSCE questions? We divide each station into 5 key areas We also ask examiners to make a separate overall judgement about your competence We ask them to mark fairly but rigorously We check up on them

24. The mark sheet There are 3 parts: The ‘five sections’ The overall global mark Written feedback if student is clearly failing All sections are equally important

25. Checking for fairness We compare performance at different bases We ask external examiners to visit different sites to check fairness Senior internal examiners as well

26. What is the most reliable way to fail the OSCE?

27. Top 5 ways to fail the OSCE Find out what came up in last year’s OSCE and base revision around these Don’t waste time going on wards, clinic or general practice – stick to books and the skills lab Give an Oscar-winning performance Relax – I got through the OSCEs this year so I’ll be OK Don’t worry about knowledge – just bluff it

28. Aiming to be good enough is not good enough

30. Why should you aim for excellence? Ranking Prizes on your CV Future patients

31. How can we help you achieve excellence?

34. Our aim for Feedback Within 4 weeks of each exam: Detailed breakdown of each station Detailed feedback about group as a whole If you fail an individual station, ask your hospital if the examiner made any written comments

35. But feedback is not without its difficulties But feedback is not without its difficulties

36. Key Learning Points The exams test both knowledge and skills It’s important to spend lots of time with real patients Book work and the skills lab are a useful top-up Think: how will I manage this patient when I’m a doctor? Aim for excellence

37. Thank you andGood luck!