1 / 44

Workplace-based Assessment and Feedback

Workplace-based Assessment and Feedback. Dr. Kevin West Director of Examinations and Assessment. Why change?. “Something must be done!” Training more geared to “Good Medical Practice” “Bad” doctors must be weeded out before let loose on public Assessments other than exams necessary.

Télécharger la présentation

Workplace-based Assessment and Feedback

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Workplace-based Assessment and Feedback Dr. Kevin West Director of Examinations and Assessment

  2. Why change? • “Something must be done!” • Training more geared to “Good Medical Practice” • “Bad” doctors must be weeded out before let loose on public • Assessments other than exams necessary

  3. PMETB/GMC and WBA • PMETB established by Parliament 2003 • Started operation 30th September 2005 but groundwork already underway • Established standards for assessment • Under GMC from April 2010

  4. PMETB strategy on assessment • PMETB principles for assessment 2004 • Nine principles for assessment systems • Now rewritten as 17 standards for curricula and assessment systems • Principles based on “overarching strategy” • Workplace-based assessment • Examinations of knowledge • Examinations of clinical skills • Specialty curricula and assessment systems now approved by GMC

  5. Value of multiple assessments • Key is several modes of assessment – triangulation • A single exam system does not provide adequate coverage of clinical practice • WBA is not enough on it own (Assessment System Guidance – PMETB)

  6. The College Assessment system: Miller’s Pyramid WBA – MSF, CbD Does Action WBA e.g. Mini-CEX, DOPS Shows FRCPath Part 2 Practical Performance Knows how Competence Knows FRCPath Part 1 Knowledge

  7. Workplace-based Assessment • Assessment of performance in the workplace and measured there • Assessed locally by supervisors as well as staff of all grades • Assessment occurs during normal work immediately followed by feedback so trainee receives continuous feedback on their routine performance • Results recorded in trainee portfolio for annual review - important for trainee development • Starts the process of CPD and revalidation

  8. Workplace-based Assessment • The College uses workplace-based assessments similar to other specialties with some adaptations • Only Chemical Pathology and Medical Microbiology use Mini-CEX • DOPs and CBD are relevant to all specialties • Histopathology pioneered Evaluation of Clinical Events for assessment of trainee interactions with other medical teams • The College uses a specialty-specific MSF

  9. Essential elements of WBPA • Requires assessment by as many observers as possible for reliability (at least 7 for MSF and 6-8 for other tools) • Aim is to show progression so repeating a failed assessment is perfectly acceptable but best if both recorded and available (doesn’t happen in practice) • Aim is to show satisfactory progression through stages of training to full professional outcome • Requires professional judgement

  10. Problems with WBA • “I just wanted to ensure (the trainee) had sufficient satisfactory assessments to ensure I didn’t need to do any more” • “I don’t want to be responsible for their failing this year and repeating it” (here) • “I don’t feel I can judge whether it was satisfactory or not” • “I’ll reserve my comments for the educational supervisor’s report”

  11. College Assessment System • Workplace-based assessment • Year 1 assessment • FRCPath Part 1 (test of knowledge) • FRCPath Part 2 (test of skills and applied knowledge) • Feedback on regular basis (WBA) • Educational supervisor’s report • ARCP (Annual Review of Competency Progression)

  12. Examinations and Assessment • Examinations are one form of assessment • Assessments are: • formative (involve feedback and intended to guide training) or • summative (assess whether reach given standard) e.g. driving test, FRCPath • All assessments should be “blueprinted” to the curriculum to demonstrate that the most appropriate method is being used

  13. How Workplace-based Assessment fits with FRCPath • Workplace-based assessment depends on assessment by local staff, examinations rely on assessment by those who have had no contact with the trainee • Examinations provide an independent assessment of knowledge and practical skills to national standards • Workplace-based assessment provides a local insight into trainees’ performance in routine tasks and interaction with staff (and patients)

  14. How Workplace-based Assessment fits with FRCPath • WBA should provide insight into attitudes and functioning day to day – but does it succeed? • Workplace-based assessment is a local process governed by deaneries to GMC and College standards • Comments, complaints and appeals about WBA go to deanery • WBA is a continuum with F1 and F2 assessment and is being used in medical schools

  15. Workplace-based Assessment – Advantages • Measurement occurs in more authentic environment • Assessment may be retrospective review of actual routine day to day performance • Trainees can take assessment in their own time • The assessment tools should provide extensive feedback to the trainee (formative) • Use of computerised forms should make them easy to record

  16. Workplace-based Assessment – Disadvantages • Computerised forms can over-simplify • Do we recognise excellence? • A robust rating can only occur if there are a number of independent observers – which can be difficult in small specialties • Assessors may be unwilling to be critical of trainees they interact with on a day to day basis - are we just too nice? • Summative v formative – more later

  17. Annual progression during training

  18. The Training Environment • Workplace-based assessment utilises the skills of a wide variety of professional groups as assessors: medical, nursing, BMS, clinical scientist, ancillary staff • The College cannot train or quality assure all assessors • Training centres are recommended to develop a training environment where those with the skills needed for assessment are recognised

  19. Undertaking assessments Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

  20. “Overview of WBA” document • Customised for each specialty • Similar content • Introduction to WBA • Outline of WBA tools • Who can be an assessor? • Standards for assessment • Record keeping • Examples of scenarios • Hyperlinks to relevant guidance notes and forms

  21. Guidance note for each WBA tool • Aimed at assessors and trainees • Customised for each specialty and each tool • Content: • Explanation of the WBA tool • Suitable cases • Who can be an assessor • How the assessment works • Standards for assessment • Outcome of assessment • Feedback • Record keeping

  22. Opportunities & scenarios Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

  23. How to use opportunities for workplace-based assessment • Numerous opportunities/scenarios available - refer to guidance notes • Be realistic about assessment • duration • complexity • appropriateness • Sampling

  24. Undertaking assessments Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

  25. Conducting the assessment • Ensure both you and the trainee have read and understood the guidance • Check appropriateness of event • Adequate time to undertake assessment and provide feedback • Appropriate place for feedback • Have the assessment form with you and make notes on it

  26. The assessment form

  27. Information about the assessment …

  28. Benefits of using the assessment form • Gives structure to the encounter • Encourages a systematic approach • Generic forms, so some items will be not applicable (“unable to comment”) • Reduces bias • Helpful in providing feedback • Helpful in identifying developmental points • Helpful in commenting on behaviour

  29. The future of WBA • GMC has just approved new Foundation Curriculum • WBA will be Supervised Learning Events • Formative - no pass/fail • Feedback will be unavoidable • No hiding behind a set of ticks

  30. Feedback Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

  31. What is Feedback? Information given in response to a product, performance etc., used as a basis for improvement. Modification or control of a process or system by its results or effects.

  32. What is Feedback? Information given in response to a product, performance etc., used as a basis for improvement. Modification or control of a process or system by its results or effects. A loud whining sound.

  33. Psychologists’ view of feedback

  34. Giving feedback – why? • Feedback • one of the most important ways in which we learn • one of the most important aspects of teaching and relatively neglected in medical education • an essential component of workplace based assessment which is intended to be formative • Emphasised by GMC

  35. Giving feedback • Why is feedback so powerful? • It connects with feelings and helps us reflect • When did you last get some good feedback • how did you feel afterwards? • Do you remember how you felt when someone was really critical of your performance?

  36. Giving feedback - skills • Traditionally medical education has used destructive feedback • Times are changing • Greater emphasis on • Training the trainers • Teaching the Teachers • Educational Supervisor courses

  37. Giving feedback - skills • Praising is easy, makes people feel better but needs to be constructive – “That was OK but…” is not positive! • Skill requires: • detailed evaluation • active listening • constructive criticism

  38. Feedback skills - active listening • Non-verbal behaviour • Focus on speaker’s experience of activity • Acceptance (don’t dismiss ideas) • Empathy – relate to their feelings • Probing – need to get to what is behind what is said • Check and summarise • Advise – e.g. alternatives

  39. Feedback skills - Constructive criticism • Let the trainee speak first • Begin with the good points • Plan a solution for each problem • Be sensitive to person • Show interest – use your experience to coach and counsel not score points • Be constructive and encourage trainee to participate in finding solution

  40. Feedback skills - Constructive criticism • Deal with one point at a time • Criticise the act (deal with facts) not the individual • Do not hyperbolise • Don’t joke – it can be taken the wrong way • Don’t compare with other people • Be constructive, the aim is to improve performance

  41. A simple framework for feedback • Acknowledge the role of feelings • Start with description before moving on to evaluation and action planning • Allow trainee to speak first at each stage • Consider positive aspects before negative • Plan for future action and offer support

  42. Summary • Understand the assessment tools and process • Use your existing people, places and opportunities • Train the trainers/assessors and trainees • Allay their anxieties • Use it as a formative process

  43. Summary • Encourage trainees to plan WBA • Schedule WBA throughout attachments • Avoid last minute panic • Look out for hawks and doves • Trainees will know who they are! • Give proper feedback • Can be difficult with weak trainees

  44. Clear vision required

More Related