Training day for West Midlands Trainers

1. Training day for West Midlands Trainers Dr Jane Mamelok RCGP WPBA Clinical Lead

2. Aims for today • Inform trainers of role and purpose of WPBA & portfolio assessment • Identify the problems in design and delivery • Identify problems for users, trainees and trainers • Discuss educational supervision • Benchmarking of portfolios and log entries, and ES judgments. • Case Based Discussion

3. THE TRIPOS

4. Psychomotor skills

5. WPBA reaches the parts other assessments cannot reach

6. “The evaluation of a doctor’s progress over time, in their performance in those areas of professional practice best tested in the workplace” FORMATIVE Summative judgment made by panels informed by evidence from WPBA Workplace Based Assessment

7. WPBA Action Does CSA Performance Shows How CSA Competence Knows How Knowledge AKT Knows Miller’s Pyramid

8. Communication & Consulting Skills Data Gathering & Interpretation Making a Diagnosis / Making decisions Practising Holistically Clinical Management Working with Colleagues & in Teams Managing Medical Complexity Maintaining an Ethical Approach to Practice Fitness to Practise Primary Care Administration & IMT Community Orientation Maintaining Performance, Learning & Teaching RELATIONSHIP DIAGNOSTICS Professionalism MANAGEMENT

9. Archive tool Analytical tool It is more than just a depository for information. It is more than the sum of the required assessments. Offers the opportunity for good practice, CPD collecting evidence for appraisal & revalidation. The function of the portfolio

10. Collection of evidence of achievement of competence. Reflection on learning Evaluation of evidence Defence of evidence Assessment decision The five stages of portfolio assessment.

11. RCGP has defined what should be the minimum framework and tools for evidence collection. The quality and content of the portfolio is dependent on the portfolio builder. 1. Collection of evidence

12. CBD Mini CEX COT DOPs MSF PSQ Naturally occurring evidence – audit SEA learning logs etc. 7 tools in the box

13. This is definitely “Needs further development” – NFD Log entries tend to be descriptive rather than reflective and analytical Sharing of negative entries can be a problem. Lack of balance. 2. Reflection on learning

14. There is a risk of teaching to the assessment hurdles. WPBA is not just 12 COTs & 12 CBDs Risk with e-portfolio that the richness of qualitative information available is lost by concentrating on gathering the minimum evidence only. Concentrate on building balanced portfolios upon which performance judgments can be made. Support trainees to develop insight, using the portfolios for reflection on performance. (Kolb’s learning cycle) Changing the mind-set

15. Kolb experiental learning cycle?

16. This is the role of the educational supervisor ES uses the information in the e-portfolio, to make an assessment of performance based on observed professional behaviours (using some of the tools) and evidence from the trainees reflective learning log and learning portfolio. Variable understanding competency framework and how that is applied secondary vs primary care – needs support. 3. Evaluation of the evidence

17. Clinical supervision describes the framework for regular, structured encounters reflecting on casework in the context of the post or specialty in which the health professionals are working and aims to identify areas of best practice and developmental needs. It has an important clinical governance function. Clinical Supervision

18. Educational supervision is organised supervision taking place in the context of a training programme, it is aimed more to act as an umbrella to guide the trainee through the training programme than assessing and discussing individual cases (which is in the domain of the clinical supervisor). Educational supervision

19. What do we expect from secondary care trainers? • Understanding of objectives for GPST and assessment to enable curriculum delivery in the context of the specialty. • To offer clinical supervision. • Work with training programme directors to identify relevant educational opportunities in each post. • To make dependable judgments of performance based evidence and observed professional behaviours using the tools and evidence in the WPBA portfolio. • To give formative feedback to trainees.

20. Senior educator with an understanding of training programme, assessments and employment issues. Good listener offering supportive challenge. Can give constructive feedback Professionalism, recognises the conflicting roles, sets boundaries etc. What makes a good educational supervisor ?

21. Senior educator with an understanding of training programme, assessments and employment issues. Good listener offering supportive challenge. Can give constructive feedback Professionalism, recognises the conflicting roles, sets boundaries etc. What makes a good educational supervisor ?

22. This can be summarised as providing an overarching umbrella of guidance helping the trainee get the best out of their programme and navigating their way through it. However, the ES assesses the rate of progress and development of competence contributing to a summative judgment. The role of the ES in GP specialty training.

23. Guidance and support with performance appraisal Learning needs assessment, empowering and facilitating the learner. Conduit and network function Appraiser and counsellor Coaching and career counselling Guardian role Varied roles of the Educational Supervisor

24. Identify you learner, ST1 ST2 or ST3. Arrange an early meeting to set boundaries, frequency of meetings etc. The ES needs to be able to navigate the e-portfolio proficiently. Review the evidence. Give formative feedback and define PDP at the staged reviews. So you are supervising a GP specialty trainee, what do you need to do?

25. Check assessment schedule. Review the self assessment ratings. Review PDP objectives if follow on review. Review competency and curriculum coverage Purposeful sampling of the learning log – descriptive v reflective. Look at CSR is there evidence to support their judgement? Hone in on some of the COTs and CBDs Reviewing the evidence

26. The Review page e-portfolio

27. Check the assessment schedule

28. Review the PDP

29. Review curriculum & competency coverage and self rating scales

30. Sample the learning log

31. Review the Clinical Supervisors report

32. How much is enough? How much evidence do you need to make a jugement? What do we mean by competence? Questions for debate

33. The evidence gathered from WPBA builds up a “picture” of the competent GP. How much evidence is enough?

35. Feedback on WPBA • Severn conducted a qualitative study to establish the experience of GPSTs undertaking WPBA during hospital posts. • Findings helpful and illustrated the problems for which we had anecdotal evidence.

36. Severn study 1 • Too many assessments, not valid. • CBD ok, CEX & DOPs not relevant. • Value feedback from MSF • CBD not being used as intended, assessors 2 care < 1 care better. • Rating scales and CF not used appropriately and poorly understood by educators and trainees.

37. Severn study - “Feedback” • Trainees value feedback, particularly MSF. • Too little, too late, needs to be contextual and time in busy posts. • Comments are bland & non-specific • Some poor practice amongst assessors reported.

38. The Clinical Supervisors report

39. The modified CSR

42. www.nesc.nhs.uk

43. Congruence

44. Who should review log entries? • Should CS validate log entries • What does validation mean? • Do 2 care understand curriculum? • Do 2 care understand the competency framework?

45. Summary • Set WPBA in context • Outlined competency framework that underpins it. • Issues in Clinical & Educational supervision • Making judgments dependable, improving feedback and reflection. • Quality Assurance

46. There’s always one in every holiday snap.