Applications of Performance Assessment

2. Professional Regulation & Governance Should focus on health professionals� performance & fitness to practise rather than just their qualifications & experience. However, as I have already mentioned, most jurisdictions are now moving to a system where doctors will be required to show evidence of involvement in CME as part of their annual registration. It is no longer acceptable or appropriate that a doctor receives full registration , renewed without question for 50 or 60 years on completion of internship. It would be naive to think that all doctors are active in maintaining their knowledge and skill, although this is generally the argument of those opposed to re-certification in any form. There are the cynics who say that CME is no guarantee of good performance, but on the other hand, few could argue that a practitioner not involved in CME is fulfilling their professional responsibilities. It is an imperfect tool, admittedly, but as an accountable and self-regulating profession, it is a nettle we have to grasp. However, as I have already mentioned, most jurisdictions are now moving to a system where doctors will be required to show evidence of involvement in CME as part of their annual registration. It is no longer acceptable or appropriate that a doctor receives full registration , renewed without question for 50 or 60 years on completion of internship. It would be naive to think that all doctors are active in maintaining their knowledge and skill, although this is generally the argument of those opposed to re-certification in any form. There are the cynics who say that CME is no guarantee of good performance, but on the other hand, few could argue that a practitioner not involved in CME is fulfilling their professional responsibilities. It is an imperfect tool, admittedly, but as an accountable and self-regulating profession, it is a nettle we have to grasp.

4. Public Expectations Health professionals are assessed regularly, like pilots. Assessments are comprehensive and reliable. Results are acted upon appropriately.

5. The Irony of Assessment Pre-tertiary education University Postgraduate training Employed practice Independent practice

6. The language of performance assessment

7. Competence The possession of knowledge and skill. i.e. What the health professional knows, and knows how to do. I have introduced the terms competence and performance, and I want to be clear about the difference between them. Competence is�.. I terms of the breadth of their knowledge, a final year medical student is probably at the peak of their competence, so you can see why I mostly talk about performance!I have introduced the terms competence and performance, and I want to be clear about the difference between them. Competence is�.. I terms of the breadth of their knowledge, a final year medical student is probably at the peak of their competence, so you can see why I mostly talk about performance!

8. Performance The application of knowledge and skill. i.e. What the health professional actually does for real patients in a real setting. Performance is..Performance is..

11. Competence v Performance It is certain that an incompetent health professional will perform poorly. However, a competent health professional cannot be guaranteed to perform well. How do competence and performance inter-relate? Quote� The first of these statements have important implications for screening practitioners, although personally, I have some difficulty with the concept of �screening for bad apples�How do competence and performance inter-relate? Quote� The first of these statements have important implications for screening practitioners, although personally, I have some difficulty with the concept of �screening for bad apples�

12. Competence v Performance Experience System factors Distracters So why doesn�t competence translate into acceptable performance? �� If the aim of assessing performance is to rectify and remediate, then you very soon come to realise that you are wasting your time, effort and somebody�s money is you don�t deal with these factors first. So why doesn�t competence translate into acceptable performance? �� If the aim of assessing performance is to rectify and remediate, then you very soon come to realise that you are wasting your time, effort and somebody�s money is you don�t deal with these factors first.

14. Myth #1 Attendance at Continuing Professional Development sessions ensures competence.

15. Myth #2 Health professionals: know what education they need, know when they need it, immediately acquire it when they need it, always benefit from it.

16. Approaches to assessing performance Routine: for all health professionals in relation to registration / employment (aimed at improving quality, NOT finding bad apples.) We find our participants in a number of ways�..We find our participants in a number of ways�..

17. Approaches to assessing performance contd. Opportunistic: expressions of concern complaints in response to unsatisfactory routine assessment We find our participants in a number of ways�..We find our participants in a number of ways�..

18. Choosing assessment tools Demonstrates performance Do-able Defensible

19. Assessment Tools Assessment validity increases: with more than one assessor with multiple assessment tools in the health professional�s natural habitat

20. Assessment Domains Clinical practice Communication Collaboration Management practices Personal development Education / training Professional behaviour Personal health / wellbeing etc

21. Assessment tool kit Performance appraisal interview CPD / portfolios Tests of knowledge, eg MCQs OSCE / Case based oral / CSR Record review Outcome audits Interviews with colleagues / patients Observed practice (Mini CEX)

23. Clinical interview / Case based oral / OSCE

25. Interview with colleagues/patients

27. Observed practice

29. Assessment tools No single tool fully addresses performance in every domain. Aim for overlap along the competence / performance spectrum. The more tools the better.

30. NSW Medical BoardPerformance Assessment Program In place since 2000 Opportunistic ~ 30 full PAs / year Early intervention / remediation Peer assessment Environmental validity

32. Supplementary assessment Focused assessment Medical assessment Neuropsychometric testing Simulator

33. Common assessment triggers Multiple complaints History with Board Multi-faceted concerns Isolation Age

35. Discussion Topics Can a well governed healthcare system afford NOT to do Performance Assessment? Where should the principles of Performance Assessment be applied? What assessment tools would be applicable? What barriers would need to be overcome? How can students be prepared for career-long assessment?