Resident Evaluation

1. Resident Evaluation Clinical Competency Laura Kezar, MD Associate Professor Physical Medicine and Rehabilitation

2. Roles of Oversight Agencies • ACGME • Residents in training • RRC accreditation process • ABMS • Initial board certification and maintenance of certification Groups perceived the need for developing description of “competent physician” • Response to external forces • Errors in Medicine Report, etc.

3. What is Clinical Competency? • Critical knowledge and ability to perform defining acts of our profession. • Responsibility of program director and teaching faculty to verify that residents possess the skills, knowledge, and attitudes necessary to competently practice patient care.

4. How Do We Evaluate It? • Formative Evaluations – day to day • Feedback given to residents on a regular basis to help them improve performance • Daily verbal interaction • Correction of errors on H&P’s, notes • 360 evaluations • Peer evaluations • Reflections on lecture, workshop, committees • Do not go into the “permanent record.” • Portfolios often to show work and document improvement over time.

5. Daily Feedback • I see … • give specific information • I feel … • pleased, disappointed, frustrated • I think … • this was unprofessional • you did a great job • I want … • you to study this tonight and we will re-evaluate it tomorrow • You should put this in your portfolio

6. How Do We Evaluate It? • Summative Evaluations • Formal Global Rotation Evaluations • Semiannual Global Evaluations • Evaluation at completion of residency to specialty board

7. Six Core Competencies1. Patient Care • Patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health.

8. Six Core Competencies2. Medical Knowledge • Medical knowledge about established and evolving biomedical, clinical, and cognate (epidemiological and social-behavioral) sciences and the application of this knowledge to patient care

9. Six Core Competencies3. Practice-based Learning and Improvement • Involves investigation and evaluation of physician’s own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care

10. Six Core Competencies4. Interpersonal and Communication Skills • Interpersonal and communication skills that result in effective information exchange and teaming with patients, families, other health care professionals

11. Six Core Competencies5. Professionalism • Professionalism, as manifested through a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population.

12. Six Core Competencies6. Systems-based Practice • Systems-based practice • manifested by actions that demonstrate an awareness of and responsiveness to the larger context and system of health care • ability to effectively call on system resources to provide care that is optimal value.

13. Typical Evaluation MethodsSee the ACGME “Toolbox” • Teaching physician observations • Written exams • Learning objectives and evaluations • Medical record audits • 360 Evals

14. Evaluation Methods • Computer simulated patient encounters • Clinical Evaluation Exercise (CEX) • Mini CEX • Standardized patients (Objective Structured Clinical Examination (OSCE)

15. Learning Objectives and Evaluations • Supervision of residents by clinically competent physicians - the ideal assessment site but difficult to document. • Objectives delineate what resident should be able to do after completion of rotation. • Learning objectives should span all domains of learning - cognitive, affective, psychomotor

16. Learning Objectives • Reasonable, attainable, and measurable • Should be specific to • Clinical setting - inpatient or outpatient • Technical skills needed - ability to perform the physical exam, injections, EMG’s, surgical procedures

17. Global Faculty Evaluations • Directly reflect objectives • Be specific to clinical situations • Advantages: prolonged observation, direct assessment over time • Disadvantages: “one shot” phenomenon, time consuming, decreasing time with residents due to financial constraints, feedback required, difficult to standardize

18. Additional Evaluation Methods • Medical record audit • Checklist looking for documentation of specific information • Judgment about decision-making • Can be done longitudinally • Requires substantial faculty time • Records do not always reflect what happens in patient encounters • Improves documentation but not health care

19. Computer Simulated Clinical Encounter • Computerized patient management problems • At best, a partial representation of a complete patient • Must capture key features of interaction • Benefits: exposure to “core” of disorders, consistency, detailed feedback • Disadvantages: cost, time

20. OSCE • Benefits: consistently display tasks, clinical task scaled to skills needed to be assessed, predetermined grading scale • Disadvantages: LABOR INTENSIVE, costly

21. Clinical Evaluation Exercise • Observing a trainee obtaining and performing a comprehensive history and physical examination on a new patient • Allowing time to write up case, impression, management plans • Presentation of case with discussion of findings, impressions, recommendations • Immediate feedback to trainee

22. Mini-Clinical Evaluation Exercise • Observation of focused H&P • Discussion of diagnosis and Rx plan • Eval of performance and feedback • Total time 20-35 minutes • More accurately reflects clinical practice

23. Mini-CEX • 4 areas evaluated • history • physical exam • clinical judgment • humanistic qualities • Rating scale used for evaluation • 4-10 needed for reliability - more needed for borderline performers

24. Mini-CEX • Coupled with oral exam • Could be done at end of each rotation or at end of academic year • Real patients or trained “standardized” patients