Competency-based Evaluation (and Competency in Evaluation!)

1. Competency-based Evaluation(and Competency in Evaluation!) Kathy O’Kane Kreutzer, M.Ed. VCU School of Medicine Richmond, VA

2. “We are confronted with insurmountable opportunities.” Pogo

3. Program Evaluation and Planning Look at the Big Picture Map out your teaching and evaluation activities against the competencies Start with what you do best Choose new teaching or evaluation activities to fill gaps

4. Program Evaluation and Planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and evaluation and planning and planning and evaluation and planning and evaluation and planning This is an ongoing process! Excellent Programs are Never “Done”

5. 'It doesn't happen all at once… you become. It takes a long time.’ The Skin Horse The Velveteen Rabbit Margery Williams

6. Program Evaluation and Planning Have clear learning objectives and criteria Make trainees and faculty aware of the objectives Evaluate what you are teaching Document what you evaluate Communicate feedback to trainees in a timely manner Have a process for addressing individual needs of trainees

7. Trainee Evaluation Use different methods to evaluate different competencies and multiple methods for each competency Direct observation is the Gold Standard Be sure that the evaluation method matches the competency ACGME website – go to “Outcome Project” and then “Assessment” for excellent examples and descriptions of assessment tools and how and when they are most effectively used

8. Trainee Evaluationadapted from ACGME/ABMS Toolbox v 1.1

9. I’ll do it because I want to, but not because you tell me to! Mary O’Toole The Answer-Backer Cure Mrs. Piggle Wiggle Betty MacDonald

10. “Own” your evaluation methods and results If you don’t believe it don’t do it If you don’t use it don’t do it If you don’t know why you’re doing it don’t do it

11. Feedback is the breakfast of champions Ken Blanchard

12. Ongoing Feedback and Addressing Needs are Key Challenge yourself to provide more frequent feedback Weekly is not too often “5 minute feedback” can be effective Ask about and listen to learner needs and concerns Develop a plan to address needs Document

13. The “FED” Mnemonic Feedback Encouragement Direction Hershey S. Bell, MD

14. “Familiarity breeds content” Anna Quindlen

15. Challenges for Small Programs A trainee in a small program deserves as much planning and forethought in support of learning and development as a trainee in a large program

16. Challenges for Small Programs Be sure your trainees have a peer group  Fellows are not exactly residents, not exactly faculty; identify opportunities for them to spend time with other fellows  Rounds, journal clubs, interdisciplinary case presentations, research projects, writing

17. Challenges for Small Programs • Communication • A close group can make some communication lesseffective • Your team may be a family, but it is a family with a purpose • Be explicit about when you are giving feedback • Addressing any concerns promptly is especially important • See Medio’s Top Ten Reasons We Avoid Talking to SomeoneAbout a Problem

18. Challenges for Small Programs • Lack of anonymity is a fact in small programs •  Respect •  Privacy •  Focus on the behavior, not the person • Confidentiality where possible • Model receptivity to feedback

19. “Education by design, not by chance” Kreutzer’s Accreditation Axim

20. Unless someone like you cares a whole awful lot, nothing is going to get better. It’s not. Dr. Seuss The Lorax

21. Today is your day! Your mountain is waiting. So…get on your way. Oh, the Places You'll Go! Dr. Seuss