1. Ongoing Evaluation of Physician Performance: Developing a Performance Portfolio Cary Sennett, MD, PhDMedBiquitous Annual Conference May 15, 2008

2. Overview • “Performance Portfolio”—what are we talking about? • Design Criteria—what would a portfolio look like? • One—I hope promising—effort (in a bit more detail) • Summary…

3. Defining terms • Portfolio: A (comprehensive) information set that a physician can use • To establish his/her qualifications • To improve care • A “performance portfolio” to improve care • Must begin with information about practice performance • Must contain elements that speak to the capability of those factors that determine practice performance

4. Elements that “speak to capability” • How my practice is performing depends on • My competence (my knowledge and skill) • How effectively that competence is deployed • “Deployment”—that is, the conversion of capability to results—depends upon environmental factors • “Micro-environment” • “Macro-environment” • And (of course) the patient… • For purposes of our discussion, it may be less important to consider the “macro-environment”

5. So a portfolio should help a physician evaluate • Practice performance • Are we achieving the results that we could? • What limits our ability to do better? • His or her own competence • Do I have the knowledge and skill necessary to deliver excellent care? • And how can I close critical gaps? • (Micro)-system • Is the (micro-) system in which I function optimally configured to support my efforts to deploy that competence? • Am I using it effectively? • And, if not, how can I improve it? • My patients • Have I maximized their ability to achieve desired health results? • And, if not, how can I do so more effectively?

6. Designing a portfolio (to support practice improvement) • Practice results • Physician knowledge and skill • Systems infrastructure • Patient self-management  Need for broad set of inputs

7. What “Practice Results?” • Must address the range of results that are relevant to patients (and other customers…) • It may be helpful to think of that range as the IOM does: care that is • Safe • Timely • Effective • Efficient (delivered at appropriate cost) • Equitable • Patient-Centered • But, in any event, it is essential to recognize that performance is multidimensional

8. What “Knowledge and Skill?” ACGME (ABMS) competencies • Professionalism • Patient Care • Medical Knowledge • Communication and Interpersonal Skill • Systems-based Practice • Practice-based Learning and Improvement

9. What “System Infrastructure?” • Information management • Patient activation • Access and communication with patients • Safety and efficiency • Consultation and referral • Team function • Improvement process

10. How do we get there? • A lot of activity right now—but widely distributed • ABMS Certification/Maintenance of Certification may be a helpful “seed” around which this work can organize • Step on the path—but not the end of the road

11. Board Certification • Professional effort to evaluate competency (capability) of individual physician • Structured so as to offer a window into other key portfolio elements • Practice results • Practice systems infrastructure • Patient self-management • Designed—and objective is—to support improvement • But essential function is assessment (both formative and summative)

12. The Structure of Board Certification • Specific reference to ACGME competencies • Two elements • Initial certification • Periodic recertification/Maintenance of Certification • Structure common across 24 ABMS Boards • Assessment of actions against license • Self-assessment of medical knowledge • High stakes secure examination of cognitive knowledge and judgment • Self-assessment and improvement of practice performance • Boards vary with respect to implementation—strategies and timelines

13. Self-Assessment of Practice Performance • Designed to force “a new way of thinking” about quality and QI • Quality is about what the system produces • Quality improvement is not about working harder (or knowing more)—but about “diagnosing and treating” system problems • Designed to promote adult (experiential) learning • Physicians can receive up to 20 units category I CME credit, as well as credit toward renewing their certificate

14. The ABIM Practice Improvement Module (PIM™) Apply quality measures to practice Chart review Patient survey Practice survey Examine practice infrastructure and process Performance Report Compare performance to guidelines Improvement Plan Test a process change aimed at improving care Do Act Impact Study Report what was learned

15. The PIM as Portfolio • Provides window on practice results • Can link to other information about physician competency (knowledge and skill) • Provides window on systems infrastructure • Potential window on “patient competence”

16. Increasing robustness • Need broader window on practice performance—more, and more diverse—data • Need tighter and more intentional link to information about individual physician competency • Need mechanism to track (all elements over time) • Need expanded window on “patient competence”

17. Getting from here to there: “PIM Future” • Links to extant datasets—power practice and self-assessment off of available data • Performance (and personal) “dashboards”: turning data into information that speaks (meaningfully and reliably) to practice performance and to core drivers • Links to—truly integration with—support needed to go from • “I see an opportunity to improve” to • “I’m able to capitalize on that opportunity to improve”

19. What will it take? • More (more standardized and more available) data from many, many sources • More (much more) research • Collaboration/partnerships that will link assessment capability to improvement capability • Collaboration/partnerships to organize and focus energy and resources

20. Summary • It is possible to conceive of a portfolio that will support practice improvement • Such a portfolio will need to address a broad ranges of issues • Board certification (Maintenance of Certification) may be an important part of early efforts to build a portfolio • To get where we need to go will require collaboration—which will require much, much more standardization