Commitment to Change: Research and Theory Informing Practice

1. Commitment to Change: Research and Theory Informing Practice Jack Dolcourt, MD, MEd University of Utah School of Medicine Primary Children’s Medical Center Salt Lake City, Utah Contact information: jack.dolcourt@hsc.utah.edu

2. Disclosure The content of this presentation does not, will not and cannot relate to any product of a commercial interest; therefore, I have no relationships to report.

3. My ‘Aha’ Moment Commitment to change is like Radio-Immuno Assay Thyroid Foundation of Canada Egan KL. Commitment for change-a Radioimmunoassay for Continuing Medical Education. In: Davidoff F, ed. Who has seen a blood sugar? : reflections on medical education. Philadelphia: American College of Physicians, 1996:29-33.

4. Commitment to Change • Easy, explicit/observable, change in practice • Avoids cueing because it is open ended • Compare learner responses with faculty goals • CTC is a force to change/reinforcement • Direct measure of CME’s effectiveness: survey for accomplishment Egan KL. Commitment for change. In: Davidoff F, ed. Who has seen a blood sugar? : reflections on medical education. Philadelphia: American College of physicians, 1996:29-33.

5. CTC Make Thoughts Visible “I can now get inside their head.”

6. My Questions • How can I do a CTC for myself? • Is self-reporting valid? • Are CTCs effective in facilitating behavioral change? Are they predictive of actual behavioral changes? • What is the theoretical basis for CTCs?

7. Malcolm Knowles Assumption: Adult learners’ orientation to education • Education: increased competence to achieve full potential • Want to apply knowledge • Learning experienced around competency-development • People are performance-centered Knowles MS, Holton III EF, and Swanson RA. The Adult Learner (5th ed). Houston: Gulf Publishing Co, 1998.

8. Question 1: How can I do a CTC? Purkis (1982): instrument for evaluating CME courses • n=39 physicians at a pain symposium • 72% made 1-3 commitments (total 67 CTC) • 93% had made at least 1 change at 2 months • 63% of CTCs implemented at 2 months • 27% not implemented b/o lack of suitable cases • Indirect evidence for changes in behavior • Reinforcement of learning • Identify teaching points that had greatest impact Purkis IE. Commitment for change: an instrument for evaluating CME courses. J Med Educ 1982; 57(1):61-63.

9. Question 2: Is Self-Reporting Valid? • Curry L & Purkis IE (1986) used carbon paper prescription pads (n=61) • Behaviorist model (SR) • Self reports ‘sufficiently valid’ actual behavioral change, when there was an intention to change [at -6 vs. +16 wks] ranged from p<0.001 p=0.04 • Many CTCs unrelated to instructors’ essential points, combination of teaching points or were tangential, minor or not taught at all. • “match between course content & reported change will be discouragingly small” • Self-reportingis valid but why discouragingly small? Curry L, Purkis IE. Validity of self-reports of behavior changes by participants after a CME course. J Med Educ 1986; 61(7):579-584.

10. Educational models Constructivist model • People construct their own understanding and knowledge of the world through experiencing things and reflecting on those experiences. Tyler model Step 1: What’s the Problem/Need? Step 2: Define the goals or Objectives Step 4: Evaluation Learning Cycle Step 3: Teaching Tyler RW. Basic principles of curriculum and instruction. Chicago: University of Chicago, 1949.

11. Question 2 (cont’d): Is Self-Reporting Valid? Wakefield J et al (2003) • Prescription records -1 yr to +6 months • n=207 physicians in peer learning • 91% planned to make at least 1 change • Total of 209 committed-to changes • 71% Rx CTCs accomplished Wakefield J et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof 2003; 23:89-93.

12. Question 3: Do CTCs Facilitate Behavioral Change? Pereles L, Lockyer J. et al (1997) • N=26 • CTC group made more changes & types of changes were more difficult – p=0.07 Pereles L, Lockyer J et al. Effectiveness of commitment contracts in facilitating change in continuing medical education intervention. J Contin Educ Health Prof 1997; 17:27-31.

13. CTCs to Learn More About • Educational model: Tyler (teacher centered) vs. Constructivist (learner centered) • Egan: ‘This fundamental question about CME has never been answered…” • Purkis: evaluation for teaching points that had the greatest impact

14. CTC Form

15. Results-part 1 • 61 attendees, multidisciplinary, 70% participated • 1 month : n=33 completers with 119 CTCs • 54% of the CTCs were self-reported to have been implemented by 1 month • 88% of unimplemented CTC still in process • No relationship number of CTCs and audience perception of attaining predetermined objectives Dolcourt JL. Commitment to change: a strategy for promoting educational effectiveness. J Cont Ed Health Prof 2000; 20:156-163.

16. Results-part 2: Unanticipated Learning • Predicates: CTCs compared to predetermined instructional objectives • 68% CTC predicates matched objectives • 32% CTC unmatched (unanticipated learning) • No CTCs for 34% objectives CTC Objectives Unanticipated learning Dolcourt JL, Zuckerman G. Unanticipated learning outcomes associated with commitment to change in continuing medical education. J Cont Ed Health Prof 2003; 23:173-181.

17. Take Home Messages • CTCs better and fuller evaluation tool than meeting objectives • Instructional objectives don’t account for all learning and behavioral changes • Learners interpret & adapt new knowledge with consideration for previous life experiences

18. CTC for Understanding Use of Knowledge and Skills • Degree implemented (full, part, none) • n=352 physicians at 21 centers - 1,635 CTCs • 6 mos: 57% provided F/U data • 55% CTCs implemented; of these, 67% fully implemented; 48% of fully implemented CTC in 2 areas-58% course time allocation • Is follow-up part of the reflection exercise or an intervention in its own right? Lockyer JM, Fidler H et al. Commitment to change statements: a way of understanding how participants use information and skills taught in an educational session. J Cont Ed Health Prof 2001; 21:82-89.

19. Theoretical Foundations • Unclear conceptual psychological framework underlying CTCs • What does ‘commitment’ mean? • Binding an individual to a behavioral act? • Attitude and belief? • More than an evaluation tool • Strategy for  likelihood of follow-through Mazmanian PE et al. Commitment to change: ideational roots, empirical evidence, and ethical implications. J Cont Ed Health Prof 1997; 17:133-140. Mazmanian PE, Mazmanian PM. Commitment to Change: theoretical foundations, methods, and outcomes. J Cont Ed Health Prof 1999; 19:200-207. Overton GK, MacVicar R. Requesting a commitment to change: conditions that produce behavioral or attitudinal commitment. J Cont Ed Health Prof 2008; 28(2):60-66.

20. Lectures do affect change CTC more than eval tool CTC may be force for change/reinforcement Self-reporting is valid Easy, explicit/observable, change in practice CTCs better and fuller evaluation tool than meeting objectives-categorization By topic-what worked Unintended learning Time allocation Direct measure of CME’s effectiveness: survey for accomplishment What this Research Tells Us

21. Score Card Canada Scotland USA

22. Bibliography Curry L, Purkis IE. Validity of self-reports of behavior changes by participants after a CME course. J Med Educ 1986; 61(7):579-584. Dolcourt JL. Commitment to change: a strategy for promoting educational effectiveness. J Cont Ed Health Prof 2000; 20:156-163. Dolcourt JL, Zuckerman G. Unanticipated learning outcomes associated with commitment to change in continuing medical education. J Cont Ed Health Prof 2003; 23:173-181. Egan KL. Commitment for change-a Radioimmunoassay for Continuing Medical Education. In: Davidoff F, ed. Who has seen a blood sugar? : reflections on medical education. Philadelphia: American College of Physicians, 1996:29-33. Knowles MS, Holton III EF, and Swanson RA. The Adult Learner (5th ed). Houston: Gulf Publishing Co, 1998. Lockyer JM, Fidler H et al. Commitment to change statements: a way of understanding how participants use information and skills taught in an educational session. J Cont Ed Health Prof 2001; 21:82-89. Mazmanian PE et al. Commitment to change: ideational roots, empirical evidence, and ethical implications. J Cont Ed Health Prof 1997; 17:133-140. Mazmanian PE, Mazmanian PM. Commitment to Change: theoretical foundations, methods, and outcomes. J Cont Ed Health Prof 1999; 19:200-207. Overton GK, MacVicar R. Requesting a commitment to change: conditions that produce behavioral or attitudinal commitment. J Cont Ed Health Prof 2008; 28(2):60-66. Pereles L, Lockyer J et al. Effectiveness of commitment contracts in facilitating change in continuing medical education intervention. J Contin Educ Health Prof 1997; 17:27-31. Purkis IE. Commitment for change: an instrument for evaluating CME courses. J Med Educ 1982; 57(1):61-63. Tyler RW. Basic principles of curriculum and instruction. Chicago: University of Chicago, 1949. Wakefield J et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof 2003; 23:89-93.