A Systematic Review of A&F interventions: the good, the bad, and the ugly

1. A Systematic Review of A&F interventions: the good, the bad, and the ugly Heather Colquhoun, CIHR and KT Canada Postdoctoral Fellow, OHRI

2. Research Team Jamie C Brehaut, Ottawa Hospital Research Institute (OHRI) Kevin Eva, University of British Columbia (Vancouver) Jeremy Grimshaw, OHRI Anne Sales, University of Michigan (Ann Arbor) Noah Ivers, Women’s College Research Institute (Toronto) Susan Michie, University College London (UK) Heather Colquhoun, OHRI Kelly Carroll, MSc, OHRI Mathieu Chalifoux, OHRI Funding: CIHR KT Priority Announcement

3. Systematic Review of studies of A&F • Not focused on effectiveness, but rather the details of the interventions • Extent, type and purpose of theory use • Consistency with theoretical constructs believed to be important for A&F • Intervention description

4. Methods • Systematic Review • 140 RCT’s of A&F from the Cochrane Review • Data extraction form, pilot X 2 • Extracted by two reviewers, one consistent • Separate extraction with consensus for disagreements

5. A&F Interventions • To whom? • What exactly? • Where and how delivered? • How much? • Why? • Give results, challenges, examples of variations

6. To whom? • Was the FB given to the person in whom the behaviour change was desired?

7. What exactly?

8. What exactly? • Feedback about ‘Other’ 45 (32%) • Accuracy of their diagnosis • Cost • Risk data for patients (stroke, infection) • Education as FB (FB on what type of antibiotic they should have prescribed) • Survey on barriers to change

9. What exactly? • Was the FB graphical? • Did the FB address the behaviour to be changed? • Was there a clear comparison in the FB?

10. Comparison for A&F Other: internal standard, “plan wide scores” for comparison, systematic review, algorithm, others previous performance + thresholds set by expert panel, others previous performance + own previous performance + drug formulations that accounted for >60% of charges + the total charges

11. How delivered? • Was the FB given face to face?

12. How much – Total feedback • Unclear: 33 (24%) • Clear: 107 (76%) • Once: 33 • Twice: 21 • Three times: 13 • Four times: 13 • >four times: 27 (range 5-78) • Frequency • Challenges

13. When and how much? • Was the lag time between collection of the FB and the provision of the FB clearly stated? • If yes, what was it? • Days: 6 • Weeks: 23 • Months: 56 • Years: 1 • Mix: 2

14. Why A&F?Intervention rationale in A&F trials

15. Reporting • “The data was presented in an easy to follow format” • “Extensive feedback was provided twice a week” • “Feedback was always discussed in detail” • “The exact nature of the A&F was decided within each audit group and supported by the researcher with help and advice from the tutor”

16. Hospital Report Card: Given to individuals not given to target, about behaviour, about the group, aggregate, comparison: multiple, given 1X, 12 indicators

17. Given to individuals (mailed), individual and group performance, aggregate data, multiple comparison, target yes, behaviour yes, key message, every 2 months for 6 months (3)

18. Given to individuals, individual performance?, ind data, target yes, behaviour no, monthly for 9 months

19. Self-monitoring, face to face, given to individual, individual performance, weekly for 4 months? “To be displayed in a conspicuous location in their office”

20. Summary • Some consistency but also wide variation • Reporting is poor in many areas • Rationale for the intervention is lacking

21. Thank you

22. Given to individuals by mail, individual and group performance, multiple comparisons, quarterly for 18 months (6), target yes, behaviour yes