Information support to evidence based management: the unappreciated in pursuit of the non-existent

1. Information support to evidence based management: the unappreciated in pursuit of the non-existent Andrew Booth, Senior Lecturer in Evidence Based Healthcare Information, ScHARR, University of Sheffield.

2. “The English country gentleman galloping after a fox - the unspeakable in pursuit of the inedible” - Oscar Wilde

3. A Reminder • “Evidence” is welcomed for its apparent objectivity, whereas • In reality, “evidence” involves value judgements and subjectivity • And is value-laden with respect to • The Nature of Evidence • Its Context and • Its Audience

4. What is Evidence? • “Evidence is a lot more than research, and it includes a lot of contextual information. …Not…a way to end today’s healthcare debates, but rather…a way of raising the level of dialogue around important decisions”. • “Evidence-informed decision-making may be a better term than evidence-based decision-making”. • “Evidence-based decision-making is a value-laden process, as is the construction of the meaning of evidence. Recognizing the role that values play will lead to a greater level of transparency and understanding, helping to improve the quality of policy and management decisions”. Canadian Health Services Research Foundation Workshop, 2004

5. SOURCE: HEALTH SERVICES RESEARCH AND... Evidence-Based Decision-Making

6. Exhibit 1 • We looked at 144 proposals. Only 6.2 per cent were supported by strong evidence. For 21.2 per cent there was fair evidence and for 38.1 per cent there was poor evidence. Search not possible for 16.8 per cent of the proposals. • There was a moderate correlation between how good the evidence was and how the proposal was scored initially for health gain (p < 0.001). But by the end of priority setting process no correlation between strength of evidence and priority ranking (p = 0.97). • Research evidence did appear to influence initial assessment of proposals. However, it had no effect on the final priority choices made in the DHA purchasing plan. Dixon, Booth and Perrett, J Public Health Med. 1997 19(3):307-12.

7. Exhibit 2 • 124 decisions identified (two-thirds concerned organisation of care). Evidence existed for less than half (48.4%) the decisions, with 33.9% favouring decision and 14.5% where evidence was equivocal or unfavourable. Relevant non-randomised quantitative studies and qualitative studies were identified for only half of a subset of ten decisions. Evidence from economic evaluations only identified for one decision. Johnstone & Lacey. J Health Serv Res Policy. 2002 7(3):166-9.

8. Exhibit 3 - Systematic Review • 24 studies including 2041 interviews with health policy-makers. • Facilitators included personal contact (13/24), timely relevance (13/24), and the inclusion of summaries with policy recommendations (11/24). • Barriers were absence of personal contact (11/24), lack of timeliness or relevance of research (9/24), mutual mistrust (8/24) and power and budget struggles (7/24). • Two-way personal communication, (most common suggestion), may improve appropriate use of research evidence, but it might also promote selective (inappropriate) use of research evidence. Innvaer et al, 2002

9. The Research-Transfer Paradox • Research’s best chance at being used in decision-making depends on how unlike research its presentation can be. • Emotion, trust, storytelling and careful rationing are crucial ingredients to successful research transfer.

10. MANAGERS Active See the “big picture” Are Selective Emphasise Outcomes and Deliverables Like stories Set agendas Have a limited attention sp……… Have a profound aversion to circumlocution LIBRARIANS Reflective Focus on the detail Aim to be Comprehensive Emphasise Structure and Process Like facts Respond to agendas Will stick at it to the bitter end …and then some! Use jargon When twotribes go to war ( Hollywood FGT,1984)

11. Bad news, Good news • Managers generally are not evidence based BUT • This means that we do not need to use evidence based tactics and approaches to influence them which is GOOD • Because I have yet to find any evidence that proves that our libraries are cost-effective!

12. Why can’t a librarian be more like a manager • We DON’T have to change WHAT we do (although we can extend and refine it) • We DO need to change HOW WE PRESENT it: • Grab attention • Summarise • Synthesise • Digest • Energise – (Do this, not Read this) • Tell stories

13. Evidence Based Decision making • Four characteristics • transparency; • reliability; • inclusiveness; and • Explicitness • Criticised for “the privileging of research evidence over organizational evidence and political evidence” (Klein, 2004). • But correcting the imbalance?

14. Researchers Users Decision makers Knowledge Brokers A solution needs to get research evidence to decision-makers and (hopefully) improve the prestige of information/knowledge workers

15. Bridging the Gap • Knowledge Exchange (formerly Knowledge Transfer) • Knowledge Brokering

16. Knowledge exchange • Knowledge exchange is collaborative problem-solving between researchers and decision makers that happens through linkage and exchange. • Effective knowledge exchange involves interaction between decision makers and researchers and results in mutual learning through the process of planning, producing, disseminating, and applying existing or new research in decision-making.

17. Knowledge Exchange/Transfer Products Evidence Boost • http://www.chsrf.ca/mythbusters/index_e.php Myth busters • http://www.chsrf.ca/mythbusters/index_e.php

20. Knowledge Exchange/Transfer Methods • Communication Notes http://www.chsrf.ca/knowledge_transfer/resources_e.php • Developing a dissemination plan • Dealing With the Media • Designing a Great Poster • How to Give a Research Presentation to Decision Makers • Reader-Friendly Writing - 1:3:25 • Self-Editing - Putting Your Readers First

23. Knowledge brokering - 1 • Knowledge brokering links decision makers and researchers, facilitating their interaction so that they are able to better understand each other's goals and professional cultures, influence each other's work, forge new partnerships, and promote the use of research-based evidence in decision-making.

24. Knowledge Brokering - 2 • Includes: • finding the right players to influence research use in decision-making, • bringing these players together, • creating and helping to sustain relationships among them, • and helping them to engage in collaborative problem-solving.

25. KNOWLEDGE BROKERS (the artists formally/formerly known as librarians) Active (Brokers) See the “big picture” (Understanding the research and management worlds) Are Selective (Targeting of messages and audiences) Emphasise Outcomes and Deliverables (Succinct action points) Use stories (Use anecdotal approaches – good practice/cases) Set agendas (Implications of research) Capture limited attention spans (Marketing/Communication) Concise messages (Avoiding jargon) Bridging the gap

26. Requisite Tasks and Abilities • Understanding of both the research and decision making environments (evidence & management!) • Ability to find and assess relevant research • Entrepreneurial skills (networking, problem-solving skills, innovative solutions, etc) • Mediation and negotiation • Understanding of the principles of adult learning • Communication skills • Credibility…

27. Questions for Discussion • Are we up to the challenge? • If not, would we like to be? • If so, what training is required? • Is such training available?

28. Resources • Canadian Health Services Research Foundation: Knowledge Transfer & Exchange http://www.chsrf.ca/knowledge_transfer/index_e.php • RCN Knowledge Utilisation Site http://www.rcn-ku.org.uk/ • Chair on Knowledge Transfer and Utilisation http://kuuc.chair.ulaval.ca • Research To Policy (McMaster University) http://www.researchtopolicy.ca • Davis D et al. The case for knowledge translation: shortening the journey from evidence to effect. BMJ. 2003 Jul 5;327(7405):33-5.