Evidence-Based Management ( S earching & Finding S cientific E vidence)

1. Evidence-Based Management (Searching & Finding Scientific Evidence) Pitstop Evidence-Based HR, VOV lerendnetwerk, Antwerpen, 11 januari 2013

2. Evidence-Based Management A (very) Short Introduction

3. Medicine: Founding fathers David Sackett Gordon Guyatt McMaster University Medical School, Canada

4. How it all started

5. Problem I: too much information • More than 1 million articles in 40,000 medical journals per year (= 1995; now probably more than 2 million). For a specialist to keep up this means reading 25 articles every day (for a GP more than 100!) • Most of the new insights and treatment methods don’t reach the target group

6. Problem I: too much information • HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR manager to keep up this means reading 3 to 4 articles every day (for a ‘general’ manager more than 50!)

7. Problem II: persistent convictions if you’re hyperventilating breathe into a bag

8. Problem III: jumping to conclusions elderly people who have an irregular heartbeat are much more likely to die of coronary disease give them a drug that reduces the number of irregular beats

9. How 40,000 cardiologists can be wrong In the early1980s newly introduced anti-arrhythmic drugs were found to behighly successful at suppressing arrhythmias. Notuntil a RCT was performed was it realized that, althoughthese drugs suppressed arrhythmias, they actually increasedmortality. The CAST trial revealedExcess mortality of 56/1000. By the time the results of this trial were published, at least100,000 such patients had been taking these drugs.

10. David Sackett • Half of what you learn in medical school will be shown to be either dead wrong or out-of-date within 5 years of your graduation; the trouble is that nobody can tell you which half. • The most important thing to learn is how to learn on your own. • (Remember that your teachers are as full of bullshit as your parents)

11. Evidence-Based Practice 1991Medicine 1998Education 1999Social care 2000Nursing 2000Criminal justice ????Management?

12. 2. Evidence based management: What is it?

13. Definition Evidence based practice: Improve information to support decision making

14. Definition Evidence-based management means making decisions about the management of employees, teams or organizations through the conscientious, explicit and judicious use of four sources of information: 1. The best available scientific evidence 2. Organizational facts, metrics and characteristics 3. Stakeholders’ values and concerns 4. Practitioner expertise and judgment

15. Four sources

16. Evidence-based practice: • Focuses on the decision making process • Thinks in terms of probability (instead of golden bullets).

17. 3. Evidence-based management: Why do we need it?

18. Four sources

19. Trust me: 20 years of management experience!

20. Bounded rationality

21. Bounded rationality • System 1 • Fast • Intuitive, associative • heuristics & biases • System 2 • Slow (lazy) • Deliberate, ‘reasoning’ • Rational

22. System 1: prone to biases • Seeing order in randomness • Mental corner cutting • Misinterpretation of incomplete data • Halo effect • False consensus effect • Group think • Self serving bias • Sunk cost fallacy • Cognitive dissonance reduction • Confirmation bias • Authority bias • Small numbers fallacy • In-group bias • Recall bias • Anchoring bias • Inaccurate covariation detection • Distortions due to plausibility

23. 2. Searching & Finding The Best Available Scientific Evidence

24. Step 1: Formulate a focused question

25. 5-step approach EBMgt is a 5-step approach Formulate an answerable question (PICOC) Search for the best available evidence Critical appraise the quality of the found evidence Integrate the evidence with managerial expertise and organizational concerns and apply Monitor and evaluate the results

26. Asking the right question? Effect vs Non-effect

27. Types of questions: effect Does it work? Does it work better than ....? Does it have an effect on ....? What is the success factor for ....? What is required to make it work ...? Will it do more good than harm? Effect

28. Types of questions: non-effect Needs: What do people want or need? Attitude: What do people think or feel? Experience: What are peoples’ experiences? Prevalence: How many / often do people / organizations ...? Procedure: How can we implement ...? Process: How does it work? Explanation: Why / how does it work? Economics: How much does it cost?

29. Focused question? • Does team-building work? • What are the costs and benefits of self-steering teams? • What are the success factors for culture change? • Does management development improve the performance of managers? • Does employee participation prevent resistance to change? • How do employees feel about 360 degree feedback?

30. Foreground question? • What is a ‘team’? • What kind of teams? • In what contexts/settings? • What counts as ‘team-building’? • What does ‘work’ mean? • What outcomes are relevant? • Over what time periods? • Does team-building work?

31. Answerable question: PICOC P= Population I = Intervention (or success factor) C= Comparison O= Outcome C = Context

32. Example: merger

33. Answerable question: PICOC Imagine you are a consultant, your client is the board of directors of a large Canadian health-care organization. The board of directors has plans for a merger with a smaller healthcare organization in a nearby town. However, it’s been said that the organizational culture differs widely between the two organizations. The board of directors asks you if this culture-difference can impede a successful outcome of a merger. Most of them intuitively sense that cultural differences matter, but they want an evidence-based advice.

34. Answerable question: PICOC What else would you like to know?

35. Answerable question: PICOC P: What kind of Population are we talking about? Middle managers, back-office employees, medical staff, clerical staff? O: What kind of Outcome are we aiming for? Employee productivity, return on investment, profit margin, competitive position, innovation power, market share, customer satisfaction? P/C: And how is the assumed cultural difference assessed? Is it the personal view of some managers or is it measured by a validated instrument?

36. Answerable question: PICOC According to the board the objective of the merger is to integrate the back-office of the two organizations (ICT, finance, purchasing, facilities, personnel administration, etc.) in order to create economy of scale. The front offices and primary process of the two organizations will remain separate. The cultural difference is not objectively assessed (it is the perception of the senior managers of both organizations).

37. Answerable question: PICOC P= back office employees in a healthcare organisation I = merger, integration back office C= status quo O = economy of scale C = different organizational culture, unequal

38. Exercise Read the following five scenario’s Formulate on the basis of each scenario a focused question (use the PICOC format).

39. Scenario 2 “Ik werk als manager bij een grote financiële dienstverlener. Het concern is een holdingmaatschappij met daaronder een aantal divisies met een eigen P&L verantwoordelijkheid. Het accountmanagement van de divisies wordt aangestuurd op basis van een eigen set kritische prestatie indicatoren (kpi’s) die niet onderling is afgestemd. Het gevolg is dat de accountmanagers elkaar regelmatig in de wielen rijden bij gezamenlijke klanten en de omzet wordt ‘weggekaapt’ bij de ander. Kortom, er wordt onvoldoende samengewerkt. Nu wordt door de Executive Board van de holding een beroep gedaan op de divisiedirecties om meer met elkaar te gaan samenwerken. Mijn vraag is: staat het feit dat de divisies geen gedeelde kpi’s hebben een goede interne samenwerking in de weg of hoeft dit geen probleem te zijn?”

40. Scenario 3 • “I am a Dutch quality manager at a large international brewery. In the past 4 years I facilitated a lean management program that is running for more than 10 years in this organization. Since a couple of years, however, we see a decline in results, such as: • The number of problem analysis and improvement teams has been increased, but the amount of breakdowns and short stops in the production lines has not decreased. • There is slow progress for the total program for the Netherlands at this moment. • According to the middle managers the improvement program does not address really important topics.I would therefore like to know how our employees feel about lean management.”

41. Neem een actueel probleem / vraag / issue dat speelt in de organisatie waar je stage hebt gelopen (of een vraagstuk dat je persoonlijke interesse heeft). Formuleer op basis daarvan een onderzoeksvraag Geef aan wat voor type vraag het is (effect of non-effect) Benoem de PICOC elementen.

42. Step 2: Search for the best available evidence

43. 5-step approach EBMgt is a 5-step approach Formulate an answerable question (PICOC) Search for the best available evidence Critical appraise the quality of the found evidence Integrate the evidence with managerial expertise and organizational concerns and apply Monitor and evaluate the results

44. Searching evidence What do we search?

45. Company Annual Reports, Datastream, Factiva.com, Amadeus Wall street Journal, Financial Times, Business week, Financieel Dagblad ABI/INFORM, Business Source Premier, Emerald, PsychInfo, Science Direct Current Information Overview of a subject General background Academic Information Statistical Information Theories about a subject Company information CBS Statline, Eurostat Textbooks and encyclopedias Textbooks and popular books Encyclopedias, yearbooks & book reviews Information sources Type of Information Source

46. Articles in peer reviewed journals

47. Searching for evidence

48. Databases Postgraduate Course • ABI/INFORM • Business Source Elite • PsycINFO • Web of Knowledge • ERIC • Google Scholar

49. Searching for evidence

50. Searching evidence Postgraduate Course How do we search? Search Strategy