1. Evidence-Base Practice

3. Evidence-based practice? Decisions should be based on a combination of critical thinking and the ‘best available evidence‘.

4. Evidence = Various types of information outcome of scientific research, organizational facts & data, benchmarking, best practices, collective experience, personal experience, intuition

5. All managers base their decisions on ‘evidence’

6. However ...

7. Many managers pay little or no attention to the quality of the evidence they base their decisions on

8. Trust me, 20 years of management experience

9. Teach managers how to critically evaluate the validity, and generalizability of the evidence and help them find ‘the best available’ evidence

10. Best available evidence • Experiential evidence: the professional insight, understanding, and expertise of practitioners • Organizational evidence; data, facts and figures, business intelligence, and benchmarks • Scientific evidence: outcome from scientific research published in peer reviewed journals • Organizational values and stakeholders’ concerns

11. Evidence based decision • Best available organizational evidence • Best available experiential evidence • Decision making process • Best available scientific evidence • Organizational values and stakeholders’ concerns

12. Evidence based decision • Best available organizational evidence • Best available experiential evidence • Decision making process • Best available scientific evidence diagnosis intervention • Organizational values and stakeholders’ concerns

13. Evidence based decision It’s about probabilities (not golden bullets)

14. Evidence-Based Practice 1991Medicine 1998Education 1999Social care, public policy 2000Nursing 2000Criminal justice ????Management?

15. Evidence-Based Practice

16. Evidence-Based Practice

17. Evidence-Based Practice

18. Evidence-Based Practice

19. Evidence-Based Practice

20. What’s the evidence for evidence-based practice?

21. Got evidence? • Forecasts or risk assessments based on the aggregated experience of multiple persons are more accurate than forecasts based on the experience of one person (provided that the forecasts are made independently before being averaged together) • Choudhry, N.K., et al. Systematic review: the relationshipbetweenclinicalexperienceandquality of health care. Ann Intern Med. 2005; 142 (4) • Silver, N. The Signaland the Noise: WhySoManyPredictionsFail - but SomeDon't. Penguin: London, 2012; p 286 and p 690 • Bauer A., et al. Forecast Evaluation with Cross Sectional Data: The Blue Chip Surveys. Economic Review, Federal Reserva bank of Atlanta, 2003. • Servan-Schreiber, E., et al. Prediction Markets: Does Money Matter? Electronic Markets, 2004: 14 (31). • Scott Armstrong, J. CombiningForecasts, in Principles of Forecasting: A handbookforResearchersandPractitioners, Kluwer AcademicPublishers, New York, 2001

22. Got evidence? • Professional judgments based on hard data or statistical models are more accurate than judgments solely based on experiential evidence • Yaniv, I., & Choshen-Hillel, S. (2011). Exploiting the Wisdom of Othersto Make BetterDecisions: Suspending JudgmentReducesEgocentrismandIncreasesAccuracy, Journal of BehavioralDecision Making, 2012; 25 (5) p 427–434 • Lewis, M. Moneyball: The Art of Winning an Unfair Game. Barnes andNoble, 2003 • Grove, W.M. Clinical Versus Statistical Prediction. Journal of ClinicalPsychology, 2005; 61(10), p 1233–1243

23. Got evidence? • Adecision based on the combination of different types of evidence (experiential, organizational, scientific) leads tobetteroutcomesthan a decision based on justone source of evidence • Antman, E.M. et al, A comparison of results of meta-analyses of randomized control trials andrecommendations of clinical experts, JAMA, 1992: 268 (2) p 240 – 248 • McNees, S.K. The Role of Judgment in MacroeconomicForecastingAccuracy, International Journal of Forecasting, 1990; 6 (3), p 28-299 • Silver, N. The Signaland the Noise: WhySoManyPredictionsFail - but SomeDon't. Penguin: London, 2012; p 286 and p 690 • Tetlock, P. E. Expert PoliticalJudgement, Princeton, NJ: Princeton University Press, 2006

24. Evidence-Base Practice

25. Post mortem analysis

26. Evidence-based perspective • NOT: Did they make the right decision? • BUT: Assessment of the decision making process • What kind of evidence was there? • Was this the best available evidence? • Is there evidence from scientific research to support (or call into question) the approach taken?

27. Decision making process

28. Hospital, The Netherlands

29. Organization • 550 beds • 3300 employees • 210 medical specialists • 225,000 admissions • Top Clinical & Teaching hospital • Structure: Business Units

30. Decision: Leadership training All managers: • Board of directors • Division managers • Unit managers • Head nurses

31. Cause I: reorganization From: 20 business units - 20 senior managers - 40 supervisors To: 20 business units – 8 senior managers - 40 supervisors

32. Cause I: reorganization • Ambition • Experience • Education (MBA) • Assessment • Selection Gap

33. Cause I: reorganization • larger span of control, more responsibilities = extra skills Theyneed extra skills We need extra skills

34. Cause II: leadershipclimate Employee / Job satisfaction: leadership

35. Cause II: leadershipclimate Board of directors

36. Cause III: management development

37. Project team • 10 members • Expertise (HR) • Representation (nurses, doctors, managers) • Support base

38. Process: months! • Sessions with all stakeholders (medical staff, nurses, managers, staff council) • What is leadership? • What kind of leadership does the OLVG need? OLVG leadershipvision

39. OLVG Leadershipvision • Passion and business • Inspire and connect • Leadership concerns us all

40. Selection procedure Longlist: 30 • References (other hospitals, network), reputation Shortlist: 5 • Academic, Educational, Training, HR Consulting, Wild card First round: 3 • Paper pitch: proposal based on documents Final round: 1 • Carousel: board, medical staff, head nurses, managers

41. Decision making process

42. Decision making process What was the problem / issue What kind of evidence was there? Was this the best available evidence?

43. 3 steps Problem identification Surfacing assumptions Logic model

44. Step 1: What is the problem? • For which problem is ….. the solution? • For who(m) • Why? • How big? • How do we know (what is the evidence?)

45. Decision making process What was the problem / issue What kind of evidence was there? Was this the best available evidence?

46. Step 2: What are the assumptions? Assumptions are often hidden

47. 3 steps Problem identification Surfacing assumptions Logic model

48. Logic model

49. Decision making process What was the problem / issue? What kind of evidence was there? Was this the best available evidence?

50. Best available evidence? • Best available organizational evidence • Best available experiential evidence • Decision making process • Best available scientific evidence • Organizational values and stakeholders’ concerns