Evidence-Based Management: Teaching the Practice of Management

1. PDW, Annual AOM 2012, Boston Evidence-Based Management: Three New Approaches to Teaching the Practice of Management Eric Barends Blake Jelley Wendy Carroll Denise Rousseau

2. PDW, Annual AOM 2012, Boston • Denise Rousseau: Introduction • Wendy Carroll & Blake Jelley: Push Approach • Subgroups • Eric Barends: Pull approach • Video (9 min) • Subgroups • Denise Rousseau: Process Approach

3. Definition Evidence-based management means making decisions about the management of employees, teams or organizations through the conscientious, explicit and judicious use of four sources of information: 1. The best available scientific evidence 2. Organizational facts, metrics and characteristics 3. Stakeholders’ values and concerns 4. Practitioner expertise and judgment

4. Four sources

5. Barends, Rousseau, Carroll, & Jelley 2012 Academy of Management PDW Evidence-Based Management: Three New Approaches to Teaching the Practice of Management Teaching EBMgt: The “Push” Approach R. Blake Jelley & Wendy R. Carroll

6. Overview • Our Perspectives and Context • Importance of the “Push” Approach • Principles and Resources

7. Our Perspectives and Context • Our backgrounds • Education • Applied experiences • Teaching in the UPEI School of Business • Undergraduate • EMBA (launched 2008) • Oxford Handbook of EBMgt chapter • Jelley, Carroll, & Rousseau (2012). Reflections on teaching evidence-based management. • Less about the “push” approach

8. Importance of the “Push” Approach • Bounded Rationality, Heuristics, Biases • Kahneman (2011). Thinking, Fast and Slow • “System 1” (fast; automatic) • “System 2” (slow; deliberate) • See also Kahneman & Klein (2009). Conditions for Intuitive Expertise: A Failure to Disagree (2009, American Psychologist)

9. Importance of the “Push” Approach • A path toward the development of expertise in management? • Developing expert skill and intuition (see Kahneman & Klein, 2009; Kahneman, 2011) • A sufficiently regular, predictable environment • Opportunities to learn regularities through prolonged practice and feedback • The management domain is not highly favourable to skilled intuition • Intuition is an important consideration, not the final word • Managers need to avoid overconfidence in intuition

10. Importance of the “Push” Approach • System 1 will engage! • Expert intuition is not magic… • “You can feel [Herbert] Simon’s impatience with the mythologizing of expert intuition when he writes: ‘The situation has provided a cue; this cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition’”(Kahneman, 2011, p. 11).

11. Importance of the “Push” Approach • Making intuition more friendly to EBMgt. • “You do as much homework as possible beforehand so that the intuition is as informed as it can be”(Kahneman, In Kahneman & Klein, 2010, McKinsey Quarterly). • “It is easier to make good decisions quickly if managers are educated and evidence savvy”(John Zanardelli, 2012, p. 196; President & CEO, Ashbury Heights). • Program System 1 with evidence-based principles. • Think fast, well, and set triggers for System 2. • Bolster, not replace, more deliberate processing.

12. Importance of the “Push” Approach • Practitioners are not well-informed about management-related knowledge • E.g., Senior SHRM members = 57% correct (Rynes et al., 2002) • Are educators much better? • Various ways to “push” EB knowledge. • Management education as a key. • Also, ME can integrate push, pull, and process approaches

13. Principles and Resources • Use of Diagnostic Quizzes • Examples… • HRM (Rynes et al., 2002) • “100 things… & 50 more things you need to know” books • Advertising (Armstrong & Green’s adprin.com) • Discussion of dissemination vs. exposing students as uninformed • Links to critical thinking and the “pull” approach

14. Principles and Resources • Concerns about what and how we teach… and who does the teaching • Our body of knowledge • Benefits of systematic research • Volume of research • Focus on novelty over integration, etc. • Pluralism • Textbooks • Instructors • Teaching methods

15. Principles and Resources Other References: • Existing research syntheses; • Individual synthesis and translation articles • E.g., Allen, Bryant, & Vardaman (2010). Retaining Talent: Replacing Misconceptions with Evidence-based Strategies. AOM Perspectives [Best Paper] • SHRM Effective Practice Guidelines; • SHRM-SIOP’s new collaborative series.

16. Principles and Resources • Identify and teach the “core” management body of knowledge; less content, more practice • Focus on topics, theories, and principles that: • (a) Have a solid evidence-base • (b) Are practical to apply • Are role-relevant • Have implications for practice; address important practice issues • Involve procedural as well as declarative knowledge • (c) Are durable • Over time • Applicable in various situations (Miner, 2003; Rousseau & McCarthy, 2007)

17. Since we can’t teach everything, what are the most important evidence-based things we need to program into our students?

18. References Allen, D. G., Bryant, P. C., & Vardaman, J. M. (2010). Retaining talent: Replacing misconceptions with evidence-based strategies. Academy of Management Perspectives, 24(2), 48-64. Armstrong, J. S., & Green, K. C. (2012). Advertising principles: Evidence-based knowledge on persuasion through advertising. Retrieved from http://advertisingprinciples.com/ [see http://advertisingprinciples.com/en/try/test-your-advertising-iq] Charlier, S. D., Brown, K. G., & Rynes, S. L. (2011). Teaching evidence-based management in MBA programs: What evidence is there? Academy of Management Learning & Education, 10(2), 222-236. Eichinger, R. W., Lombardo, M. M., & Ulrich, D. (2004). 100 things you need to know: Best people practices for managers & HR (Vol. 1). Minneapolis, MN: Lominger. Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Strauss, Giroux. Kahneman, D. & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515-526. Kahneman, D. & Klein, G. (2010). When can you trust your gut? McKinsey Quarterly, Issue 2, 58-67. Jelley, R. B., Carroll, W. R., & Rousseau, D. M. (2012). Reflections on teaching evidence-based management. In D. M. Rousseau (Ed.), The Oxford Handbook of Evidence-Based Management (pp. 337-355). New York: Oxford University Press. Latham, G. P. (2009). Becoming the evidence-based manager: Making the science of management work for you. Boston: Davies-Black. Locke, E. A. (2009). Handbook of principles of organizational behavior: Indispensable knowledge for evidence-based management (2nd ed.). Chichester, UK: Wiley. Miner, J. B. (2003). The rated importance, scientific validity, and practical usefulness of organizational behavior theories: A quantitative review. Academy of Management Learning & Education, 2(3), 250-268. Pearce, J. L. (2009). Organizational behavior: Real research for real managers. Irvine, CA: Melvin & Leigh.

19. References Pearce, J. L. (2012). Creating evidence-based management textbooks. In D. M. Rousseau (Ed.), The Oxford Handbook of Evidence-Based Management (pp. 377-386). New York: Oxford University Press. Pfeffer, J., & Sutton, R. I. (2006). Hard facts, dangerous half-truths, and total nonsense. Boston, MA: Harvard Business School Press. Rousseau, D. M. (Ed.) (2012). The Oxford Handbook of Evidence-Based Management. New York: Oxford University Press. Rousseau, D. M. (2012). Designing a better business school: Channelling Herbert Simon, addressing the critics, and developing actionable knowledge for professionalizing managers. Journal of Management Studies, 49(3), 600-618. Rousseau, D. M., & McCarthy, S. (2007). Educating managers from an evidence-based perspective. Academy of Management Learning & Education, 6, 84–101. Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR professionals’ beliefs about effective human resource practices: Correspondence between research and practice. Human Resource Management, 41(2), 149–174. Society for Human Resource Management Foundation (2012). Effective practice guidelines series. http://www.shrm.org/about/foundation/products/pages/default.aspx Society for Industrial and Organizational Psychology & Society for Human Resource Management (2012). Publication and dissemination of science to practice: A research collaboration between the Society for Human Resource Management (SHRM) and the Society for Industrial and Organizational Psychology (SIOP). http://www.siop.org/SIOP-SHRM%5Cdefault.aspx Ulrich, D., Eichinger, R., Kulas, J., & De Meuse, K. (2007). 50 more things you need to know: The science behind best people practices for managers & HR professionals (Vol. 2). Minneapolis, MN: Lominger. Zanardelli, J. (2012). At the intersection of the academy and practice at Ashbury Heights. In D. M. Rousseau (Ed.), The Oxford Handbook of Evidence-Based Management (pp. 191-197). New York: Oxford University Press.

20. PDW, Annual AOM 2012, Boston Evidence-Based Management: Three New Approaches to Teaching the Practice of Management Part 2: The 5-step pull approach Eric Barends

21. Trust me, I’m a manager.

22. Push vs Pull Push: teaching management principles based upon a convergent body of research and telling students what to do. Pull: teaching students how to find, appraise and apply the outcome of research (evidence) by themselves

23. Why do we need a pull approach?

24. Problem I: too much ‘evidence’ • HRM: 1,400 articles in 2011 (ABI/INFORM). For an HR manager to keep up this means reading 3 to 4 articles every day (for a ‘general’ manager more than 50!)

25. Problem II: false information • Half of what you learn will be shown to be either dead wrong or out-of-date within 7 years of your graduation; the trouble is that nobody can tell you which half

26. Problem III: half time value 5 years? 7 years? 10 years?

27. Pull Pull: teaching students how to find, appraise and apply evidence by themselves

28. Starting point Start with a practical question, (not with an academic answer) • Problem based • Real live case • Just in time

29. The 5 steps of ‘pull’ EBP • Formulate a focused question (Ask) • Search for the best available evidence (Acquire) • Critically appraise the evidence (Appraise) • Integrate the evidence with your managerial expertise and organisational concerns and apply (Apply) • Monitor the outcome (Assess)

30. 1. Formulate a focused question

31. Asking the right question? Postgraduate Course • Does team-building work? • Does the introduction of self-steering teams work? • Does management development improve the performance of managers? • Does employee participation prevent resistance to change? • Is 360 degree feedback effective?

32. What is a ‘team’? What kind of team? In what contexts/ settings? What counts as ‘team-building’? What does ‘work’ mean? What outcomes are relevant? Over what time periods? Focused question? • Does team-building work?

33. Answerable question: PICOC Postgraduate Course P= Population I = Intervention or success factor C= Comparison O= Outcome C = Context

34. Focused question: PICOC Postgraduate Course P= Population I = Intervention or successfactor C= Comparison O= Outcome C = Context • Employee productivity? • Job satisfaction? • Return on investment? • Market share? • Organizational commitment?

35. Postgraduate Course 2. Finding the best available evidence

36. Searching evidence Postgraduate Course Where do we search?

37. Databases Postgraduate Course • ABI/INFORM • Business Source Elite • PsycINFO • Web of Knowledge • ERIC • Google Scholar

38. Searching evidence Postgraduate Course How do we search? Search Strategy

39. Two types of search strategies Building blocks method Snowball method Search strategy Postgraduate Course

40. Hands on instruction Postgraduate Course

41. 3. Critical appraisal of studies Making sense of evidence

42. The best available evidence = Studies with the highest internal validity (does it work?) Studies with the highest external validity (does it work for my employees / my organization?)

43. Research designs Whichstudyforwhich question? The “best” evidence depends on the question type !

44. Which design for which question? Explanation

45. Best research design?

46. Critical appraisal

47. Critical appraisal Postgraduate Course Is the study design appropriate to the stated aims? Was a control group used? Was a pretest used? Are the measurements likely to be valid and reliable? Could bias or confounding have occurred? How large was the effect size?

48. Step 4: Turning evidence into practice

49. Organization concerns Always ask yourself to what extent the evidence is applicable in your situation: • Is your organization / population so different from those in the study that its results are difficult to apply? • How relevant is the study (or outcome) to what you are seeking to understand or decide? • What are your organization’s potential benefits and harms from the intervention? • Is the intervention feasible in your setting?

50. Four sources