Evidence-Based Management

1. University of Edinburgh, May 2nd 2014 Evidence-Based Practice InsightsfromKeyDomains Evidence-Based Management Eric Barends Rob Briner

2. Evidence-Based Management A brief history Currentdevelopments Futuredevelopments Challenges Implicationsfor practice & research

3. Evidence based management: A brief history

4. Management? Manager > ‘Manage’ introducedmid 16th century: fromItalianmaneggiare, “to handle”, based on Latin manus, “hand”. (we’reall managers)

5. History of management/business schools Late 1800s - started as trade schools 1950s - Ford and Carnegie reports criticised business schools for not being sufficiently academic or scientific 1980s onwards – criticised for producing research irrelevant to practice and students without relevant skills or knowledge

6. History of management/business schools On the one hand they fear . . . the scorn of other, more traditional academic subjects. On the other hand, they often stand accused of being less than relevant to business. (Grey, 2001) 2000s onwards – criticized for producing ideas and students that lead to financial crisis and poor ethical behaviour (e.g., Enron, only one US president has MBA, MBA students cheat more than other graduate students)

7. Management schools are big business • Nat. Center for Education Statistics (USA, 2011-’12) • 20% of all Batchelor’s Degrees • 25% of all Master’s Degrees • Higher Education Statistics Agency (UK, 2012-’13) • 27% of all Higher Degrees awarded • 18% of all Higher Education qualifications obtained • UK Council for International Student Affairs (2011-’12) • 36% of all business school students are international • 30% of all international students studying business

8. EBP in Management Many similar ideas in past (e.g., Mode 1-Mode 2, collaborative research, rigour and relevance) Rousseau (2005) Presidential address Peffer and Sutton (2006) Hard Facts, Dangerous Half-Truths, and Total Nonsense: Profiting from Evidence-based Management

9. EBP in Management EBMgt Collaborative (2007-’08) Rousseau/Carnegie Mellon Wiley Evidence-Based Management Insights (2007-’08) (attempt to create Cochrane database for management) Briner and Denyer Presentation to AoM Board of Governors (2008) (attempt to persuade to develop systematic reviews in management) Center for Evidence-Based Management – more later

10. Evidence based management: What is it?

12. Evidence-based practice Central Premise: Decisions should be based on a combination of critical thinking and the ‘best available evidence‘.

13. Evidence? findings from scientific research, organizational facts & figures, benchmarking, best practices, professional experience

14. All managers base their decisions on ‘evidence’

15. But…many managers pay little or no attention to the quality of the evidence they base their decisions on

16. Trust me, 20 years of management experience

17. SO ...

18. Teach managers how to critically evaluate the validity, and generalizability of the evidence and help them find ‘the best available’ evidence

19. Definition Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources by 1. Asking:translating a practical issue or problem into an answerable question 2. Acquiring:systematically searching for and retrieving the evidence 3. Appraising:critically judging the trustworthiness and relevance of the evidence 4. Aggregating: weighing and pulling together the evidence 5. Applying:incorporating the evidence in the decision-making process 6. Assessing:evaluating the outcome of the decision taken to increase the likelihood of a favorable outcome.

20. Evidence based decision • Organizational data, facts and figures • Professional experience and judgment • Ask • Acquire • Appraise • Aggregate • Apply • Assess • Scientific research findings • Stakeholders’ values and concerns

21. Evidence-based practice: Focus on the decision making process Think in terms of probability

22. Differences? • Research: low internal validity

23. Controlled vs uncontrolled studies

24. Differences? • Research: low internal validity • Lack of evidence summaries • Focus on multiple sources • Organizational (BIG) data • Stakeholders’ concerns

25. How evidence-based are we (managers)? “I’ve never thought I need more evidence before making a decision; I know what needs to be done, we get on with it and we get results.”

26. How evidence-based are we? HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008) • 959 (US) + 626 (Dutch) HR professionals • 35 statements, based on an extensive body of evidence • true / false / uncertain

27. True (likely) or false (not likely)? Incompetent people benefit more from feedback than highly competent people. Task conflict improves work group performance while relational conflict harms it. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals.

28. Outcome: not better than random chance

29. Evidence based management: Current developments

31. Postgraduate Course

32. Postgraduate Course

33. Current developments Postgraduate Course • Education • Access to research databases • REAs • Building a community

34. Evidence based management: Future developments

35. Future developments: practice Postgraduate Course • CEBMa Database of Evidence Summaries • Online learning modules • Accreditational bodies

36. Future developments: academia Postgraduate Course • Some move towards systematic review methods in management • Limited interest in teaching EBP in management schools • Management schools still a bit trapped: Cash cows for universities; focus on ‘top quality’ academic research; few incentives to be relevant

37. Future developments: academia Postgraduate Course • Though issues around accountability, ethics, corporate social responsibility could in part be resolved by research and teaching EBP • Need for a professional doctorate?

38. Evidence based management: Challenges

39. Challenges (sameforpractitionersandacademics) In general incentives to do non-EBP and punishment (or no incentives) for doing EBP

40. Incentives for managers Not rewarded for doing what ‘works’ – few evaluations Speed and action valued more highly than accuracy and analysis Managing and understanding power and politics to get things done more valued than understanding and using evidence to make decisions It may be too late to change existing senior managers…

41. Other challenges • Barriers from academic / buss school context • In publishing and research • In teaching content and teaching style • Barriers from organizational contexts • Managers love fads and quick fixes • Power and politics

42. In general: academics don’t like EBMgt

43. Why don’t academics like EBMgt? • Ambivalence about the value and applicability of management research • Few incentives to get involved • Primary research (collecting new data) valued more highly than secondary research (reviewing existing data) • EBMgt not academics‘responsibility – this is about practice not research • Some concern that systematic reviews will expose the limited nature of management research • Some academics are like ‘gurus’ and feel that EBMgt might show their claims to be untrue

44. Espoused and more implicit goals of management school educators ESPOUSED GOALS • To develop critical thinking • To help students understand the full body of knowledge • To discuss gaps and limitations of our knowledge • To challenge thinking and assumptions • To educate • To teach students how to think for themselves • To maintain quality standards IMPLICIT GOALS • To help students feel successful • To select those bits of that are interesting or digestible or ‘cutting edge’ fads • To reassure students that what we’re teaching them is solid • To make sure students are satisfied • To entertain (edutainment) • Teach students what they need to pass assessments • To give ever-higher grades

45. Espoused and more implicit goals of management school researchers IMPLICIT GOALS • To advance career • Use whatever techniques will get you published • Publishing (mostly) only positive results, no replications • Identifying ‘new’ or trendy topics – creating empires • Exaggerating how much we know • Locking up our evidence behind publishers’ pay walls • Competition for resources, slots in journals, between universities ESPOUSED GOALS • To advance scientific understanding • Using the best research techniques • Publishing all results and replications – unbiased • Focus on what’s important • Being honest about existing evidence • To disseminate all our evidence and make publically available • Collaboration & cooperation

46. Implications for research • Less focus on collecting new primary data • More focus on systematic reviews and understanding what we know and don’t know • Focusing research efforts on what is academically and/or practically important without compromising ‘academic freedom’ • Broader-based research training – management researchers highly and narrowly specialized

47. In general: managers don’t like EBMgt WHAT’S STOPING US?

48. Why don’t managers like EBMgt? • Undermines formal authority • They feel it constrains freedom to make managerial decisions • Speed valued and rewarded more than accuracy • Feel they cannot use their own experience and judgment (not true) • Managers not necessarily rewarded for doing what works (organizations rarely evaluate) • THEY LOVE FADS & QUICK FIXES

49. Fads & quick fixes Huge (peer) pressure to adopt fads & quick fixes