Implementing Evidence-Based Practices in Mental Health: Insights from Dartmouth’s EBP Project

1. Getting Practices That Work to People Who Need Them William C. Torrey M.D. Geisel School of Medicine At Dartmouth May 4, 2012

2. Outline of the Talk • Overview of the issue • Lessons from the National Implementing Evidenced-Based Practices Project • Lessons from the Collaborative Care implementing literature • Current research and clinical efforts at Dartmouth • Discussion

3. My Background

4. Overview of the Issue • Three legged stool • Clinical model that works • Operations • Finance

5. The National Implementing EBP Project

6. The good news There has been a dramatic expansion of knowledge about what works.

7. The bad news Very few people have access to the practices that work.

8. 1998 RWJ Conference on Evidence-Based Practices • Assertive community treatment (ACT) • Supported employment (SE) • Integrated dual disorder treatment (IDDT) • Illness management and recovery (IMR) • Family psychoeducation (FPE)

9. Practices do not implement themselves

10. Phase I : Develop the Implementation Resources • Develop the implementation model • Create “toolkits” • Organize the training and consultation

11. What does not work • Dissemination of information (guidelines and research literature) alone • Training alone

12. Promoting change • Motivating change: Why change? • Enabling change: How to change? • Reinforcing change: How to maintain and extend the gains?

13. Why Implement ? • Introductory brochures for different stakeholders • Introductory video • Introductory Powerpoint presentation

14. How to Implement ? • Implementation tips • Clinician workbook • Skills video

15. How to Maintain and Extend the Gains ? • Fidelity measurement • Outcomes measurement • Feedback recommendations

17. Phase II : Field Test Examined 5 psychosocial EBPs 53 sites started in 8 states Each state implemented 2 different EBPs in multiple sites 2 years of qualitative observation of implementation factors Fidelity reviews every 6 months

19. Fidelity

20. Dimensions of Implementation • Fidelity • Affordability • Effectiveness • Appropriateness • Penetration (how many people gain access to the practice)

21. EBP Fidelity Degree to which a particular program follows the standards of the practice that has been shown to work.

22. Uses of Fidelity Scales • Research • Quality improvement • Accreditation

23. EBP Fidelity Scales • Quantitative multi-item scales based on objective criteria derived from model specification • Assessment based on daylong site visits • Items rated on 5-point behaviorally-anchored continuum • ≥ 4.0 considered good implementation

24. National EBP Project: 2-Year Rates of Successful Program Implementation

25. Barriers, facilitators, and strategies

26. Implementation Factors: Data Collection Procedures • Implementation monitors recorded notes at study sites ~ monthly for 2 years • Periodic interviews with key staff • Notes and interviews entered as documents in qualitative data base (Atlas)

27. Coding System • Each event coded according to • Type: Barrier, Facilitator, or Strategy • Content: 26 Dimensions grouped into 5 Domains

28. Conceptual Framework for Implementation Factors

29. Total Number of Units Coded

30. Qualitative Analysis: What helps and hinders implementation?

31. What helps implementation? • Active on-site leadership • Management of staff turnover • Getting the right staff • Technical, financial, and political support from the larger administrative environment

32. What hurts implementation? • Passive “laissez faire” administrative leaders • Overwhelming staff turnover • Passive or active opposition from physicians or other key leaders

33. Quantitative analysis of qualitative data

34. Domain Proportions in Total Sample

35. Correlations: Implementation Factors over Both Years with 24-Month EBP Fidelity

36. Conclusions from this analysis Active, observable Leadership has dramatic impact on implementation A focus on Work Flow(policies, documentation) and Reinforcement (fidelity, outcome monitoring, and feedback) may be best strategy Work Forcefacilitators and strategies had a puzzling negative relationship with fidelity

37. Do they sustain?

38. 4 years later • Money • Measurement

39. Netherlands Study on Mental Health Practice Implementation Which implemented well? • Active inspirational team leadership • Support of the management

40. Reflecting on what we learned

41. Boiled down advice • Choose an active engaged site leader and empower this person • Leader should focus on: • Picking the right staff • Actively changing the flow of daily work • Measuring and using data to manage • Don’t just train! • Provide ongoing commitment and support from larger administrative environment • $ strategy to sustain

42. How does this compare to implementing collaborative care?

43. Fidelity Matters in Collaborative Care

44. Collaborative Care Thota Meta-Analysis (2012) High fidelity collaborative care works !

45. What Improves?(Thota 2012) • Depressive symptoms • Adherence to treatment • Response to treatment • Remission of symptoms • Recovery from symptoms • Quality of life • Satisfaction with care

46. What is it that actually works?Gilbody (2006) and Thota (2012) Studies show effectiveness for collaborative care that includes 3 collaborative components: • a case manager • a primary care physician • access to mental health specialist input

48. 3 Component Model - Fidelity

49. Jürgen Unützer, MD, MPH, MAon Fidelity • Studies on the correlation between fidelity and depression outcomes going on now • Recommends tracking depression outcomes and comparing to benchmark. If low move toward more fidelity. • Most important fidelity item? – track depression outcomes.

50. Barriers and facilitators to implementing and sustaining collaborative care