Making Changes Last- Sustainability for the OTP Project Thomas R Zastowny, PhD

1. Making Changes Last- Sustainability for the OTP Project Thomas R Zastowny, PhD NIATx Coach & Healthcare Consultant & all OTP coaches Reduce Waiting & No-Shows  Increase Admissions & Continuation All authors share equally in this presentation

2. PRESENTATION • Background & Theory • A Field Study • Work-Sheets & Methods • British Sustainability Model • Summary • Bonus Slides

3. CONCEPTS NIATx has achieved “first stage validity”

4. CONCEPTS • Diffusion, spread and sustainability are inter-related and inter-dependent • A specific plan, customized to organization, culture and system, is required for maximal stability & portability and sustainability • NIATx has achieved “first stage validity” • Precise Definitions, Planning and Methods are required for sustainability

5. Background • As we have witnessed the significant and powerful performance improvement changes associated with NIATx, our attention has turned to sustainability of improvements. Webster has defined sustainability in many ways but including these two succinct and clear elements: “A characteristic of a process or state that can be maintained indefinitely; to keep in existence, to maintain or prolong”What family of strategies then can help us maintain the important changes we have made?

6. Concept & Definition Background Diffusion Spread Sustainability

7. Concept & Definition A P S D METHOD Spread Scatter Share Switch Stretch Sustainability Diffusion

8. Four NIATx Aims Reduce Waiting Times Reduce No-Shows Increase Admissions Increase Continuation Rates

9. Change CyclesPD-SA-Sustain A P S D D S P A A P S D A P S D Changes That Result in Improvement DATA Rapid cycle changes should be do-able in 2 weeks !!!!! Maybe….. Hunches Theories Ideas

10. ASK: Sustain What? The Projects--PDSAs The Culture of Improvement Evidence Based Practice Practice Based Evidence

11. Qualitative & Quantitative Bonus Slide

12. The importance of sustainability Important to not only implement a change but also to see whether it continues – Pluye, et al. 2004 Most implementation is able to be sustained for at least a limited amount of time - Porowski, et al. 2004; Scheirer, 2004 Activities become routine when they reflect the collective values and beliefs of members – Capoccia, et al., 2007; Pluye, et al., 2004 12 NIATx 200 - March 25, 2009

13. Sustainability and Institutionalization-Business Case Effectively maintaining an operation that improves services Maintaining financial feasibility and producing positive outcomes through consistent interagency collaboration Programs ability to generate continuation, growth, and support NIATx 200 - March 25, 2009 13

14. Oregon Health & Science University Sustaining Change in NIATx Katherine (Katie) Riley, Ed.D. Traci Rieckmann, Ph.D. A Field Study

15. 2 major Research questions NIATx goals: Increase access and retention 1. Was this accomplished? 2. Were the NIATx components sustained? Quantitative & Qualitative assessment to determine if this was accomplished. NIATx 200 - March 25, 2009

16. Evaluation process Qualitative data collected from 38 agencies Quarterly calls – 302 interviews Site visits – 121 interviews + 119 focus groups Evaluator observations – 111 summaries Time period Base grant funding – 18 mo. RWJ, R1 & R2; 36 mo. STAR (2003-2006) NIDA follow-up – additional 18 mo. (2006-2008) NIATx 200 - March 25, 2009

17. Which core components remained? Change leader – 35 of 38 agencies Top leadership support – 34 + 3 partial Client focus – 34 +1 to some extent Data collection – 33 + 3 some sources Data review – 34 + 1 somewhat Change teams – 28 agencies PDSA’s – 22 agencies NIATx 200 - March 25, 2009

18. Fidelity = Sustainability Use the Tools-Project Charter Data is your Friend- collect,aggregate,analyze;-Decision Support PDSA your way to improvement THE BUSINESS CASE & MISSION CASE BONUS SLIDE

19. Other components still in use Strategy techniques: Walkthroughs – 8 + agencies Nominal group – 2 NPO ongoing learning components: Website – 1 Interest circle participation – 8 ~ others Learning collaboratives – well liked Monthly conference calls – mostly not mentioned Coaches – no longer under contract NIATx 200 - March 25, 2009

20. Change Initiatives Most were sustained - key influences: Roadblocks and barriers: Staffing turnover - 25 agencies “Change team meetings have been suspended due to staffing difficulties and the PDSA cycles are not being used.” Staff resistance – 18 agencies “They abandoned the attempt to integrate the Session Rating Scale and the Outcomes Rating Scale into clinical sessions. Counselors never were very supportive. But, they have used the experience to make their treatment planning process more client centered.” Unforeseen consequences – 11 agencies Too many changes at one time – 2 agencies NIATx 200 - March 25, 2009

21. Change Initiatives Wings beneath your feet Leadership support – 34 +3 partial Data review – 34 + 1 (ongoing monitoring) Internal coaching - 16 “I sent out emails to clinical staff, reminded them to give feedback. I didn’t do that kind of stuff with the second team so staff just forgot.” Another Change Leader said: “We need to be refreshed, reminded, and then implement [changes] again.” NIATx 200 - March 25, 2009

22. Use & Dissemination of the“Walk Through”as an accepted and powerful methodology Real Use of theRapid Cycle Performance Improvement-Real people, Real data, Real improvement, Real fast The development and creation of theBusiness Case-first combinations of Quality, Cost, Improvement and Impact Creation and accelerated use of“Learning Collaborative (s) & Communities” Nourish the Learning Community/Collaborative Bonus Slide

23. Experience speaks Monitoring prevents negative spirals: “Participants noted that it is important to continue to monitor changes after implementation. Otherwise, staff will tend to revert to old practices. Participants provided examples of previous changes undertaken in the outpatient programs that had not been sustained.” Regular meetings allow important data review: “The agency was reminded of the importance of looking at data every single month and having good indicators in place to flag problems.” NIATx 200 - March 25, 2009 23

24. Culture change Implementation of policies and procedures; another mechanism to change and sustain culture: “Staffing changes give challenges to sustainability. One good thing is that those changes that were working became standard so new staff didn’t know anything but the changed practices. It’s not extra; it’s just what we do!” NIATx 200 - March 25, 2009 24

25. Not every change is a keeper! Data revelations! “We’ve sustained most of the changes but when something hasn’t worked, we abandon it.” Client revelations! “We tried [making reminder calls] for group [attendance] but it was too much: clients didn’t like all of the phone calls—they could be getting 4 phone calls a week. We amended the change because of their feedback.” NIATx 200 - March 25, 2009 25

26. Inquiry (1) What organizational and operational structures (e.g., policies, procedures, work guidelines for providing service, ownership of the process?) are in place to hold the gain?

27. Inquiry (2) What are the clinical, business and oversight processes that assure continuation? (e.g., stability, integration into the organization culture, allowance of fluctuations within statistical control, sustain plans and staff ownership)

28. BONUS SLIDE-Project Charter/Change Form Sustainability Plan (only complete if you are sustaining the changes) A. Who is the SUSTAIN LEADER? B. What CHANGES do you want TO SUSTAIN? C. What SUSTAIN STEPS are being taken to ensure that the changes stay in place and that it is not possible to revert back to the old way of doing things? D. What is the TARGET SUSTAIN MEASURE, i.e. the point at which the Change Team would intervene to get the project back on track? E. What system is in place to effectively MONITOR the SUSTAIN MEASURE?

29. Inquiry (3) What are the outputs, impacts and outcomes we continue to want to see to know sustainability is working? (e.g., definitions-, i.e. same day service, continuation & client participation, and intervals of measurement to gauge stability, increases in access and retention).

30. Inquiry (4) What is the Business Case for the Improvement and Sustainability? ( e.g. cost benefit, dollars and sense, new revenue)

31. One Story ▲One organization’s plan for sustainability for access included the following…(a) two policy changes to ensure same day treatment, (b) a mandate to revisit the process if same day treatment was unsuccessful for 2 contiguous days, (c) continuous measurement in the first year after the improvement using a step down approach (e.g., measure weekly, then monthly, then quarterly ,then yearly).A well crafted plan for sustainability is a must for longevity. Inclusion of these dimensions can help improve the precision of the sustainability plan and cover a wide reaching set of strategies to “hold the gain” across the organization. This strategy works equally as well within single and complex organizations, and state wide collaboratives.

32. British National Health Services Sustainability Model The NHS Sustainability Model is a diagnostic scoring model designed to be used by each member of the core project team. The information collected provides the whole team with a really good understanding of areas that need improving in order to maximize the potential of sustainability within the project. An accompanying Sustainability Guide provides a range of practical ideas about how to actually make those improvements. http://www.institute.nhs.uk/sustainability_model/general/welcome_to_sustainability.html

33. British National Health Services Sustainability Model http://www.institute.nhs.uk/sustainability_model/general/welcome_to_sustainability.html The Model Interactive Assessment Version Examples & Sustainability Videos Bonus

34. British National Health Services Sustainability Model

35. British National Health Services Sustainability Model Service Sustainability Rating for ____________________________ (Name of Program, Project or Improvement Initiative) Select the level of each factor that best describes your situation and circle the corresponding number. Your complete honesty is appreciated. The closer your score is to 100, the better chance of successful sustainability. A score of 55 or higher offers reason for optimism while a score of 45 or lower suggests reason for concern

36. British National Health Services Sustainability Model The NHS Sustainability Model provides a useful framework which helps you: • Assess your improvement initiative against a number of key criterion for sustaining change • Recognize and understand key barriers to sustainability relating to the specific local context • Identify your current strengths in sustaining improvement • Monitor progress over time

37. British National Health Services Sustainability Model • Benefits Beyond Helping Patients • Credibility to Affected Staff of Benefits From Improvement • Adaptability of Improved Process • Staff Involvement & Training to Sustain Process • Staff Attitude Toward Sustaining the Improved Process • Senior Leadership Responsibility for the Process

38. British National Health Services Sustainability Model 7. Clinical Leadership responsibility for the Process 8. Effectiveness of the System to Monitor Progress & Process 9. Fit with Organization’s Strategic Aims & Culture 10. Infra Structure to Sustain- e.g. Staff, Facilities, Equipment,Time

39. WORKSHEET (1) What organizational and operational structures (e.g., policies, procedures, work guidelines for providing service, ownership of the process?) are in place to hold the gain? (2) What are the clinical, business and oversight processes that assure continuation? (e.g., stability, integration into the organization culture, allowance of fluctuations within statistical control, sustain plans and staff ownership) (3) What are the outputs, impacts and outcomes we continue to want to see to know sustainability is working? (e.g., definitions-, i.e. same day service, continuation & client participation, and intervals of measurement to gauge stability, increases in access and retention). (4) What is the Business Case for the Improvement and Sustainability? ( e.g. cost benefit, dollars and sense, new revenue)

40. WORKSHEETSustaining the Gains Describe the change you want to sustain What are the organizational structures that can be put in place to help preserve the process changes you have made?  What are the ongoing data needs that will help the organization know if the desired change is being sustained? Who will gather this data? Who will review it and when? Is there a standard meeting that could own this responsibility?  What is the business case for the improvement you want to sustain?

41. SUMMARYSustaining the Gains • PDSA SUSTAIN • Fidelity • Worksheets • Business Case • Sustainability Model • Plan for Attrition-Staff & Leadership • Build Culture of Sustainability • A simple plan for sustainability • Build it in-not added on-Quality By Design • Nourish the learning Collaborative

43. References Akerlund, K. M. (2000). Prevention program sustainability: The state’s perspective. Journal of Community Psychology, 28, 353–362. . Backer, T.E. (2000). The Failure of Success: Challenges of Disseminating Effective Substance Abuse Prevention Programs. Journal of Community Psychology, 28 (3), 363-373. Capoccia, V.A., et al. (2007). Making “Stone Soup”: Improvement in Clinic Access and Retention in Addiction Treatment. Journal on Quality and Patient Safety, 33 (2), 95-103. Commons, M., McGuire, T.G., Riordan, M.H. (1997). Performance Contracting for Substance Abuse Treatment. HSR: Health Services Research, 32 (5), 631- 650. Fitzgerald, M. (2000). Operator assistance with process improvement. Addictions Management, 21-22. Gustafson D.H.: Designing systems to improve addiction treatment: The foundation. Alcoholism and Drug Abuse Weekly 14, Nov. 4, 2002. Hoffman, , K.A., Ford, J.H., Choi, D, Gustafson, D.H., McCarty, D (2008). Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction treatment, Drug and Alcohol Dependence 98 (1-2) 63-69. Johnson, K., Hays, C., Center, H., Daley, C. (2004). Building Capacity and sustainable prevention innovations: a sustainability planning model. Evaluation and Program Planning, 27, 135-149. Lake, B., Walker R. (2005). Report on Sustainability and Expansion of North Carolina’s Drug Treatment Courts. Administrative Office of the Courts, 1-57 NIATx 200 - March 25, 2009

44. References McCarty, D. et al. (2007). The Network for the Improvement of Addiction treatment (NIATx): Enhancing Access and Retention. Drug Alcohol Depend, 88(2-3), 138-145 Porowski, A.W., Burgdorf, K., Herrell, J.M. (2004) Effectiveness and sustainability of residential substance abuse programs for pregnant and parenting women. Evaluation and Program Planning, 27, 191 Pluye, P., Potvin, L., Denis, J. L.(2004). Making public health programs last: conceptualizing sustainability. Evaluation and Program Planning, 27, 121-133 Pluye, P., Potvin, L. Denis, J.L., Pelletier, J. (2004). Program Sustainability: focus on organizational routines. Health Promotion International. 19 (4), 489- 500 Scheirer, M.A., (2005). Is sustainability Possible? A review commentary on Empirical studies of program sustainably. American Journal of Evaluation, 26, 3, 320- 347 Thompson, B., Lichtenstein, E., Corbett, K., Nettekoven, L., & Feng, Z. (2000). Durability of tobacco control efforts in the 22 Community Intervention Trial for Smoking Cessation (COMMIT) communities 2 years after the end of interventions. Health Education Research, 15, 353–366. Thompson, B., &Winner, C. (1999). Durability of community intervention programs: definitions, empirical studies, and strategic planning. In N. Bracht (Ed.), Health promotion at the community level (pp. 137–154). Thousand Oaks: Sage Wisdom, J. P., Ford, J. H., Hayes, R. A., Edmondson, E., Hoffman, K., & McCarty, D. (2006). Addiction treatment agencies' use of data: a qualitative assessment. Journal of Behavioral Health Services & Research, 33(4), 394-407. NIATx 200 - March 25, 2009