Assertive Community Treatment in Maine: Evaluating Fidelity, Service Use and Outcomes

1. Assertive Community Treatment in Maine: Evaluating Fidelity, Service Use and Outcomes Karen Glew, M.S.P.A. James T. Yoe, Ph.D. Contact: Karen Glew, MSPA Karen.Glew@maine.gov (207)287-4210

2. Evidence Based Practice Use of Evidence Based Practices in Assertive Community Treatment have demonstrated positive outcomes in multiple research studies ACT • Lower use of inpatient service • Better quality of life • More independent living • Better substance abuse outcomes • Higher rate of competitive employment • Greater consumer and family member satisfaction

3. Vision & Implementation of EBP in Maine • DHHS EBP Coordinating Committee has been working to develop an EBP Policy Framework in order to: (1) Develop a shared vision for the development and implementation of EBP, (2) Articulate a policy framework and principles to guide work, (3) Continued quality improvement and decision making based on data, and (4) Guide the Department concerning the place of EBPs in the delivery of human services. • Guiding Principles: Broad definition of health, Wide spectrum of Interventions, Person-centeredness, Partnership building, Decision-making based on data, Outcome oriented, Quality services, Informed choice, respect, Individualized Service and Cultural Competence.

4. Purpose of ACT Implementation Review • To evaluate the level of fidelity of the 10 ACT teams in Maine. • To evaluate relationships between team fidelity, service use and consumer outcomes. • To improve understanding of individuals receiving ACT services. • To understand the nature and consistency of practices across service providers and the extent to which practices meet national implementation guidelines. • Use results to improve the quality and consistency of ACT services.

5. ACT in Maine: Organizing the Assessment • The fidelity evaluation was coordinated and conducted through the Office of Quality Improvement in collaboration with Office of Adult Mental Health Services. • Letter was sent to agencies providing ACT services describing the process and asking for a contact. • Fidelity was assessed using the ACT Implementation Resource Kit (SAMHSA). • Training: • Consumers and representatives of the DHHS Office of Quality Improvement and Office of Adult Mental Health Services participated in a five week training. • Interviewer training included mock interviews to allow for practical experience in interviewing, record reviews and using the fidelity scale.

6. Data Collection • Inter-rater Reliability: Teams of two evaluators reviewed all documentation and jointly conducted all interviews. • Pre Review Data Collection: Data collected prior to the site visit included information such as: • Number of individuals served • List of all current staff • Policies such as admission and discharge criteria • Information on ACT consumers admissions and discharges from hospitals. • Random Record Review: From each team 10 individual case files were reviewed. • Interviews: Assessors conducted interviews with 10 individuals, a clinician, the team leader, and the Substance Abuse Specialist.

7. Findings: Who Receives ACT Services • Primary Diagnosis (n=231) • 51.5% of ACT recipient’s primary diagnosis was Schizophrenia • 17.7% Bipolar Disorder • 14.3% Depression • 11.3% Trauma Related • 5.2% Other • Age (n=231) • 54.6% of recipients of ACT services were between 30 and 49 years old. • Gender (n=231) • Male and females are evenly dispersed across ACT teams (males = 51.1 % and females 48.9%)

8. Level of Implementation • Overall, ACT teams scored moderately high on the fidelity scale. Teams averaged 4.05 on a scale of 1 to 5. • Average of the 10 ACT teams ranged from 3.32 – 4.56. • Level of implementation demonstrated some variability: • Human Resources: 3.78 to 4.6. • Organizational Boundaries: 4.14 to 4.86. • Nature of Services: 2.11 to 4.22.

9. Human Resources Domain

10. Human Resources Implementation Strength Scores indicate that teams have adequate personnel and clinical staff to provide multi-disciplinary ACT services and appropriate consumer to staff ratios. Need Current implementation of H4: Practicing Team Leader at 25%, based on national ACT fidelity standards, full implementation would result in supervisor or team leader to provide direct services at least 50% of time.

11. Organizational BoundariesDomain

12. Organizational Boundaries Implementation Strength ACT teams scored uniformly high on O3: Full Responsibility for Treatment Services with an average Statewide Fidelity of 4.6. Seven of the 10 ACT Teams received a rating of 5, indicating full implementation. Need Responsibility for hospital Admissions (O5) received the lowest statewide fidelity rating in this area. A rating of 4 indicates that the ACT team is involved in 65% to 94% of admissions. Full Implementation requires 95% to 100%.

13. Nature of Services • Fidelity ratings for this domain were quite variable across ACT teams • Fidelity ratings yielded substantially lower fidelity ratings compared to domain areas of Human Resources and Organizational Boundaries • As shown in the figure, individual ACT teams differed widely with fidelity ratings ranging from 2.11 to 4.22

14. Nature of Services Implementation Strength: • Intensity of Contact Need: • Integrated Treatment • Community Contact • Work with Informal System • Frequency of Contact

15. Using Fidelity for Outcomes • Assigned each individual in sample the overall fidelity average from his/her ACT team. • Divided the sample group into high and low based on a median split of the State fidelity average. • Compared differences in LOCUS* scores from initial assessment to assessment completed approximately 12 months. • Grouped the individuals as improved, no change or decreased based on the difference of locus scores. * Level of Care Utilization System

16. Fidelity Related to Outcomes • Individuals receiving ACT services from teams with higher implementation to the ACT model, showed greater improvement in LOCUS scores over time. • Individuals receiving ACT services from teams with lower implementation to the ACT model, showed no improvement or remained stable in LOCUS scores over time. n=209

17. Fidelity Items Linked to Outcomes • Full Responsibility for Treatment Services • Work with Informal Support System • Frequency of Contact

18. Summary of Findings • Critical personnel resources of ACT services are present and teams are mostly providing services as a team • Teams are challenged with providing substance abuse services both at an individual and group level to those with a dual disorder • Teams are not necessarily providing the services from a community orientation as shown by the low implementation of the informal network system • The following items were found to have made a significant difference in outcomes • High number of face-to-face contacts per week • In addition to case management and psychiatric services, program directly provides counseling/psychotherapy, housing support, substance abuse treatment, employment and rehabilitative services • With or without ACT services present, provide support and skill for individual support network in the community (family, landlord, employers, etc.)

19. Next Steps • Continued evaluation on the relationship between ACT team fidelity scores and individual outcomes • Investigate the relationship between team fidelity and the use of high cost services • Re-examining the relationship between team fidelity scores and functional outcomes using a larger sample of individuals receiving ACT services • To improve areas of need based on ACT implementation reviews