Milestones of Recovery: Presentation for the CMHDA Adult System of Care Committee

1. Milestones of Recovery:Presentation for the CMHDAAdult System of Care Committee Dave Pilon, Ph.D., C.P.R.P. National Mental Health Association of Greater Los Angeles dpilon@mhala.org February 14, 2007

2. Goals of the Training • Discuss difficulties in the measurement of “Recovery” • Describe our attempts to create an instrument that classifies consumers according to their current “milestone of recovery” • Begin the discussion of how the Milestones can be used create “flow” through the system and increase program and system accountability.

3. Recovery as the Basis for Services under MHSA (From Section 7) “Planning for services shall be consistent with the philosophy, principles, and practices of the Recovery Vision for mental health consumers: (1) To promote concepts key to the recovery for individuals who have mental illness: hope, personal empowerment, respect, social connections, self-responsibility, and self-determination.”

4. What is Recovery? • Many consumers speak of recovery in terms of their own internal experience – often phrased in such terms as “becoming empowered,” “taking charge of their own lives,” “improving their self-esteem,” or “becoming responsible for themselves.” • The mitigation of psychiatric symptoms (or symptom distress) and improvement in functioning. • Identifying and taking on meaningful roles in one’s life.

5. Recovery Definition Matrix

6. Limitations of the Milestones • By definition, no classification system can do justice to the uniqueness of the individuals it is classifying • Many consumers resent (justifiably) any attempt to reduce their experience to a single number

7. Milestones of Recovery Project • In 1997, with a grant from the Center for Mental Health Services, the California Association of Social Rehabilitation Agencies (CASRA) convened a group of 50 administrators, clinicians and consumers. • This group was given the task of creating a system that would classify consumers in particular clusters according to their needs in a way that would enable the providers of services to be held accountable for the outcomes of their services.

8. CASRA CSP PROJECTOUTCOME DOMAINS • Risk of Involuntary Treatment • ADL/ILS Functioning Capacity • Employment / Role Performance • Symptom Distress • Living Arrangements

9. CASRA CSP PROJECT • Working independently in small groups, the participants all came to a similar conclusion: consumers could be assigned to clusters based on their level of risk, their level of coping skills, and their level of engagement with the mental health system. • The participants concluded that the movement of consumers from one group or cluster to another could itself be viewed as an outcome. • It also appeared to the participants that movement from one group to another could be reasonably seen as a description of “the process of recovery.”

10. Components of Recovery Level of Risk Level of Engagement Level of Skills and Supports Milestones of Recovery Extreme Risk High Risk/Unengaged High Risk/Engaged Poorly Coping/ Unengaged Poorly Coping/Engaged Coping/Rehabilitating Early Recovery Advanced Recovery Components and Milestones of Recovery

11. The Underlying Dimensions What do we mean by: • RISK • ENGAGEMENT • SKILLS AND SUPPORTS

12. RISK The consumer’s LEVEL OF RISK is comprised of three primary factors: 1) the consumer’s likelihood of causing physical harm to self or others, 2) the consumer’s level of participation in risky or unsafe behaviors, and 3) the consumer’s level of co-occurring disorders.

13. ENGAGEMENT • The consumer’s LEVEL OF ENGAGEMENT is the level of “connection” between the consumer and the mental health service system.

14. What Engagement ISN’T • Engagement DOES NOT mean medication compliance • Engagement DOES NOT mean “insight” into or “acceptance” of one’s mental illness • Engagement DOES NOT mean total agreement with the service and treatment approaches of staff • Engagement DOES NOT mean the total amount of services received

15. What Engagement IS • Engagement is about the quality of the relationship (“connectedness”) between consumer and staff and does not require the consumer to accept that s/he has a mental illness. It merely requires that the consumer is “tolerant” of the presence of staff in his/her life.

16. SKILLS AND SUPPORTS • The consumer’s LEVEL OF SKILLS AND SUPPORTS should be viewed as the combination of the member’s abilities and support network(s) and the level to which the consumer needs staff support to meet his/her needs.

17. “Prototypical” Relationships of the Underlying Dimensions by Milestone

18. Can the Milestones be used “Clinically?” • Milestones are designed as an “administrative” tool – to DESCRIBE the process, not to DIRECT the process • Difficulties in comparing the “objective” and the “subjective” experiences of recovery • The milestones are written in a way to take advantage of our experience with the “generalized consumer.” In reality, there is no such thing as a “general” consumer. Every consumer has her own experience and it is our relationship to the nuances and uniqueness of that particular experience that ultimately creates the ability to change (i.e., has what is usually called a clinical effect)

19. Stability of Ratings within the Village ISA January 2005 – March 2005

20. Reliability and Validity Data

21. Inter-Rater Reliability of MORS • October 2005 • 52 raters • All clients rated by 2-5 raters • All clients rated by psychiatrist, PSC (case manager), neighborhood leader and one other staff person who knew them well

22. Inter-rater Reliability (2) • Intra-class correlation coefficient • All analysis done in SAS 9.01.03 using PROC MIXED (Littell, et al., 2006) • r = .85 • .70 used as meeting acceptable reliability (Nunnally & Bernstein, 1994)

23. Inter-rater Reliability (Vinfen) • Data collected in April 2006 • Each client assigned a primary (case manager) and secondary rater (second best), blind to others’ rating

24. Inter-rater Reliability (Vinfen) • 105 clients rated at two points in time (one client missing second time point) • r = .86, 95%CI = .80, .90

25. Test-Retest Reliability • Conducted at two points in time in 2005—September 1 and September 21 (Range 10-20 days) • 381 clients rated at both points in time (431 at time 1 and 381 at time 2) • r = .846, 95% CI .81, .87

26. Validity • Level of Care Utilization System (LOCUS) used as validity measure (Sowers, George & Thompson, 1999). • .49 used as meeting acceptable validity

27. LOCUS • Underlying subscales include • Risk of harm • Functional status • Medical, Addictive, and Psychiatric Co-morbidity • Recovery Environment—Level of Stress • Recovery Environment—Level of Support • Treatment and Recovery History • Engagement

28. LOCUS • Six months of LOCUS data • Administered January through June 2005 • MORS also done every month • Pearson correlation coefficients

29. LOCUS Validity Coefficients

30. Village Members Residential Type by Milestone of Recovery March 2005

31. S.H.’s Milestones of Recovery DataJanuary 2005 – October 2005

32. Future Directions and Questions • Are different services more or less effective at different milestones of recovery? Can we use the milestones to assign consumers to different types of care? • What is the “typical” path of a person in recovery? If a person enters our system as “high risk, unengaged,” how long (on average) will it take for her to become “coping/rehabilitating?” How long until she is in “advanced recovery.” • Is it possible to establish “benchmarks” for service providers? Can we hold service providers accountable for moving people through the milestones? Should we set expectations for service providers to move certain percentages of their consumers to higher milestones over a set amount of time?