1 / 63

Using Routine Data on Needs and Outcomes to Improve Clinical Practice

Using Routine Data on Needs and Outcomes to Improve Clinical Practice. Michael Dennis, Ph.D. Chestnut Health Systems, Normal, IL

marinel
Télécharger la présentation

Using Routine Data on Needs and Outcomes to Improve Clinical Practice

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Routine Data on Needs and Outcomes to Improve Clinical Practice Michael Dennis, Ph.D. Chestnut Health Systems, Normal, IL Presentation at “Serving EAP Clients: The Roles of Mental Health Practitioners in Managing Workplace Mental Health.”, Weaver Ridge, IL, October 1, 2010. Baltimore, MD, August 24-26, 2010.. This presentation reports on treatment & research funded by SAMHSA contract 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at www.chestnut.org/LI/Posters or by contacting Joan Unsicker at 448 Wylie Drive, Normal, IL 61761, phone: (309) 451-7801, Fax: (309) 451-7763, e-mail: junsicker@Chestnut.Org

  2. Examine the strengths and weakness of common performance measures Explore epidemiological and research data on what we should expect Illustrate how to use routinely collected data to improve the identification of client needs, target services, and improve outcomes in private and agency practices Goals of this Presentation are to

  3. While I will draw many examples from substance abuse treatment & recovery research (my field), they easily generalize to mental health While I will use data from the Global Appraisal of Individual Needs (GAIN) (Chestnut’s instrument) the points are generic and apply to other measures as well. Two Key Qualifiers..

  4. Examples of Common Record Based Performance Measures * NQF: National Quality Forum; WCG: Washington Circle Group; CSAT: Center for Substance Abuse Treatment evaluations; NOMS: National Outcome Monitoring System; NIATX: Network for the Improvement of Addiction Treatment; PFP: Pay for Performance evaluations

  5. Evaluation of these Existing Measures Strengths: Easy to collect/ calculate in electronic health records Give broad overview of where problems Useful for program evaluation and pay for performance Weaknesses: Doesn’t lead to specific changes or intervention at the individual level Doesn’t address comorbidity or case mix Doesn’t easily lead to specific improvement at the program level Doesn’t address relationships with other gaps in the macro system

  6. Examples of Additional Standards of Care Being Considered by NQF Annual screening for tobacco, alcohol and other drugs using systematic methods Referral for further multidimensional assessment to guide patient-centered treatment planning Brief intervention, referral to treatment and supportive services where needed Pharmacotherapy to help manage withdrawal, tobacco, alcohol and opioid dependence Provision of empirically validated psychosocial interventions Monitoring and the provision of continuing care Source: www.tresearch.org/centers/nqf_docs/NQF_Crosswalk.pdf

  7. With electronic health records we can also focus on more substantive measures 218/224=97% to targeted 553/771=72% unmet need 771/982=79% in need Size of the Problem Extent to which services are not reaching those in most need Extent to which services are currently being targeted Source: 2008 CSAT AAFT Summary Analytic Dataset

  8. Mental Health Problem (at intake) vs. Any MH Treatment by 3 months Source: 2008 CSAT AAFT Summary Analytic Dataset

  9. Why Do We Care About Unmet Need? If we subset to those in need, getting mental health services predicts reduced mental health problems Both psychosocial and medication interventions are associated with reduced problems If we subset to those NOT in need, getting mental health services does NOT predict change in mental health problems Conversely, we also care about services being poorly targeted to those in need.

  10. Residential Treatment need (at intake) vs. 7+ Residential days at 3 months Opportunity to redirect existing funds through better targeting Source: 2008 CSAT AAFT Summary Analytic Dataset

  11. Prevalence of Lifetime Disorders and Past Year Remission in the US Household Population INT SUD EXT 100% 90% Lifetime Disorder 80% Past Year Remission 70% 47% 60% 37% 50% 31% 40% 25% 20% 19% 30% 15% 13% 13% 12% 10% 10% 8% 8% 8% 8% 20% 7% 7% 5% 4% 2% 2% 10% 0% ADHD Dysthymia Agoraphobia Any Disorder Drug Disorder Social Phobia Bi-Polar I or II Panic Disorder Alcohol Disorder Conduct Disorder Oppositional Defiant Any Mood Disorder: Intermittent Explosive Internalizing Disorder Other Specific Phobia Major Depressive Epi. Externalizing Disorder Any Anxiety Disorder: Any Substance Disorder Generalized Anxiety Dis. Posttraumatic Stress Dis. Adult Separation Anxiety Source: Dennis, Scott, Funk & Chan forthcoming; National Co morbidity Study Replication

  12. Data can help give our clients “HOPE”Recovery “Rates” (Remission/Lifetime) INT SUD EXT 89% 89% 100% Past Year Recovery Rate 83% 90% 77% 71% 66% 80% 57% 58% 56% 70% 50% 45% 48% 48% 43% 44% 41% 42% 60% 44% 41% 39% 30% 50% 31% 40% 30% 20% 10% 0% ADHD Dysthymia Agoraphobia Any Disorder Drug Disorder Social Phobia Bi-Polar I or II Panic Disorder Alcohol Disorder Conduct Disorder Oppositional Defiant Any Mood Disorder: Intermittent Explosive Internalizing Disorder Other Specific Phobia Major Depressive Epi. Externalizing Disorder Any Anxiety Disorder: Any Substance Disorder Generalized Anxiety Dis. Posttraumatic Stress Dis. Adult Separation Anxiety Source: Dennis, Scott, Funk & Chanforthcoming; National Co morbidity Study Replication

  13. Data Teaches us that Comorbidity is the NORM (28%/46% Any)= 61% Co-occurring Lifetime Pattern of Disorders Lifetime Number of Disorders (13%/16% SUD)= 81% Co-occurring (19%/24% Ext)= 79% Co-occurring (22%/43% Int.)= 51% Co-occurring Source: Dennis, Scott, Funk & Chanforthcoming; National Co morbidity Study Replication

  14. Comorbidity is also related who enters treatment.. Pattern of Disorders Number of Disorders Source: Dennis, Scott, Funk & Chan forthcoming; National Co morbidity Study Replication

  15. 100% 90% 80% 68% Past Year 65% 64% 70% Recovery Rate 51% 50% 60% 41% 50% 40% 26% 24% 19% 30% 16% 20% 10% 0% None None 1 Disorder 2 Disorders Substance Only 3 to 16 Disorders Internalizing Only Sub. + Ext. + Int. Externalizing Only Substance+Internalizing Substance+Externalizing Externalizing+Internalizing ..And the likelihood of Recovery Pattern of Disorders Number of Disorders Source: Dennis, Scott, Funk & Chan forthcoming; National Co morbidity Study Replication

  16. Patterns of Comorbidity change with Age Internalizing Disorders go up with age Externalizing Disorders go down with age (but do NOT go away) Source: Chan, YF; Dennis, M L.; Funk, RR. (2008). Prevalence and comorbidity of major internalizing and externalizing problems among adolescents and adults presenting to substance abuse treatment. Journal of Substance Abuse Treatment, 34(1) 14-24 .

  17. Substance Use & Disorders Also Vary by Age Over 90% of use and problems start between the ages of 12-20 It takes decades before most recover or die NSDUH Age Groups 100 People with drug dependence die an average of 22.5 years sooner than those without a diagnosis 90 80 70 60 Severity Category 50 No Alcohol or Drug Use Light Alcohol Use Only 40 Any Infrequent Drug Use 30 Regular AOD Use 20 Abuse 10 Dependence 0 65+ 12-13 14-15 16-17 18-20 21-29 30-34 35-49 50-64 Source: 2002 NSDUH and Dennis & Scott 2007

  18. Higher Severity is Associated with Higher Annual Cost to Society Per Person Mean (95% CI) $3,058 This includes people who are in recovery, elderly, or do not use because of health problems Higher Costs $1,613 $1,528 $1,309 $1,078 $948 $4,000 Median (50th percentile) $3,500 $3,000 $2,500 $2,000 $1,500 $1,000 $725 $406 $500 $231 $231 $0 $0 $0 No Alcohol or Light Alcohol Regular AOD Any Dependence Abuse Infrequent Drug Use Use Only Drug Use Use Source: 2002 NSDUH

  19. pain Adolescent Brain Development Occurs from the Inside to Out and from Back to Front Photo courtesy of the NIDA Web site. From A Slide Teaching Packet: The Brain and the Actions of Cocaine, Opiates, and Marijuana.

  20. There 41.4 Million Under Age or Problem Drinkers in the U.S. 17.6 Million under age drinkers (46% of 38.1 Mil) 28.4 Million (12%) Problem Drinkers (4.6m/12% of youth, 23.8m/11% of adult) Source: SAMHSA 2006. National Survey On Drug Use And Health, 2006 [Computer file]

  21. Potential Screening/ Intervention Sites: Age 12 to 20 (38.1 million) Key potential of Workplace (e.g., EAP, Wellness,HRA) and School (e.g., SAP, EI, Prevention) Programs NOTE: Not asked about work if under age 15 in NSDUH Source: SAMHSA 2006. National Survey On Drug Use And Health, 2006 [Computer file]

  22. Potential Screening/ Intervention Sites: Age 21+ (207.9 million) Key potential of Workplace Programs NOTE: Not asked about School if over age 18 in NSDUH Source: SAMHSA 2006. National Survey On Drug Use And Health, 2006 [Computer file]

  23. How does data related to the move towards Evidence Based Practice (EBP)? EBP means introducing explicit intervention protocols Targeted at specific problems/subgroups and outcomes Having explicit quality assurance procedures to cause adherence at the individual level and implementation at the program level Reliable and valid assessment is needed that can be used to Immediately guide clinical judgments about diagnosis/severity, placement, treatment planning, and the response to treatment at the individual level Drive longer term program evaluation, needs assessment, performance monitoring and program planning Allow evaluation of the same person or program over time Allow comparisons with other people or interventions

  24. Major Predictors of Bigger Effects Found in Multiple Meta Analyses Triage to focus on the highest severity subgroup A strong intervention protocol based on prior evidence Quality assurance to ensure protocol adherence and project implementation Proactive case supervision of individual

  25. Impact of the numbers of these Favorable features on Recidivism in 509 Juvenile Justice Studies in Lipsey Meta Analysis The more features, the lower the recidivism Average Practice Source: Adapted from Lipsey, 1997, 2005

  26. No problems (0-25%ile) 1-3 problems (25-50%ile) 4-8 problems (50-75%ile) 9+ problems (75-100%ile) Impact of Intake Severity on Outcome 10 SPSM groupings OVERALL 8 6 Substance Problem Scale (0-16 Past Month Symptoms) 4 Dot/Lines show Means 2 Intake Severity Correlated -.66 with amount of change 0 0 6 Wave • Programs with low severity look better with absolute outcomes (e.g. abstinence) • Programs with high severity look better with amount of change Source: ATM Main Findings data set

  27. Example of Generic vs. Targeted Effects 0.27 0.20 0.15 0.10 0.01 0.00 0.00 -0.04 -0.08 -0.29 -0.39 -0.69 Targeted Generic 0.40 0.20 0.00 -0.02 -0.03 -0.10 Cohen's Effect Size d -0.20 -0.40 Unprotected Sex Acts (f=.14) Days of Victimization (f=.22) -0.60 Days of Needle Use (f=1.19) -0.80 A. B. C. D. Total Low Risk W/O Trauma Mod. Risk W/O Trauma Mod. Risk With Trauma High Risk With Trauma Source: Lloyd et al 2007

  28. Evidenced Based Treatment (EBT) that Typically do Better than Usual Practice in Reducing Juvenile Recidivism (29% vs. 40%) Aggression Replacement Training Reasoning & Rehabilitation Moral Reconation Therapy Thinking for a Change Interpersonal Social Problem Solving MET/CBT combinations and Other manualized CBT Multisystemic Therapy (MST) Functional Family Therapy (FFT) Multidimensional Family Therapy (MDFT) Adolescent Community Reinforcement Approach (ACRA) Assertive Continuing Care NOTE: There is generally little or no differences in mean effect size between these brand names Source: Adapted from Lipsey et al 2001, Waldron et al, 2001, Dennis et al, 2004

  29. Implementation is Essential (Reduction in Recidivism from .50 Control Group Rate) The best is to have a strong program implemented well The effect of a well implemented weak program is as big as a strong program implemented poorly Thus one should optimally pick the strongest intervention that one can implement well Source: Adapted from Lipsey, 1997, 2005

  30. Percentage Change in Abstinence (6 mo-Intake) by level of Adolescent Community Reinforcement Approach (A-CRA) Quality Assurance Effects associated with intensity of quality assurance and monitoring (OR=13.5) Source: CSAT 2008 SA Dataset subset to 6 Month Follow up (n=1,961) 30

  31. Illustration of the Need for Proactive case Supervision of Individual: Prevalence of 12 problems Source: CSAT 2009 Summary Analytic Data Set (n=20,826) 31

  32. The Number of Major Clinical Problems by Level of Care Significantly more likely to have 5+ problems (OR=5.8) Source: CSAT 2009 Summary Analytic Data Set (n=21,332) 32

  33. The Number of Major Clinical Problemsis highly related to Victimization But this is the issue staff least like to ask about! Significantly more likely to have 5+ problems (OR=13.9) Source: CSAT 2009 Summary Analytic Data Set (n=21,784) 33

  34. Overcoming Person or Staff Reluctance with the GAIN General Victimization Scale Source: CSAT 2009 Summary Analytic Data Set (n=19,318) 34

  35. The GAIN is .. A family of instruments ranging from screening, to quick assessment to a full Biopsychosocial and monitoring tools Designed to integrate clinical and research assessment Designed to support clinical decision making at the individual client level Designed to support evaluation and planning at program level Designed to support secondary analyses and comparisons across individuals and programs The GAIN is NOT an electronic health record (EHR), but a component that can interface with and support EHRs.

  36. Global Appraisal of Individual Needs (GAIN) Network of Collaborators NH WA VT ME MT MN ND MA OR WI ID SD NY RI MI WY CT PA IA NV NE NJ OH UT IL IN CA DE CO WV MO VA MD KY KS DC NC TN OK NM State or Regional System GAIN-Short Screener GAIN-Quick GAIN-Full No of GAIN Sites AR AZ SC GA AL None (Yet) MS 1 to 14 TX LA 15 to 30 AK 31 to 165 FL HI More in BZ, CA, CN, JP, MX VI PR 3/10 36

  37. Some numbers as of June 2010 1,501 Licensed GAIN administrative units from 49 states (all by ND) and 7 countries 3,270 users in 396 Agencies using GAIN ABS 60,380 intake assessments (largest in field) 22,045 (88% w 1+ follow-up) from 278 CSAT grantees 22 states, 12 Federal, 6 Canadian provinces, 6 other countries, and 3 foundations mandate or strongly encourage its use 4 dozen researchers have published 179 GAIN-related research publications to date 37

  38. Crosses a Continuum of Measurement (Common Measures) Screening to Identify Who Needs to be “Assessed” (5-10 min) Focus on brevity, simplicity for administration & scoring Needs to be adequate for triage and referral GAIN Short Screener for SUD, MH & Crime ASSIST, AUDIT, CAGE, CRAFT, DAST, MAST for SUD SCL, HSCL, BSI, CANS for Mental Health LSI, MAYSI, YLS for Crime Quick Assessment for Targeted Referral (20-30 min) Assessment of who needs a feedback, brief intervention or referral for more specialized assessment or treatment Needs to be adequate for brief intervention GAIN Quick ADI, ASI, SASSI, T-ASI, MINI Comprehensive Biopsychosocial (1-2 hours) Used to identify common problems and how they are interrelated Needs to be adequate for diagnosis, treatment planning and placement of common problems GAIN Initial (Clinical Core and Full) CASI, A-CASI, MATE Specialized Assessment (additional time per area) Additional assessment by a specialist (e.g., psychiatrist, MD, nurse, spec ed) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan CIDI, DISC, KSADS, PDI, SCAN More Extensive / Longer/ Expensive Screener Quick Comprehensive Special

  39. Longer assessments identify more areas to address in treatment planning Most substance users have multiple problems 5 min. 20 min 30 min 1-2 hr Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192) 39

  40. Expected Factor Structure of Psychopathology and Psychopathy Source: Dennis, Chan, and Funk (2006) 40

  41. GAIN Short Screener (GAIN-SS) Administration Time: A 5-minute screener Purpose: Used in general populations to identify or rule out clients who will be identified as having any behavioral health disorders on the 60-120 min versions of the GAIN triage area of problem serve as a simple measure of change ease administration and interpretation by staff with minimal training or direct supervision Mode: Designed for self- or staff administration, with paper and pen, computer, or on the web Languages: English, Spanish, French, Portuguese, Simple & Traditional Chinese & 15 other languages Scales: Four screeners for Internalizing Disorders, Externalizing Disorders, Substance Disorders, and Crime/Violence Disorders, and a Total Disorder Screener

  42. Response Set: Recency of 20 problems rated past month (3), 2-12 months ago (2), more than a year ago (1), never (0) Interpretation: Combined by cumulative time period as: Past-month count (3s) to measure change Past-year count (2s or 3s) to predict diagnosis Lifetime count (1s, 2s, or 3s) as a measure of peak severity Can be classified within time period as low (0), moderate (1-2), or high (3) Can also be used to classify remission as Early (lifetime but not past month) Sustained (lifetime but not past year) Reports: Narrative, tabular, and graphical reports built into web- based GAIN ABS or ASP application for local hosting GAIN Short Screener (GAIN-SS) (continued) Source: Dennis, Chan, and Funk (2006) www.chestnut.org/LI/gain/GAIN_SS

  43. Screener items were selected using the Rasch Measurement Model Items around key decision point Source: Riley et al 2007 -1.89 -.8 -.32 +.28 +.71 43

  44. Why do we use a cut point of 1 on the Substance Disorder Screener? A cut point of 1 has 96% sensitivity and 73% specificity (i.e., it gets most real cases but has some false cases) A cut point of 1 has 68% sensitivity and 100% specificity (i.e., it misses almost a third of real cases but has virtually no false cases Best Recommendation: 1+ on SDScr and 3+ on TDScr Source: Dennis et al 2006 44

  45. Construct Validity of GSS Internalizing Disorder Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002) 45

  46. Construct Validity of GSS Externalizing Disorder Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002) 46

  47. Construct Validity of GSS Substance Disorder Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002) 47

  48. Construct Validity of GSS Crime/Violence Screener Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002) 48

  49. Adolescent Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting in Washington State Problems could be easily identified Comorbidity is common Source: Lucenko et al. (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

  50. Adolescent Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records by Setting in Washington State Two page measure closely approximated all found in the clinical record after the next two years Source: Lucenko et al. (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

More Related