Implementing a Two-staged Screening Process in Addiction Agencies in Ontario: Trials and Triumphs When Going from Rese

1. Implementing a Two-staged Screening • Process in Addiction Agencies in • Ontario: Trials and Triumphs When • Going from Research to Practice • Brian Rush • Centre for Addiction and Mental Health • and • University of Toronto • Dept. of Psychiatry

2. Overview of “Science to Service” on CD Screening (2006 – present) • Consultation with treatment providers and stakeholders • International symposium of researchers, clinicians and policy makers • Validation and comparison of tools including the GAIN-SS adults (done) • Research synthesis – tools for children and adolescents – validation of GAIN-SS underway • Knowledge translation – back to the community with clinical protocols and field evaluation • Formal program of implementation research

4. Progressive Continuum of Measurement More Extensive / Longer / Expensive Screener Quick Comprehensive Special • Screening to identify who needs to be assessed (5-10 min) • Focus on brevity, simplicity for administration & scoring • Needs to be adequate for triage and referral • GAIN Short Screener for SUD, MH, and crime • Quick assessment for targeted referral (20-30 min) • Assessment of who needs a feedback, brief intervention, or referral for more specialized assessment or treatment • Needs to be adequate for brief intervention • Comprehensive biopsychosocial (1-2 hours) • Used to identify common problems and how they are interrelated • Needs to be adequate for diagnosis, treatment planning, and placement of common problems • Specialized assessment (additional time per area) • Additional assessment by a specialist (e.g., psychiatrist, MD, nurse, spec. ed.) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan

5. Research synthesis on screening tools for children and adolescents >>>>>

6. Overview of the Steps in the Search and Retrieval Process

7. The Tools…

8. STARD Ratings • STARD = Standards for Reporting of Diagnostic Accuracy • Assists investigators in reporting results in clear fashion • Provides an assessment of the quality of the reporting of essential features of all phases of a validation study • Does NOT yield an assessment of the research findings per se

9. Summary of Screening Options for Ages 12 - 17

10. Feedback from Treatment System Stakeholders • Tools that screen for both mental and substance use problems/disorders seen as most valued • Top selection criteria for end-users: • strong reliability and validity data • practical issues such as ease of administration, scoring and brevity. • cost is an important consideration but does not dominate among various selection criteria. • Good support for the staged screening model – role for very brief tools as well as longer, more comprehensive screeners

11. Staged Model of Screening and Assessment Goal Brief Description Stage 1: Screening –Tentative identification of generalized caseness - Case Finding Brief screen for possibility of any substance use or mental disorder GAIN Short Screener Stage 2: Screening—Tentative identification of disorder-specific caseness - Case Defining Longer screen for specific substance use or mental disorders DISC Predictive Scales, or POSIT Stage 3: Assessment—Confirmation of specific disorders Diagnostic assessment and treatment planning (e.g., Schedule for Affective Disorders and Schizophrenia for School Aged Children – Present and Lifetime Version (K-SADS); full GAIN assessment.

12. Validation and comparison of four screening tools in Adult SUD treatment population • 545 study participants (adults) administered four tools • GAIN-SS (IDSsr) • Kessler’s K6 • ASI-Psychiatric sub-scale • Psychiatric Diagnostic Screening Questionnaire (PDSQ) • Gold standard SCID interview

13. Results • For broad classes of psychiatric disorders such as “any disorder”, “any internalizing disorders” or “any mood disorders” all performed with about the same accuracy (e.g, AUC - .77 to .82 for “any disorder”) • GAIN-SS-IDSsr most efficient brief screener given its length (5 items) (AUC = .77 (95% CI= .73 to .81) for any internalizing disorder) • For specific diagnoses (e.g., Major Depressive Disorder, Panic Disorder, PTSD; Psychotic Disorder) the PDSQ performed much better • PTSD: The AUC = .83 for the PDSQ compared to .72 for the GAIN-SS-IDSsr • OCD: The AUC = .80 for the PDSQ compared to .66 for the GAIN-SS-IDSsr

14. Client Engagement Phase Developmental Phase Children Adolescents Adults Older Adults Screening Stage 1: Case finding Mental health, substance use, problem gambling, and other co-occurring conditions Stage 2: Case defining Assessment Outcome Monitoring

15. Knowledge translation – back to community settings with two-stage screening protocol • Developed clinical protocol for two-staged screening process – GAIN-SS and then the PDSQ • Three study sites • Mental health and addiction program in remote area of northern Ontario – “where the road ends” (no screening tools used at all) • Outpatient assessment and treatment program in small city/rural Ontario (using the Modified Mini screening tool) • Rainbow (lesbian-gay) program at large multi-functional Centre in downtown Toronto (were using both tools but not in a staged way) • $25,000 budget !

16. Objectives: • Evaluate utility of the staged model • Explore impact of implementation on agency practice • Identify organizational factors influencing adoption

17. Implementation Approach • Plain language version of the research report prepared • Clinical protocol developed and one-day on-site training provided in administration, scoring and interpretation of the two tools • Web-based community of practice developed for agency staff and project team and Advisory Committee to discuss implementation issues • Proactive phone check-ins and availability of team for support

18. Data Collection • Individual or group interviews with program managers/ senior staff (five) • On-line survey of staff (11 of 15 participated) • Feedback from COP, including Advisory Committee (18) • Question areas: • administration strategy, client response, perceived usefulness, plans to continue use of change practice, suggested enhancements • “levels of use” scale • Readiness for adoption(perceived advantages outweigh disadvantages) • Trialability(e.g., fits with current process, within scope of resources) • Observability(can see the benefits, impact on service quality) • Persuasion(could explain benefits to others)

19. Administration depended on agency mandate and existing processes GAIN-SS-IDScr • two sites self-selected to do more than the 5-item IDScr (one used all 20 items; one the IDScr and the substance abuse scale) • : one site administered at intake over the phone, the others self-administered during the first face-to-face meeting for assessment (one by computer, the other paper and pencil) PDSQ • One site administered it within routine assessment (between appointments 2-4); another felt they needed a separate appointment just for the PDSQ; another did both tools at the first interview • All had clients self-administer the tool with staff present for assistance if needed

20. Reported client response GAIN-SS-IDScr • Generally favourable at all sites • Some literacy issues • At one site a few clients objected to the crime/violence questions in that sub-scale PDSQ • Length was a common concern, some parts seen as repetitive • But seen as comprehensive and professional • Some literacy challenges • Tendency for clients to want to talk about their responses and the issues raised for them

21. Cut-offs and completion rates GAIN-SS-IDScr • Total of 191 clients were screened in study period. • Very high percentage of clients exceeded the recommended cut-off of one positive item of the 5-item sub-scale (92.5%,91.5%, and 85.7%) • One site (MH and A) felt it confirmed what they were getting before the project from their routine interview but that it didn’t add anything new; the other two sites were more positive despite the high percentage flagged

22. Cut-offs and completion rates PDSQ • Completion rates varied: In the site with both tools done the same day at assessment, almost all completed the PDSQ if required by the protocol • In the site with assessment spread over 4 sessions 62% of those scoring positive on the GAIN-SS were administered the PDSQ • In the site bringing the person in for a special mental health appointment for the PDSQ 52% declined the offer and another 16% did not show (so only 32% got the PSDQ as required) • Reasons for not administering the PDSQ included: already connected to mental health service, believed they did not need help with mental health issues • One site the results seen as confirmatory but not adding new; at the others seen as complementary to clinical interview and relationship building

23. Levels of use of the two-stage protocol • Results from staff depended on the site and confirmed what we were getting from the managers and senior staff • Readiness: “advantages over other tools and strategies” • 36% agree; 36% strongly agree • Trialability: “fits with organizational policies and processes” • 27% agree; 55% strongly agree • Observability: “will enhance the quality of our assessment.. ” • 27% agree; 27% strongly agree • Persuasion: “will have no difficulty explaining the benefits” • 55% agree; 27% strongly agree

24. Plans to continue with the two-staged process • The integrated mental health and addictions site : no plans to continue - just not seen as adding much • In the site using interview approach and a previous MH screener, the 2-stage protocol would replace that process when/if they have support from community partners accepting MH referrals • In the site using the two tools but not in a staged way they planned to continue with the staged process

25. Lessons learned from a low-budget KT pilot • Must work to fit screening tools and protocol into the agency flow and dynamics – start where they are. Must be flexible. • Integrated MH and addiction agencies have different experiences and needs for MH screening that “pure” addiction services • Two-staged protocol holds some promise but a lot more needs to be done before larger (provincial) application (e.g., the cut-points on the GAIN-SS in different settings; advantages)

26. What lies ahead? • Provincial and national dissemination of the GAIN-SS (it has legs as they say) and in both adult and youth services/systems • Longitudinal study of youth underway that will provide more validation data for children and adolescents in our populations • Need a stronger more thoughtful implementation science approach with more supports for uptake and sustainability