1 / 34

Getting To Outcomes Presentation Outline

Getting To Outcomes Demonstration: Building Capacity for High Quality Prevention Matt Chinman and Sarah Hunter, RAND Shereen Khatapoush, CADA AHSR 2005 October 25, 2005. Getting To Outcomes Presentation Outline. Overview Getting To Outcomes Demonstration Getting To Outcomes Evaluation

lana-mccall
Télécharger la présentation

Getting To Outcomes Presentation Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Getting To Outcomes Demonstration: Building Capacity for High Quality PreventionMatt Chinman and Sarah Hunter, RANDShereen Khatapoush, CADAAHSR 2005October 25, 2005

  2. Getting To Outcomes Presentation Outline • Overview • Getting To Outcomes Demonstration • Getting To Outcomes Evaluation • Conclusions and next steps

  3. Community Based Programs Face Challenges • AOD prevention programs can help communities but face many challenges: • “Off the shelf” programs can be difficult to implement well • Accountability requirements are increasing • With less funding available, programs need to make strategic choices • Often do not demonstrate expected positive outcomes (gap between science and practice)

  4. What Is Getting To Outcomes-2004? • A system based on ten empowerment evaluation and accountability questions • Helps community practitioners plan, implement, and evaluate their own programs and get results • Published by the RAND Corporation in 2004 (quality review) • Available at no cost (and soon in Spanish) at: http://www.rand.org/publications/TR/TR101/ • “Best Practice Process” - CSAP

  5. Getting To Outcomes Presentation Outline • Overview • Getting To Outcomes Demonstration • Getting To Outcomes Evaluation • Conclusions and next steps

  6. Getting To Outcome Demonstration: Intervention Components • Manual • Training • Technical Assistance (TA)

  7. GTO-04 Manual • Text information

  8. GTO-04 Manual • Tools

  9. GTO-04 Manual • WINNERS Example

  10. GTO-04 Manual • Up to date model program descriptions

  11. GTO-04 Training • Annual full day training, first two years • Train staff on the 10 accountability questions • Provide examples of how each question can be successfully addressed with tools from manual • Interactive and engaging: • Discussions of staff’s prevention experience • Exercises for practicing the 10 questions using both hypothetical and real scenarios

  12. GTO Technical Assistance • Conduct TA needs assessment and planning tools to decide course of TA • Establish monthly “GTO Report” and provide support for its completion • Weekly TA meeting with key staff • Quarterly group meetings with Coalition leadership • Develop data collection tools and tracking system • Develop outcome evaluation materials

  13. Getting To Outcomes Demonstration: Project Organization

  14. Treatment • Cannabis Youth Treatment • Daniel Bryant Youth/Family Center • Teen Court • Youth Services System • Daniel Bryant Youth/Family Center - Moderate • Daniel Bryant Youth/Family Center - Intensive • Cannabis Youth Treatment • SUPER • Teen Court • Truancy • Youth Services System Maintenance Prevention • Friday Night Live • Mediation • Mentoring • Parent Project • Youth Services System Standard Treatment • Daniel Bryant Youth/Family Center Case Identification Indicated – Designed for people already experimenting with drugs Compliance with Long-term Treatment (Goal: Reduction in Relapse and Recurrence Selective – Designed for groups at risk or subsets of the general population, such as poor school achievers • Friday Night Live • Life Skills • Media/Public Awareness • Youth Services System • Daniel Bryant Youth/Family Center Universal – Designed for the general population, such as all students in a school After-Care (Including Rehabilitation) CADA Adolescent Programs Continuum of Care* *Adapted from:Institute of Medicine, Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research, National Academies Press, 1994

  15. A private, non-profit agency • Also one of 33 county alcohol and drug abuse authorities recognized by the state of South Carolina • Offers a wide array of prevention, intervention and treatment programs • Guiding Good Choices Parenting Program • Social Norms Media Campaign

  16. Monthly GTO Report Created to Communicate Program Update

  17. YSS Program Adapted GTO Tool to Fit Needs

  18. Getting To Outcomes Presentation Outline • Overview • Getting To Outcomes Demonstration • Getting To Outcomes Evaluation • Conclusions and Next Steps

  19. Design of GTO Evaluation • Collaborative effort between RAND, CADA and L/RADAC • Includes comparison programs within the coalitions not receiving the GTO manual, training and TA • Two Approaches • Process: How and to what extent is GTO used? • Outcome: What effect does GTO have on prevention capacity?

  20. Conceptual Approach Demonstration Existing prevention capacity • GTO • Utilization • Training • T.A. • Manual Improved prevention capacity

  21. Measures Used in Evaluation Follow Conceptual Approach Demonstration Existing prevention capacity -Individual level: Coalition survey (Baseline) -Program level: IC Maps (Baseline) • GTO • Utilization • Training • T.A. • Manual • Process evaluation • TA notes • Focus groups (Yr1) • Interviews (Yr2) Improved prevention capacity Outcomes evaluation Individual level: Coalition survey (Years 1, 2) Program level: IC Maps: (Years 1, 2) Evaluation

  22. The Amount and Type of Technical Assistance Provided was Documented

  23. Improvements in Prevention Capacity Assessed on Two Levels for the Outcomes Evaluation Coalition survey measures the frequency of prevention practices targeted by GTO’s 10 steps at the individual level IC Maps measures the quality of prevention practices targeted by GTO’s 10 steps at the program level

  24. Innovation Configuration (IC) Maps Measured Changes in Prevention Capacity • “IC Maps” are a framework that can be tailored to evaluate the quality of use of any innovation (Hall and Hord, 2001) • 7–point scales from “highly faithful” to “highly divergent” on 14 GTO skills (10 steps +) • Ratings based on interviews with program staff by TA and site liaison at each site, supplemented by knowledge of programs • Found good inter-rater reliability conducted across 3 time periods

  25. IC Map Ratings Indicate Greater Improvement in Prevention Capacity in GTO Programs Consistent integration of GTO step with other steps Basic minimum of GTO step implemented as designed Opposition to principles and concepts GTO Programs Comparison Programs

  26. TA hours related to improvement in accountability practices from BL toYr2 Correlation (r)=.55, p=.09, n=10

  27. Focus Groups Results:Benefits of GTO Overall • Teach a new language • Provide a new mindset (more “proactive”, “focused”, “orderly”) • Provide additional legitimacy to community and funders Planning • Improve communication with grant writer • Better understand limits of own capacity • Help to structure priorities • Help to develop realistic goals and objectives • Assess whether new potential programs are evidence- based Evaluation • Collect better data • Help add new evaluation measures

  28. Focus Groups Results: Challenges of GTO • Coalition Issues • Lack of time • Lack of resources • Overcoming the usual way things are done • GTO Issues • Some language was hard to understand • Length of manual • 10 steps is a lot (“GTO for dummies?”) • Additional training and ongoing TA are needed

  29. Getting To Outcomes Presentation Outline • Overview • Getting To Outcomes Demonstration • Getting To Outcomes Evaluation • Conclusions and Next Steps

  30. Getting To Outcomes Evaluation: Conclusions • GTO can improve practitioner capacity to perform tasks associated with high quality prevention (planning, evaluation, etc.) • Those with greater exposure to GTO demonstrated more gains in capacity • GTO can be difficult to absorb; ongoing TA is critical to the success of GTO

  31. Getting To Outcomes Next Steps: Research • Continue Data Analysis • interviews, 3rd wave of coalition survey • Research Questions: • If GTO improves capacity, does that lead to improved program outcomes? • GTO demonstration involved significant dose of TA; what is the minimum amount needed for impact? • What is the impact of organizational capacity (e.g. leadership, communication, decision making, etc) on how much of an effect GTO can have?

  32. Getting To Outcomes Next Steps: Projects • New grant from CDC. Assess GTO’s impact on the quality of violence prevention programs in schools • iGTO. Combines the interactive web-based system of PBPS (Performance Based Prevention System-by Kit Solutions) with the logic of GTO model • Assets-GTO. Combines the youth development perspective with the planning, implementation, and evaluation guidance of GTO • Preventing Underage Drinking: Using the SAMHSA Strategic Prevention Framework and Getting To Outcomes.Published by CSAP, will provide guidance for such strategies as sobriety checkpoints, keg registrations, passing new zoning laws, and other environmental prevention strategies using the GTO framework • GTO-Espanola. Worked with Adela de la Torre, Ph.D., Professor & Director Chicana/o Studies Program at University of California, Davis to translate RAND-published manual • DELTA. CDC-funded multi-site demonstration on domestic violence

More Related