1 / 44

Outline

Implementation Research for Children and Families in a State Policy Context Kimberly Eaton Hoagwood Columbia University December 6, 2010. Outline. Definitions State context characteristics Theories Fitting effective practices to the state context: Which context?

courtney-bryan
Télécharger la présentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Implementation Research for Children and Families in a State Policy Context Kimberly Eaton Hoagwood Columbia University December 6, 2010

  2. Outline Definitions State context characteristics Theories Fitting effective practices to the state context: Which context? State D-I research examples Concluding thoughts

  3. Definitions of D-I Constructs(McHugh & Barlow, 2010) • ADOPTION: The decision by a provider or system to learn and implement an intervention. • IMPLEMENTATION—The process of transferring an intervention to a setting; it includes training approaches, consultation, and decision supports. The process of introducing or changing practice in a specific local setting. • DISSEMINATION—The effort to facilitate initial adoption through targeted distribution of a well-defined set of information • Focuses on how information is created, packaged, transmitted, and interpreted among various stakeholder groups. • DIFFUSION--The intended or unintended spread of information or treatments

  4. Effectiveness vs. Implementation • How well an intervention works, and how well it is implemented, are two different things (Schoenwald, 2006) • Effective treatments can be implemented poorly; and, ineffective treatments can be implemented well. (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005)

  5. State Initiatives to Improve the Quality of Services for Children and Families • 20+ State Consortium—EBP-CA (Bruns et al, 2008): CA, CO, CT, DE, HA, MD, MI, MN, NM, NY, NJ, OH, OK, PA, WA, Canada • Most common therapies • Multi-systemic therapies (MST) • Functional family therapy (FFT) • Treatment foster care (TFC) • Cognitive-Behavioral Therapies for Trauma • CBT for depression • Range of behavior management programs

  6. State Mental Health Budgets: Overall Picture (NASMHPD 2009) • According to the Center on Budget and Policy Priorities • (June 29, 2008): • Cumulative state budget gaps are expected to exceed $350 billion for the period of FY2009 – FY2011. • 48 States are facing shortfalls in FY2010 totaling $166 billion. • At least 29 states expect total gaps of $38 billion alone in 2011. • The current recession has a greater impact on state budgets than the previous recession due to high unemployment and lack of revenue from property and income taxes. • Thus an interest in improving quality and containing costs

  7. # of States with Anticipated Deficits (FY 09-11) NASMHPD Research Institute, Inc. 2009 2010 2011 No deficit • As of 12/08 32 32 13 10 • As of 6/09 32 40 22 5

  8. Some Special Challenges • Policies to promote uptake of Evidence-Based Practices move faster than the knowledge base: creates jagged interface • Pressures for egalitarianism may preclude experimental designs • Accountability standards and billable time in conflict with need for ongoing consultation • Outcomes of interest usually functional and pragmatic: feasible measures for assessing change hard to find • Clinical service workforce is largely unprepared: Retraining has to be intensive • Technological supports for clinical decision-making and tracking outcomes are absent (few Electronic Medical Records) • Fundamental difference in epistemologies between science and policy. Need for people who can translate between the 2 worlds

  9. Science Cumulative, builds on past knowledge Generates more questions than answers Proceeds slowly, incrementally Self-correcting Scientific community—not locally bound although may be discipline-bound and specialized Policies Driven by immediate political needs Can move quickly, errantly, erratically Pragmatic, action-oriented 1 decision can affect millions of lives for better or worse Locally driven (counties, states, countries) Different epistemologies

  10. Three Relevant Theories for State Implementation: Can they be integrated? • Unified Theory of Behavior Change (Jaccard et al, 2002) drawn from basic behavioral sciences to understand the triggers for behavior change within individuals. Incorporates principles from the Theory of Reasoned Action (Fishbein & Ajzen, 1975; Ajzen & Fishbein, 1981) and Theory of Planned Behavior (Ajzen & Madden, 1986; Ajzen, 1991), and Self-Efficacy (Bandura) • Organizational social context theory (Glisson 2002) to understand the organizational and team processes (culture, climate, structure) that affect agency adoption of new practices. • Participatory action research based on Habermas’ (1990) theory of communicative action, to support shared decision-making among diverse constituents. • Our working model in New York State integrates these theories with the goal of improving the practical utility of the findings.

  11. System & Policy Context of the State methods of reimbursement, fiscal incentives, linkages to other healthcare systems, certification, accreditation, licensing standards Organizational Social Context of Agencies Culture, Climate, Structure Family Engagement Clinical Care ImprovementTraining on EBP’s, supervision, consultation and support, outcome monitoring, measurement feedbk Family Empowerment Attitudes, Beliefs & Expectancies of Clinicians and Supervisors Attitudes, Beliefs & Expectancies of Families & Youth Improved Implementation Efficiency & Effectiveness Improved Child & Family Outcomes

  12. Prior to implementation: Consider the Fit of the Practice for the Context • Appropriateness for system: population, setting, fiscal support, policies • Appropriateness for setting/organization: Social-organizational context including provider attitudes, beliefs, expectations • Appropriateness for family context and perspectives

  13. Examples of Practical Fit • Implementation Resource Guide (Toolkit) for Disruptive Behavior Disorders (BJ Burns, 2007) • Hawaii EB Services Report (Chorpita & Dalaiden, 2007): Strength of evidence, levels, and practice elements • PracticeWise (Dalaiden & Chorpita) • Contextualized Feedback Systems (Bickman & Kelley, 2010)

  14. Is Study Population Comparable to Yours? Race/Ethnicity White – Black –Hispanic Native Am. - Asian Am. Level of Evidence Age Range Gender Boys or Girls INTERVENTION

  15. Fit with Agency Resources ? Training and Coaching/Consultation Is Follow-Up Coaching Available? How Much Does Training Cost? Is Training Provided by Developer? What is Length of Training? Where is Training Provided? INTERVENTION

  16. Implementation Research in New York State for Children and Families • Focus on clinic referred youth • Youth 0-17 across New York State who are served in outpatient mental health clinics • 21,737 youth served weekly • 87,539 youth served annually with a 95% confidence interval of (77588, 99469) • 239 agencies (357 licensed clinics across 942 sites)

  17. NYS Child and Family Initiatives • Clinic Plus: early detection, partnership, public health • 85 clinics • Evidence-Based Treatment Dissemination Center • 195 clinics/72 also in Clinic Plus • Family Support Services: • Multi-family Groups for Conduct Disorders—13 clinics • Family peer advocates and parent empowerment: • 400 FPAs serving approx 20,000 parents/yr • Clinical decision supports/QI • Psychiatric Services and Clinical Knowledge Enhancement System (PSYCKES) for polypharmacy (322 clinics) • Contextualized feedback systems (4 clinics)

  18. OMH Data Tracking System for Clinic Plus: 2010

  19. Next Step Clinic fiscal restructuring Emphasis on business practices and organizational structures Learning collaborative structure to support teams

  20. Evidence-based Treatment Dissemination Center Funded by OMH to support training and one full year of expert consultation on CBT skills for specific disorders 1212 clinicians/supervisors trained in CBT for trauma, depression, and behavior therapy for disruptive behaviors

  21. Workforce Issues: Basic CBT Training in Mental Health Professions % Training Programs Weissman et al., 2006, Archives of General Psychiatry; Ns: MD=73, PhD=62; PsyD=21; MSW=62

  22. EBTDC Consultation Calls

  23. Consultation Call Findings -Attendance did not vary significantly over the course of the year

  24. What happens on the calls? Small but statistically significant relationship between consultants and therapists discussing specific CBT techniques, r = .33, p<.05

  25. Attendance – Consultant DifferencesYear 1 ConsultantClinician (%) Supervisor (%) 1 88.5 60.0 2 84.8 38.5 3 84.0 49.1 4 74.2* 25.2* Overall 83.3 39.6

  26. Certificate Completion: Consultant Differences ConsultantClinician (%) 1 94.2* 2 68.4 3 82.5 4 68.6 Overall 79.6

  27. Next steps Trauma-focused CBT for residential treatment and hospitals To include milieu-based supports Follow back on continuation of skills Simple measures of adherence Supervisor calls: Content of consultation

  28. Family Support Studies: Multifamily group (McKay & Hoagwood) • NIMH-funded R01: RCT at clinic level. 13 clinics get MFG/13 get SAU • Focus on urban, low-income children of color. • Youth 7-11 and their families • ODD or CD • New clinic referrals • Primarily low-income African American and Latino families • Designed in collaboration with parents & providers

  29. MFG Service Delivery Clinician and parent advocate co-facilitation Clinicians provide professional expertise Parent advocates provide mutual support Parent consumers helped to develop treatment manual (e.g., helped bring stress to the forefront)

  30. MFG Attendance (in comparison to rates on retention in outpatient urban individualized mental health services)

  31. Analyses Random coefficient modeling to examine change over time and differences between MFG and SAU Time modeled as months from baseline using measurements from 4 time points: baseline, mid-way through intervention, post-test , 6 month follow-up.

  32. Preliminary results Pre/Post (N=322) MFG Control Baseline Post Baseline Post Mean (SD)Mean (SD)Mean (SD)Mean (SD)Fb ODD 2.84 2.45***a 2.54 2.86 14.73*** (Iowa Conners) (0.68) (0.91) (0.77) (0.94) Inattention 2.83 2.40***a 2.73 2.73 6.43* (Iowa Conners) (0.75) (0.70) (0.84) (0.71) Social Skills 76.84 83.02***a 81.95 78.05 14.84*** (SSRS) (10.62) (10.48) (13.64) (11.24) aPaired sample t test from baseline to post test. b F tests between condition interaction for repeated measures ANOVA. *p < .05, ***p < .001

  33. Family Support: Skill Training for family peer advocates to improve parent empowerment (Olin et al., 2010) NIMH-funded R34 on Parent Empowerment N=32 family advisors and 124 parents in New York City (85% low income, minority) N=18 trained FA; N=14 comparison Impact on FPA’s knowledge of MH services, collaborative skills, and self-efficacy Parents’ working alliance, self-efficacy, empowerment, strain Significant improvements pre/post in knowledge (p<.001), skills (p<.003) and service self-efficacy (p<.02) among FA Significant difference pre/post among parents working with PEP-trained advisors in working alliance at 6 months (p<.05) but not among parents in comparison group No differences in parents’ service self-efficacy, empowerment, or strain Strongest predictor of parents’ working alliance: working with advisor who provided home/school visits (R2=.61; F=.0001) High levels of depressive symptoms among parents (CES-D average 22.6 (cut off is 16); 2/3 above clinical cut-off) Heterogeneity of agency’s social-organizational contexts and undervalued roles of family advisors

  34. Model of Behavior Change: (adapted from Jaccard et al., 2002) PARENT INTERVENTION Expected Value Social Norms/Pressure Behavioral Intentions Behavior Attitude toward Mental Health Skills/Knowledge, Habits, Environmental Obstacles, Priorities Self-Efficacy Intervention Target Possible Barriers

  35. Model of Behavior Change PARENT INTERVENTION What do I get out of this? What do important others think/do? How motivated am I? Active involvement in child MH services How do I feel about mental health/illness? (Stigma, Past experience with MH system or providers) Behavioral Intentions Skills/Knowledge, Habits, Environmental Obstacles, Priorities Do I believe I know how to navigate the system and overcome obstacles? Intervention Target Possible Barriers

  36. Next Steps MFG implemented state-wide: Will results hold up Engagement training implemented state-wide: Will results hold up FPA and PEP: New NIMH-funded R01 on content of FPA services (using Walkthrough Methods) and improving integration of FPAs into workplace: Focus on social organizational context, enhancing working alliances, improving family and youth outcomes

  37. Dimensions of Organizational Social Context (OSC) (Glisson et al., 2002, 2005, 2008, 2009, 2010) • Organizational Culture • System norms and values • “The way things are done” • What is expected and rewarded • Organizational Climate • Work environment’s psychological impact on employees • Employees’ affective response to their work environment • Employees’ sense of “psychological safety”

  38. OSC Measurement System (Glisson et al) Organizational Culture 1. Proficiency – expectation that service providers will be competent, have up-to-date knowledge, and place the well-being of clients first 2. Rigidity – expectation that service providers will have limited discretion and flexibility, and closely follow extensive bureaucratic rulesand regulations 3. Resistance – expectation that service providers will show no interest in change or new ways of providing services

  39. Five Studies to Date Link Organizational Culture to: Individualized care by 408 service providers in 30 social service organizations (Glisson, 1978) Family-centered care by 131 service providers in four emergency rooms (Hemmelgarn, Glisson & Dukes, 2001) Turnover, work attitudes, and service quality among 283 caseworkers in 33 child welfare and juvenile justice case management teams (Glisson & James, 2002) Service quality in 15 child welfare teams serving 21 urban and rural counties (Glisson & Green, 2006) New program sustainability in nationwide sample of 100 mental health clinics (Glisson, Schoenwald, Kelleher et al., 2008)

  40. OSC Measurement System OrganizationalClimate 1. Engagement – service providerperceptions of personal accomplishment, involvement and concern for clients 2. Functionality – service provider perceptions that they receive the needed cooperation and support to do their jobs 3. Stress – service provider perceptions that they are emotionally exhausted and overloaded in their work

  41. Five Studies to Date Link Organizational Climate to: Job satisfaction and commitment among 319 service providers in 22 human service organizations (Glisson & Durick, 1988) Service quality and outcomes among 250 children served by 32 children’s services offices (Glisson & Hemmelgarn, 1998) Work attitudes and service quality in 33 child welfare and juvenile justice service teams (Glisson & James, 2002) Service outcomes for 1,678 children in 88 county child welfare offices in nationwide NSCAW sample (Glisson, 2007) Therapist turnover in national sample of 100 mental health clinics (Glisson, Schoenwald, Kelleher et al., 2008)

  42. Examples of Clinic Profiles with z scores based on National Norms (Glisson et al 2008) z score z score z score 3.0 3.0 3.0 0 0 0 - 3.0 - 3.0 - 3.0 engaged func stressful engaged func stressful engaged func stressful Climate Climate Climate z score z score z score 3.0 3.0 3.0 0 0 0 - 3.0 - 3.0 - 3.0 resistant rigid proficient resistant rigid proficient resistant rigid proficient Culture Culture Culture

  43. Closing Thoughts • States offer unique laboratories for implementation research to enhance public health impact • broad population base • increasing public health emphasis—cross systems • commitment to quality improvement • commitment to efficiency, especially in current fiscal climate • Requires • Constant negotiation and revisiting of goals • Flexible methods and use of a wider array of methods (mixed) • Flexible personalities among research team • Patience

  44. Conclusion • Improving implementation of EBPs: • Attend to fit between the innovation and context • Identify, measure and track context variables (policy, fiscal, social-organizational, family/peer) • Be judicious in selection of primary context for investigation. Can’t do them all well • Identify practical measures • Use mixed methods to squeeze the most knowledge out of each study • Look for opportunities to piggy-back data collection onto electronic measurement systems already in place

More Related