DC Home visiting Implementation and impact evaluation
This presentation discusses the significance of rigorous design in evaluating evidence-based home visiting (HV) models. Federal funding for home visiting is dependent on demonstrated implementation fidelity and positive outcomes. It outlines essential implementation and impact study strategies, including fidelity assessments, capacity evaluations, and adherence to critical elements of Healthy Families America (HFA). The importance of continuous monitoring, stakeholder involvement, and effective feedback to ensure successful program implementation and scalability is emphasized. The presentation aims to empower local agencies to enhance service delivery for children and families.
DC Home visiting Implementation and impact evaluation
E N D
Presentation Transcript
DC Home visitingImplementation and impact evaluation Deborah F. Perry, PhD Georgetown University Sarah Kaye, PhD Kaye Implementation & Evaluation, LLC November 27, 2012
Importance of rigorous design • Federal funding is contingent on rigorous implementation and outcomes evaluation • Evidence-based home visiting (HV) models • Implemented with fidelity • Determine impact of HV on children and families served • All local agencies receiving funding for expanded HV must participate in the evaluation study
Evaluation design Impact Study Implementation Study Monitoring & Feedback
Implementation Study Impact Study Implementation Study Monitoring & Feedback
Importance of STUDYING implementation • Effective interventions cannot reach the majority of children and families with scaling up. • Effective interventions, when implemented poorly, do not provide the same benefit for children and families (Durlak & DuPre, 2008). • Implementing new programs and practices requires change (and change is hard). • Practice can “drift” back to business-as usual without support and reinforcement(Breitenstein et al., 2010; McGrew et al., 1994).
IMPLEMENTATION Questions • Are Healthy Families America (HFA) services delivered with fidelity to the HFA model? • Are providers operating with adherence to the 12 HFA critical elements? • Are professional development activities improving staff capacity to delivery high quality HV services? • How are implementation strategies related to fidelity of HFA implementation? • How could implementation strategies be improved to enhance implementation outcomes?
Implementation Study Impact Study Implementation Study • Readiness: qualitative interviews and focus groups • Fidelity: self report and independent raters checklists • Adherence to HFA’s critical elements: HFA Self-Assessment Checklist • Capacity: HV Capacity Assessment Implementation Strategies Review Instrument Monitoring & Feedback
Impact Study Impact Study Implementation Study Monitoring & Feedback
Impact study components • Random assignment of agencies awarded funding for HFA expansion • Cohort 1: training and implementation right away • Cohort 2: baseline data collection for 6 months, then training and implementation • Outcomes aligned with federal MIECHV benchmarks and DC Early Success Framework
Outcomes • What are the differences in outcomes when community agencies implement HFA versus a non-evidence-based home visitation model? • Maternal depressive symptoms • Maternal social support • Maternal efficacy • Breastfeeding initiation and duration • Sleep routines (i.e., sleep location and positioning)
Monitoring & Feedback Impact Study Implementation Study Monitoring & Feedback
Empowerment evaluation • The Community Advisory Board to the evaluation will assist in: • Identification of relevant indicators, • Identification of stakeholders who could benefit from timely data about the status of implementation, • Dissemination of data to pertinent stakeholders. • Implementation and intervention indicators will be shared with DOH, HV Council, provider agency leadership, and other relevant stakeholders through defined feedback loops.
