February 12 Anne Duggan, ScD Research Supported by: NIH, CDC, HRSA MCHB, ACF

1. Evidence-Based Home Visiting Modelsto Prevent Child Maltreatment – Assessing and Addressing Fidelity February 12 Anne Duggan, ScD Research Supported by: NIH, CDC, HRSA MCHB, ACF Annie E. Casey Foundation, Robert Wood Johnson Foundation, David & Lucile Packard Foundation Hawaii Department of Health, Hawaii Family Support Institute, Alaska State Health Department, Alaska Mental Health Trust, Family League of Baltimore City; Safe & Sound Initiative; United Way of Central Maryland

2. Today’s Talk • Describe Evidence-based Models of Home Visiting • Identify basic components of models and associated fidelity measures • Introduce and demonstrate use of a conceptual framework for research to increase home fidelity and impact

3. Evidence-based Practices Anevidence-based practice, also called EBP, refers to an approach to prevention or treatment that is validated by some form of documented research evidence.

4. Rand Corporation Promising Practices Criteria for Research Evidence

5. Three Major HV Models

6. Designations of the Three Models

7. Adoption of PAT, NFP and HFA Home Visiting Models PAT: >3300 sites, 50 states NFP: 118 sites, 25 states HFA: 430 sites, 35 states Red= All 3 programs Blue= 2 programs Green= 1 program

8. What are the Basic Components of a Home Visiting Model? • Families to be targeted • Outcomes to be achieved • Causal chain from inputs to outcomes

9. PAT, NFP, HFA Target Different Families

10. PAT, NFP, HFA Focus on Similar Outcomes

11. Framework for Influence of Home Visiting on Family and Child Outcomes Process Screening & Assessment Direct Home Visiting Services Medical Home Other Needed Community Services Family Functioning Parent Mental Health, SU, IPV Social Support Economic Sufficiency Program Model Outreach Trust Building Crisis Intervention Case Management Parenting Education, Role Modeling, Reinforcement enrollment Parenting Knowledge/Attitudes/Skill Parent-Child Interaction Environment for Learning CAN linkages Child Outcomes Health and Development

12. Measures to Assess Fidelity to the Basic Components of an EBHV Model • How well families are targeted • How well outcomes are achieved • How well each element in the causal chain is carried out

13. What Do We Know about Home Visiting as a Preventive Intervention? • Home visiting can improve outcomes, but effects tend to be modest and variable.

14. From Meta-Analytic Studies:HV can be effective, but effect sizes are small. ES Key Small .20 Medium .50 Large .80 *p<.05 Sweet & Applebaum, 2002

15. From single studies, we see that:Effects can vary over time, & across subgroups. 2-4 Yrs All Families No group difference Poor, Unmarried Teens No group difference Birth – 15 Yrs All Families 0.73 vs. 0.44 p<.05 Violent Families No group difference Birth – 2 Yrs All Families 10% vs. 5%, NS Poor, Unmarried Teens 19% vs. 4% p=.07

16. What Do We Know about Home Visiting as a Preventive Intervention? • Home visiting can improve outcomes, but effects tend to be modest and variable. • Programs like HSP/HFA target • the right families and individuals, • at the right time, • focusing on the right outcomes

17. HHS/HFA Home Visiting Model - Who is Targeted, and When • WHO IS TARGETED • Caregivers in Families with Multiple Malleable Risks • Kempe Family Stress Checklist (“at risk” > 25) • WHEN ARE THEY TARGETED? • Prenatally, at Child’s Birth, Shortly Thereafter

18. HHS/HFA targets the right families – those with multiple, malleable risks for poor parenting. Outcomes in 1st Grade

19. Child’s Developmental Trajectory Genotype Self-Regulation Communicating and Learning Making Friends & Getting Along Environment Risk & Protective Factors Proximal  Distal  Brain Development Caregivers Exposure down with age Peers Exposure up with age Community e.g. Socioeconomic and EC SERVICES HV Targets Right Individuals Caregivers = Primary Influence in EC Source: Adapted from Tremblay, R. E. 2006. www.excellence-earlychildhood.ca

20. HSP/HFA Targets Caregivers at the Right Time: Early Parenting Tracks into Grade School *AOR for Later Use if Tactic Used Birth–3 Years

21. Home Visiting Focuses on the Right Outcomes: Parenting and Its Determinants Partner Relations Social Network Personality / Relationship Capacity Parent’s Developmental History Child Outcomes PARENTING Child Attributes Stresses Adaptation of Belsky’s Framework

22. The quality of parent-child interaction is most vulnerable to maternal relationship insecurity under conditions of stress. Phelps JL, Belsky J and Crnic K. 1998 Mean and 95% CI

23. HHSP/HFA Targets the Right Outcomes: Association of Maternal Depression & IPV with Severe Physical Abuse of Child, Birth – 3 Years

24. HHSP/HFA Targets the Right Outcomes: Depression and IPV  Other Parenting Behaviors

25. What Do We Know about Home Visiting as a Preventive Intervention? • Home visiting can improve outcomes, but effects tend to be modest and variable. • Programs like HSP/HFA target • the right families and individuals, • at the right time, • focusing on the right outcomes • But even if the model seems right, desired outcomes might not be achieved.

26. Overall HSP/HFA Impact was Negligiblefor Most Outcomes So what’s going on? Wrong model?

27. Type III Error Dobson and Cooke, Evaluation & Program Planning, 1980 • Unless fidelity of implementation is determined, it is not possible to determine whether negative impact is due to: • Inadequacies in the model or • Departures from the model

28. “Every system is perfectly designed to achieve exactly the results it gets.” Donald M Berwick, M.D. Institute for Healthcare Improvement • Let’s look at how home visiting services were provided – let’s look at coverage, duration and frequency of visits, visit content……

29. Percent of Families Screened, by Hospital

30. HSP Screening & Assessment Rates

31. Attrition was higher than expected and there was substantial, unintended variation across sites. (Hawaii) • Mean Number of Visits • Active All • AgencyFamiliesFamiliesB 22 16A 19 11 C 28 12 p <.01 <.01 Percent of Families Active

32. Visit Content also Differed from the Model: HVers Often Failed to Respond to Parenting Risks

33. Programs Varied Substantially in Provision of Core Services

34. Framework of Determinants of Integrity (Carroll et al., 2007) Intervention Model • Service Integrity • Fidelity = • Coverage • Duration, Frequency • Visit Content • Competence Outcomes Factors for Integrity Model Complexity & Clarity Implementation System Participant Responsiveness Quality of Delivery Component Analysis To identify essential components of the model

35. We need to understand how family attributes and the implementation system moderate impact. Program Model Outreach Trust Building Crisis Intervention Case Management Parenting Education, Role Modeling, Reinforcement Process Screening & Assessment Direct Home Visiting Services Medical Home Other Needed Community Services Family Functioning Parent Mental Health, SU, IPV Social Support Economic Sufficiency enrollment Parenting Knowledge/Attitudes/Skill Parent-Child Interaction Environment for Learning CAN • Factors for Integrity • Implementation System (Hiring, Training, Supervision, Curricula, Protocols, Monitoring, Linkage Agreements) • Family Attributes linkages Child Outcomes Health and Development

36. Influence of Home Visiting Model Complexity and Clarity (Hawaii) Program impact was compromised …. by drift in the model itself that had arisen in taking the model to scale. Case Plan focused on Risks that Made Families Eligible Original CAN Prevention Program Designation as an EI Program for Children at Risk for CHCN due to Environmental Factors Parent-driven philosophy; IFSP with family as decision-makers in setting goals & strategies

37. Influence of HV Model ClarityStaff and Recipient Understanding of PAT • Similarity • Belief that a strong relationship was important and beneficial to the parents. • Differences • Perception of home visitor’s expertise • Perception of purpose of home visitor showing the child a new activity - From Hebbeler & Gerlach-Downie, 2002

38. Influence of Implementation System - Training SB6 HVers lacked basic knowledge of child development. . Tandon et al. Success by 6 Evaluation, Baltimore, 2004.

39. Influence of Implementation System - Training Training improved staff knowledge, at least short term.

40. Comparison of Study Results with ICMQ Validation Study Results* 12 Months 24 Months • % Agreement Validation Studies91% 89% This study88% 74% • Sensitivity Validation Studies72% 94% This study 0% 7% • Specificity Validation Studies92% 83% This study90% 89% *Bricker D and Squires J, the Effectiveness of Parental Screening of At-Risk Infants: The Infant Monitoring Questionnaire. Overall sensitivity 4-36 months = 63%; overall specificity = 91%

41. The implementation system must include SKILLS TRAINING & FEEDBACK & COACHING. Joyce and Showers, 2002

42. RCT of Coaching to Promote Fidelity & Impact • Statewide random assignment of home visiting teams • Data sources: parent interviews; record review; observation of home visitors; surveys of and in-depth interviews with home visitors and supervisors. Usual HSP Services 15 Teams Stratified by Caseloads and Retention Rates, then Randomly Assigned to Three Study Group Training in HFT + Usual Supervision Training in HFT + Enhanced Supervision

43. Influence of Participant ResponsivenessProgram outreach - most effective in a subset of mothers. An assertive outreach policy promoted retention of mothers with high relationship anxiety.

44. Home Visiting Impact Was Pronounced inMothers with High Relationship Anxiety Example: Depressive Symptoms

45. Where We Go from Here… • Learn What Works, for Whom • Improve Fidelity • Clarify existing models • Build implementation system infrastructure • Understand providers and recipients • Build basic skills • Enhance Home Visiting Models

46. Where We Are Going in Hawaii…5-year ACF-funded project • Staff training and supervision • CQI capacity • Targeting of families • Enhancements to the model • Reconciliation of funding incongruities