Recent Research on Intensive Family Preservation Services

1. Recent Research on Intensive Family Preservation Services and What It Tells Us About the Efficacy of the Model for Placement Prevention and Family Reunification Presented by: Raymond S Kirk, PhD Annual CBCAP/PSSF Meeting Baltimore, MD March 11-12, 2008

2. Research on Intensive Family Preservation Services in the 1980s and 1990s (1) • In retrospect, early research on IFPS was problematic • Hennighan, et al (1996) • Targeting (client eligibility, focus families) • Assessment of risk (“low or moderate” risk for most families => low probability of placement) • Random assignment (violations of assignment to Tx or control groups => non-equivalent groups • Model fidelity (substantial variation across programs comprising the study sample)

3. Research on Intensive Family Preservation Services in the 1980s and 1990s (2) • Kirk, Reed-Ashcraft & Pecora (2002) • Premature use of experimental designs to study new and evolving programs • Questionable interpretation of non-significant statistical findings • Fraser, et al (1997): Non-significant findings may result from an ineffective intervention. But, non-significant findings also may be due to a poorly designed or poorly implemented study, and the resulting failure of that study to detect a significant treatment effect.

4. Recent Research Studies (several studies since 2000) Contradict Earlier Studies • Recent studies demanded treatment fidelity to be included in the research • They controlled or accounted for risk (eligibility, targeting) • Used different statistical methods (meta-analysis; Cox Regression Models) in addition to rigorously enforced experimental methods

5. Rigorous Experimental Study • Blythe & Jayaratne (2002) • High-risk families (court had already approved child removal), true random assignment IFPS or traditional services (including foster care) • At 6 months • 94% of IFPS children living at home or with relative • 34% of non-IFPS children living at home or with relative • At 12 months • 93% of IFPS children living at home or with relative • 43 % of non-IFPS children living at home or with relative

6. Study Using Meta-Analysis • Washington State Institute for Public Policy (WSIPP, 2006) • Meta-analysis of treatment effects in programs with good model fidelity and questionable model fidelity to the Homebuilders Model • High-fidelity programs were significantly more effective at preventing out-of-home placement than other programs

7. Study Using Cox Regression Modeling and Survival Analysis • Kirk & Griffith (2004) • Retrospective study, not requiring randomization • Large sample: • 542 IFPS families, • 25,612 non-IFPS families from general Child Welfare population • IFPS families and non-IFPS families matched an a variety of variables: • Risk: only high risk families (CPS risk assessment) • Other risk factors (e.g., prior substantiated child abuse) • Demographics (child age, child gender, type of maltreatment

8. Reading Survival Curves

9. Considerations about survival curves • Survival curves always start high (~ 100%) and then fall towards the x-axis • “high” survival rates have a shallow slope • “low” survival rates have a steep slope • Defining “success” is critical to the utility of the curves • Sometimes “cumulative probability” curves are more meaningful and easier to read • Generate a “cumulative probability” curve by plotting “1 minus the survival function” • Cumulative probability curves always start at zero and rise, as the predicted events occur

10. Adjusted Cumulative Risk of Placement for IFPS and Non-IFPS Cases from the Cox Proportional Hazards Regression Model

11. Risk of Placement After CPS Report/Referral to IFPS for Children with 1 or More Prior Substantiations of Abuse/Neglect

12. Risk of Placement After CPS Report/Referral to IFPS for Children with 1 or More Prior High Risk Substantiations of Abuse/Neglect

13. Risk of Placement After CPS Report/Referral to IFPS for Children with 1 or More Prior Spells Under Placement Authority

14. The North Carolina Family Assessment Scale (NCFAS)and the North Carolina Family Assessment Scale for Reunification (NCFAS-R) • NCFAS: Assessment of family functioning on 5 domains of interest • Environment • Parental Capabilities • Family Interactions • Family Safety • Child Well-Being • NCFAS-R: includes all of NCFAS plus two additional domains unique to reunification • Ambivalence • Readiness for Reunification

15. Overall Change in Domain Ratings on the NCFAS (N=3,063)

16. Statistical Relationships Between NCFAS Domain Ratings and Placements • For 3,063 families served between 2001 and 2005: • Environment: χ2= 112.336, df = 5, p=<.001; • Parental Capabilities: χ2= 173.691, df = 5, p<.001; • Family Interactions: χ2= 155.416, df = 5, p<.001; • Family Safety: χ2= 222.434, df = 5, p<.001; and • Child Well-Being: χ2= 184.846, df = 5, p<.001.

17. Overall Change in Domain Ratings on the NCFAS-R (N=368)

18. Statistical Relationships Between NCFAS-R Domain Ratings and Reunifications • For the 368 families served between 2002 and 2005: • Environment: χ2= 56.026, df = 5, p=<.001; • Parental Capabilities: χ2= 88.234, df = 5, p<.001; • Family Interactions: χ2= 55.199, df = 5, p<.001; • Family Safety: χ2= 51.218, df = 5, p<.001; • Child Well-Being: χ2= 36.171, df = 5, p<.001; • Caregiver/Child Ambivalence: χ2= 60.670, df = 5, p<.001; and • Readiness for Reunification: χ2= 134.283, df = 5, p<.001.

19. Summary of IFPS Research: 1980s to Present • Early studies inconclusive or equivocal on effectiveness of IFPS for prevention of placements • Practice wisdom was inconsistent with research • Recent research more supportive of effectiveness of IFPS: • Program fidelity is critical • IFPS methods also show effectiveness for reunification • Effectiveness of IFPS for both placement prevention and reunification is linked to improvements on various domains of family functioning

20. An Examination of Intensive Family Preservation Services • Kirk, Griffin & Martens (2007) • Annie E Casey Foundation-funded examination of IFPS and IFRS programs in several states in which • states were experienced, • model fidelity could be estimated, • programs used NCFAS or NCFAS-R • case-level data were available

21. Data collection • States agreed to “cleanse” data of identifying information • States made good-faith efforts to provide data in a consistent format • Results of data collection: • Difficulties related to cross-site comparisons • Missing data • Varying definitions of outcome variables Imagine our surprise!!

22. Program Fidelity: IFPS • All participating sites had well-developed IFPS models • Some variations, including, for example: • length of time that services were available • permissible caseloads • Differences in the types of families served by programs, with some serving a broad mix of family types and others focusing primarily on one type of child maltreatment

23. Results: IFPS • 93% placement prevention rate (i.e., the children were living with their biological parent, adoptive parent, relative; 85% living with the bio-parent), a rate that is in line with previous research on IFPS • No significant differences in the placement prevention rates as a function of type of maltreatment

24. Additional Results: IFPS • Although race and drug data were limited there were no differences in success of placement prevention as a function of race or substance use • Findings suggest: No reason to restrict access to service on the basis of race or substance use • The findings are important to child welfare in that they suggest that IFPS may be an asset to agencies grappling with the issues of substance use and disproportionality in child welfare.

27. Program Fidelity: IFRS • All participating sites had well-developed IFRS models • Compared to IFPS, more variation among programs • length of time that services were available • permissible caseloads • Non-trivial differences in the types of families served by programs

28. Results: IFRS • IFRS data were more mixed than those for IFPS, but were largely positive • Although between-site differences exist with regard to local definitions of reunification, 69% of families were reunited as a result of these services • 54% were reunited with biological parents and the balance were living with adoptive parents, relatives, or guardians.

29. Additional Results: IFRS • Families involved with substances were reunited at similar rates to other families • Unlike IFPS, race and maltreatment type were influencing variables • Blacks had the lowest reunification rates, and those identified as “other” races (non-Whites) had the highest rates. Although the differences were not large, they were significant • families in which physical abuse was the referring problem experienced higher reunification rates than did other maltreatment types, particularly neglect

32. Summary: Kirk, Griffin & Martens (2007) • General findings support IFPS strategies for both placement prevention and reunification cases • High “success” rates • Lack of differences in IFPS outcomes based on race & substance use suggest broad application • Model strengthening and specification needed in IFRS practice settings; additional research on model differences • Differences in reunification rates based on race and maltreatment type also beg additional research

33. Resources: • Contact the National Family Preservation Network thru its website to download the full Kirk, Griffith & Martens(2007)report: www.nfpn.org • Download this presentation from the conference webpage (it will not contain the last 10 slides) • Contact me directly at ray.kirk@ilrinc.com