Outcomes of Children in Care: Sustaining Results

1. Outcomes of Children in Care:Sustaining Results Mark Testa Spears-Turner Distinguished Professor School of Social Work University of North Carolina at Chapel Hill Illinois Child Welfare Data Summit First Annual Leadership January 27-28, 2010

2. Overview • Post-permanency world • Dynamics of change • What the evidence says • Sustaining the results

3. Post-Permanency World: Illinois 51,331 July 2000 36,156 Foster Care Adoption & Guardianship 15,780

4. IL Caseload Reduction Dynamics

5. IL Caseload Reduction Dynamics Front-End Reforms

6. IL Caseload Reduction Dynamics Back-End Reforms

7. What does the Evidence Say? • Front-End Reforms: • Alternative Response – Extended Family Support (EFS) for children in pre-existing kinship care. • Safety Assessment– Child Endangerment Risk Assessment Protocol (CERAP). • Back-End Reforms: • Adoption and Guardianship – Subsidized Guardianship Waiver Demonstration. • Post-Permanency Services – Adoption Preservation & Linkages (APAL) and Maintaining Adoption Connections (MAC)

8. PICO Components of Well-Built Evaluation Questions P – Conditions of the target population for which one wishes to draw inferences. I – Intervention or program whose efficacy and effectiveness one is interested in evaluating. C– The alternative course of action with which one wants to draw a comparison. O – Intended outcome one hopes to achieve.

9. Well-Built Evaluation Questions • Alternative Response: Does the diversion of children in pre-existing kinship care arrangements (P) to Extended Family Support Programs (I) safely reduce the number of children in foster care (O) compared to regular investigation practices (C)? • Safety Assessment: Is the rate of short-term recurrence of indicated maltreatment reduced (O) for children and young persons under 18 years old (P) when structured safety assessments are conducted (I) compared to investigation processes that do not feature structured assessments(C)? • [

10. Well-Built Evaluation Questions • Adoption and Guardianship: Does the offer of subsidized guardianship (I) in addition to the usual permanency options of reunification and adoption (C)increase overall family permanence (O) for children in stable kinship foster care (P)? • Post-Permanency Services: Do former foster youth aged 12 to 17 years old who currently reside in adoptive or guardianship arrangements (P) have fewer unmeet service needs and lower risks of placement rupture (O) if their families receive APAL assessments and referrals (I) compared with similar youth whose families are eligible only for regular post- permanency services (C)? • [

11. The Problem of Selection Bias Interventionv.Comparison group δ + bias Outcome εrrorcomponent(unobserved systematic & random influences)

12. Selection Bias • Selection of intervention & comparison groups. vs. Intervention Group Comparison Group • Is this a valid comparison?

13. Balanced Groups • Selection of intervention & comparison groups. vs. Intervention Group Comparison Group • This is a much better comparison

14. Approximating the Counterfactual Stronger Weaker

15. Approximating the Counterfactual Stronger Weaker

16. Subsidized Guardianship Question (PICO) • Does the offer of subsidized guardianship (I) in addition to the usual permanency options of reunification and adoption (C)increase overall family permanence (O) for children in stable kinship foster care (P)?

17. Randomized Controlled Experiments Interventionv.Comparison group δ δ + bias ∆ = 0 Outcome εrrorcomponent(unobserved systematic & random influences)

18. Statistical Equivalence at Baseline

19. Availability of SG boosted legal permanence Illinois At wave I of the evaluation (1998) there was a 8% permanency advantage for children in the intervention group who were offered the choice of subsidized guardianship compared to children in the comparison group. Increase in both adoptions and guardianships }8.3%

20. Internal Validity SG v.Comparison group δ = 8 additional permanencies per 100 children ∆ ≈ 0 Permanence εrrorcomponent(unobserved systematic & random influences)

21. External Validity • Concern: Do these findings generalize beyond the state’s unique historical circumstances and distinctive policies on kinship foster care? • Fact: Two replications of Illinois’ waiver demonstrations in Tennessee and Milwaukee find that subsidized guardianship increases overall family permanence.

22. Waiver Findings Illinois (June, 2007) Wisconsin (November, 2007) Tennessee (November, 2008) } 6.6% } 15.0% }19.9%

23. No Adverse Impact on Reunifications Concern:SG will discourage CW agencies from reunifying children with their parents. Fact:All three randomized experiments find that reunification rates were not significantly different between families offered subsidized guardianship vs. families denied this choice.

24. No Adverse Impact on Reunifications Differences in Reunification Rates, Intervention vs. Comparison Groups

25. Cost-Effective Concern:SG will end up costing more money in the long-run. Fact: In Illinois after 10 years, offering families the SG option reduced the average length of foster care by 269 days or 22% compared to what would have happened in the absence of the offer. In Wisconsin after 3 years, offering family the SG option reduced the average length of foster care by 133 days or 24%. In Tennessee, analysis is underway.

26. With success, comes new concerns: Post-Permanency Services • Did the push for adoption and guardianships pressure too many families into making ill-considered commitments that will eventually translate into children’s returning to foster care? • Do these families have the necessary supports to weather the challenges of parenting special-needs children, especially as they enter adolescence?

27. Illinois Adoption Preservation and Linkages (APAL) Program: • To address such concerns, the Illinois Department of Children and Family Services (DCFS) funded APAL to assess the needs of adolescents aged 13 or 16 in adoptive or guardianship homes. • Private child welfare agencies in the city of Chicago conducted needs assessment either in person or over the phone when the family’s annual subsidy renewal was due. • DCFS commissioned the Children and Family Research Center (CFRC) to evaluate the impact of the program.

28. APAL Question (PICO) • Do former foster youth aged 12 to 17 years old who currently reside in adoptive or guardianship arrangements (P) have fewer unmeet service needs and lower risks of placement rupture (O) if their families receive APAL assessments and referrals (I) compared with similar youth whose families are eligible only for regular post- permanency services (C)?

29. Intervention v. Comparison • All covered families, in which one of the children was aged 13 or 16 years old, were targeted to receive an in-person or telephone APAL assessment of the needs of the focal child and other children in the home. • Caregivers who requested assistance with any unmet needs of the focal child or other children were referred to a post-permanency program or agency for follow-up. • A stratified random sample of comparison families in the same geographical areas with children aged 12, 14, 15, or 17 years old is the comparison group.

30. Prevalence of Unmet Needs An estimated 76% to 84% of caregivers reported no unmet needs for the focal child under their care.

31. Types of Met Needs Sought and usually received: Counseling was the biggest need – 1,450 or 38% of caregivers said that they needed this service, and most who sought it out received this service. The following other services were usually received if sought (86%-100%):

32. Types of Partially Met Needs Not as likely to be received if sought: Educational services, support groups and mentoring were not as likely to be received if sought – 83% to 77% of the time caregivers were able to get these services if they sought them out.

33. Types of Unmet Needs Least likely to be received: Three types of services fall into this category where caregivers who sought these services were least likely to receive them.

34. Behavior Problem Index An estimated 25% to 34% of focal children fall above 90th percentile of national norms for BPI.

35. Think of Ending Arrangement How often do you think of ending the [Adoption/Guardianship]?

36. Provider Helpfulness Overall, how helpful has DCFS and its service providers been in responding to your needs since the [Adoption/Guardianship] was finalized?

37. Provider Contact by APAL Since September 2007, were you contacted by a worker in person or by phone who wanted to talk to you about service needs for [FOCAL CHILD]? APAL is associated with 3 to 9.7 times higher odds of provider contact than non-APAL. 13 Missing responses

38. Regression Estimation of Intent to Treat (ITT) Effects Interventionv.Comparison group δ + bias ∆ ≠ 0 Outcome εrrorcomponent(unobserved systematic & random influences)

39. Statistical equivalence of groups

40. Statistical equivalence of groups

41. Statistical equivalence of groups

42. Regression Adjustment APAL(v. Non-APAL) εrror component δ Unmet needs BPI Think Ending Satisfaction Placement ruptures β Age at interview, SGH, Psy. Hosp., HH Income & Family care if ill

43. General Linear Model Results

44. General Linear Model Results

45. Relative Risk of Rupture As of September 30, 2009 45

46. Prevalence of Unmet Needs Concern: Most families do not have the necessary supports to weather the challenges of parenting special-needs children, especially as they enter adolescence. Fact: Seventy-six to 84% caregivers report focal children have no unmet needs. • Orthodontia, respite care, and family preservation are least likely to be received. • Unmet need is less an issue of linkage and more a problem of policy.

47. Risk of Post-Permanency Ruptures Concern: The late 1990s push for adoption and guardianships will eventually translate into thousands of children’s returning to foster care? Fact:Less than 10% of adoptive and guardianship placements likely to rupture for host of reasons (e.g. CG death or disability, juvenile detention, runaways, and behavioral issues).

48. EFS Question (PICO) • Does the diversion of children in pre-existing kinship care arrangements (P) to Extended Family Support Programs (I) safely reduce the number of children in foster care (O) compared to regular investigation practices (C)?

49. Alternative Response Extended Family Support

50. CERAP Question (PICO) • Is the rate of short-term recurrence of indicated maltreatment reduced (O) for children and young persons under 18 years old (P) when structured safety assessments are conducted (I) compared to investigation processes that do not feature structured assessments(C)?