Lonnie Berger

1. Leveraging Harmonized Multi-System Administrative Data to Examine Experiences and Outcomes for Child Welfare-Involved Children, Youth, and Families Lonnie Berger Institute for Research on Poverty and School of Social Work University of Wisconsin-Madison Strengthening Child Safety and Wellbeing through Integrated Data Solutions Conference Sponsored by Penn State’s Child Maltreatment Solutions Network September 27, 2018

2. Acknowledgement • Collaborators: Maria Cancian, Sarah Font, Jennifer Noyes, Vanessa Rios-Salas • We thank colleagues at the WI DCF and DPI for consultation on policy and data issues, and staff at IRP for data system construction and analysis. • The research reported here was supported by DCF, DPI, U.S. DHHS, NIH and IRP. Any interpretation is solely of the authors and may not reflect the views of the sponsors.

3. The IRP Data Core andDevelopment of the MSPF • History: Evolved from a series of evaluation projects conducted by IRP for the State of Wisconsin, including: -- Court Record Database (CRD) 1983-current -- Child Support Demonstration Evaluation (CSDE) 1997-2006 Formally organized as an IRP enterprise in 2009 • Funding: Primarily through research projects funded by grants and contracts from state and federal agencies; administrative support from UW-Madison and the IRP Core grant (USDHHS/ASPE) • Organization: • Administrative Director: Hilary Shager, IRP Associate Director • Programming & Data Supervisor: Lynn Wimer • MSPF Technical Expert: Katie Thornton • Data Sharing and Institutional Liaison: Steven Cook • Data Security Officer: Margaret Darby Townsend • Budget and Pricing: Hilary Shager, Lynn Wimer • Programmers: Mike Curtis, Dan Ross, Jane Smith, Katie Thornton, Lynn Wimer 3

4. SPECIALIZED MATCH(ad hoc/samples): Department of Public Instruction (K-12) Medicaid/Badgercare Claims (CRN, CARES) Department of Revenue Juvenile Circuit Court Records SSI records (CARES) Vital Records (births/paternity) Circuit Court Records (CCAP; foreclosures) Family Court Records (not electronic) TANF Applicants (not electronic) Parent Surveys CSPED Current Wisconsin State Administrative Data Resources CORE – fully linked: • SNAP/Food Stamps (CRN, CARES) • Child Protective Services (WiSACWIS) • AFDC/TANF (CRN, CARES) • Medicaid/Badgercare(CRN, CARES) • Child Care Subsidy (CARES) • Child Support (KIDS) • Unemployment Insurance Benefits (UI) • Incarceration (Dept. of Corrections) • Milwaukee Jail REGULAR MATCH: • Wage Records (UI)

5. Data IntegrationCurrent MSPF Model Child Welfare DOC: State Incarceration CARES: TANF, SNAP Child Care Medicaid KIDS: Child Support orders CS payments CS receipts Paternity Establishment Divorce Not shown: CRN: AFDC/Foodstamps UI benefits Milwaukee Jail School Data UI wage record 5

6. MSPF Programs and Years Available

7. Integrating Administrative Data: Difficulties for the State • Most administrative records not designed with a strict requirement to maintain a single record per individual. • No commonly agreed upon set of high quality identifying variables. • Agencies vary in quality of specific demographic variables (for very legitimate reasons). • Agencies with distinct missions face challenges in determining which data source takes precedence andhowconflicts should be resolved. • Data sets change over time and historical records of changes need to be maintained. • IRP can resolve difficulties, for research purposes – not “legally” accurate; annual point in time. • BUT: ongoing challenges with governance structure and cost

8. Data IntegrationMSPF: Process and Structure • Creation of a “Multi-Sample Person File” (MSPF) • Structure: one record per individual, without distinction between adults and children, or between male and female. • Process: Match/merge all individuals from all primary data sources, using identifying variables with some combination of these traits: a) commonly recorded: name, sex b) uniquely-identifying: SSN, ITIN c) immutable: date of birth, place of birth, birth mother, date of death d) other: parent names, race/ethnicity, across-system pins • Methods: deterministic, probabilistic, & “chaining”

9. Data IntegrationMSPF: Data Files Created MSPF Data Files available for IRP Researchers: • Multi-Sample Person File (demographics of all individuals in the data systems from the date of creation through 2016) • Case Files • Parent/Child Files • Participation Files (month/qtr – early/mid-1990s thru 2016) • Cross-Walk ID files with other research samples • Other: Location, Race/Ethnicity • All files linkable by unique IRP-constructed Personal ID • Identifiers removed • Improved, updated, expanded each year 9

10. Recent Examples of Child Welfare Research Using the MSPF • Multi-program involvement of child protective services (CPS)-involved families • Inter- and Intra-generational overlap between CPS and criminal justice involvement • Associations between home foreclosure and CPS involvement • Effect of full TANF child support pass-through on CPS involvement • Effect of child support enforcement on foster care trajectories • Randomized evaluation of an economic support intervention intended to reduce child maltreatment and CPS involvement • Educational, economic, and social outcomes for youth experiencing and aging out of foster care 10

12. 28% of CPS-involved children and 34% of children in out-of-home placement (OHP) in Milwaukee County in a given year have an incarcerated parent (jail or prison) in that calendar year • Overlap is greatest vis-à-vis fathers and for black children (not shown)

13. 18% of incarcerated (jail or prison) adults in Milwaukee County have CPS-involved children and 6% have a child in OHP in that calendar year • Incarcerated mothers and black adults are particularly likely to have CPS-involved children (not shown)

14. 29% of CPS-involved adolescents and 34% of adolescents in OHP in Milwaukee County were incarcerated (jail or prison) as young adults; overlap is particularly high for men and black individuals

15. 19% of incarcerated (jail or prison) Milwaukee County young adults were CPS-involved as adolescents 7% were in OHP; overlap is particularly high for women

16. Two Recent Papers Examining Outcomes for Youth Experiencing Out-of-Home Placement

17. Motivation • Out-of-home placement (OHP) is relatively common, particularly for disadvantaged children • Children and youth experiencing OHP (and especially aging out of) OHP are at increased risk of a variety of adverse outcomes relative to the general population • Child wellbeing is now an explicit goal of the child welfare system • The implications for policy and practice depends critically on the role that placement/aging out, per se, play in determining outcomes

18. What does that mean? • If children who experience or age out of care have a prior history of adverse experiences and development, then a continued achievement gap may simply demonstrate that placement does not (fully) compensate for past disadvantage and policy and practice may be better served by addressing pre-existing disadvantage than by focusing on the role of OHP itself • If the disruption associated with OHP or aging out significantly compromise social and economic outcomes, then policy and practice would be best served by addressing the specific role and effects of placement or aging out

19. The Challenge • OHP and aging out may be related to individual or environmental factors that are also associated with adverse educational experiences and outcomes • Especially when comparing children in OHP or those aging out to children in the general population, unmeasured differences are likely to bias estimates • Most studies lack longitudinal observations of children before, during, and after OHP, and/or have relied on relatively small samples selected based on CPS-involvement • The use of longitudinal administrative data for the population of children and youth in OHP in Wisconsin allows us to address some of these challenges

20. Approach • Compare children in care or aging out of care to other children with related experiences • Control for extensive family and child characteristics measured in the MSPF or DPI data • In some analyses, use child fixed-effects, which assess changes in children’s own outcomes over time • Analyze differences across a range of outcomes

21. Paper #1:Is OHP Causally Related to Academic Achievement?Berger, Lawrence M., Cancian, Maria, Han, Eunhee, Noyes, Jennifer, and Rios-Salas, Vanessa. (2015). “Children’s Academic Achievement and Foster Care.” Pediatrics, 135(1), e109-e116.

22. Comparison Groups • Current OHP: the child was in an OHP at the time of the test. • Recent OHP: the child was in an OHP at some point in the 12 months prior to the test, but in-home at the time of the test (the month of October in a given year). • Investigated but placed after test: the child had been investigated by CPS at some point between 1 and 5 months preceding the test and was not removed from home prior to the test, but was removed from home after the test and within 6 months of the investigation. • Investigated and not placed: the child was investigated by CPS at some point in the 12 months prior to the test, but was not removed from home prior to the test or within 6 months of the investigation (if the investigation occurred between 1 and 5 months prior to the test). • No CPS, but received SNAP: the child received SNAP in the 12 months prior to the test, but did not experience a CPS investigation or OHP.

23. Methods • Bivariate (raw mean) comparisons with WI averages and among 5 comparison groups. • Regression models adjusting for covariates when estimating differences among the comparison groups. • Specifications differ by outcome, but consist of: • a pooled OLS regression that considers placement status and test scores net of the full set of control variables; • an identical model that adds the previous year’s test score as an additional control; • a final model that includes child-specific fixed effects. • The SNAP-only group is the reference group in all models.

24. Test Scores:OHP Itself is Unlikely to Have a Causal Effect on School Achievement (by OHP Status) Unadjusted Adjusted Notes: Authors’ calculations using DPI and MSPF data. Sample includes students in grades 3 through 8 during school years 2005-06 through 2013-14 . Adjusted calculations control for child fixed-effects and other variables.

25. Conclusion • On the whole: • OHP does not appear to have a particular negative influence on academic achievement. • However, it may more directly influence other educational outcomes (grade retention, disciplinary actions, high school completion) that are more closely associated with social-emotional rather than cognitive development. • We found some differences by placement characteristics. • Generally children experiencing less stability and congregate care fare worse, even when accounting for child fixed effects.

26. Paper #2:Does OHP Lead to Teen Motherhood?Font, Sarah A., Cancian, Maria, and Berger, Lawrence M. (in press). “Prevalence and Risk Factors for Early Motherhood among Low-Income, Maltreated, and Foster Youth.” Demography.

27. Motivation • Teen births are declining, yet CPS-involved youth are estimated to be up to 9 to 10 times as likely to have an early birth: • Teen birth rate of 21% for youth aging out of foster care (Shpiegel et al., 2017) • Birth by age 20 for 18% of youth with substantiated maltreatment (King, 2017) • Is CPS involvement or OHP likely to increase risk of an early birth?

28. Approach • Compare risk of early motherhood among girls 12 to 17 • Low income (in families receiving SNAP), but no CPS involvement • CPS involved, but no foster care • Before and after involvement • Experiencing foster care • Before, during, and after placement

29. Samples • All girls in Wisconsin participating in SNAP and/or involved with CPS • born 1991-1996 (observed from ages 12-18 in CPS records available from July 2004-January 2015) • includes girls 7-12 in 2004, participation between 2004 & 18thbirthday (i.e. longer window if younger in 2004) • Samples: • SNAP-only, N= 48,913 • CPS, no foster care, N= 18,971 • Foster care, N= 4,030

30. Analysis • Survival analysis: Age of first conception resulting in a birth (-9 months),from 12 to 18 • Models include: • Time invariant: SNAP-only, CPS, foster care • Time-variant: SNAP-only, pre-CPS, post-CPS, pre-foster, during foster, post-foster • Supplementary analysis for foster care only: post-foster risk by placement and exit type

31. Risk of Early Motherhood Note: Log hazard coefficients and standard errors. HR=Hazard Ratio. N=5,017,787 person-months; 8,590 births. Models include county fixed effects. a. Applying conventional statistical tests, CPS different from foster care at p<.001 in all models. * p<.05 *** p<.001

32. Timing of CPS and Foster Care as Predictors of Early Motherhood Note: Log hazard coefficients and standard errors. HR=Hazard Ratio. N=5,017,787 person-months; 8,590 births. Models include county fixed effects and all covariates shown in Table 2. Applying conventional statistical tests, a. ≠ before CPS at p<.01; b. ≠ after CPS at p<.01; c. ≠ before foster care at p<.01; d. ≠ during foster care at p<.01; e. ≠ After foster care at p<.01 *** p<.001

33. Risk of Early Motherhood After Foster Care by Exit Type Note: HR=Hazard Ratio. Models include all controls shown in Table 1 plus county fixed effects. Other setting includes placements that were unknown, as well as placements in rare settings such as hospitals. Other exit includes those who ran away or whose exit type was unknown.

34. Conclusions • Youth involved with CPS or in foster care at elevated risk of early motherhood • Measures of family background and experience modestly reduce differentials among low-income, CPS-involved, and foster youth • Comparing outcomes before and after CPS/foster experience suggests that experience does not increase, and may mitigate, risk • Among foster youth, placement setting and exit type associated with differential risk

35. Policy Issues and Next Steps • Policy emphasis on mitigating a negative effects as a result of OHP may be overstated • Suggestive evidence of some benefits from OHP • But… all CPS involved children and youth, including those experiencing OHP and aging out, are at risk of poor educational, social, and economic outcomes • Policy should focus on this larger group of CPS-involved children and families, including foster youth after reunification

36. Contact Information Lonnie Berger lmberger@wisc.edu