Use of NDACAN Data to Inform Training Evaluation

1. Use of NDACAN Data to Inform Training Evaluation Elliott G. Smith National Data Archive on Child Abuse and Neglect National Human Services Training Evaluation Symposium June 14, 2011

2. The plan for today’s session • Who are you? • What is NDACAN? • How are secondary data relevant to your work? • Brainstorm ideas and opportunities

3. NDACAN’s Mission • Mission Promote secondary analysis of existing data related to child abuse and neglect • Funding provided by the Children’s Bureau, U.S. Department of Health and Human Services

4. Core Functions of NDACAN • Acquire child maltreatment data • Prepare the data for secondary analysis • Protect the confidentiality of respondents • License the data to eligible researchers • Encourage data use and collaboration • Support data users

5. www.ndacan.cornell.edu

7. The Spirit of the Times • More data are being collected • Expectations for accountability have grown • In addition to process data, outcome data are being collected in administrative systems • Data are easier to obtain • The Internet age and online search make data widely available • Federal initiatives encourage data sharing • Data are easier to use • Statistical software has gotten easier to use • Sophisticated analytic techniques are available to a wider research audience

8. Advantages of Secondary Data • Inexpensive • Often exempt from human subjects IRB review • Administrative data are comprehensive • Large samples allow subgroup analysis • Administrative data are collected over the long term and are repeated at regular intervals

9. Difficulties of Secondary Data • It takes longer than you expect to understand a secondary dataset • Respondents may not fit researcher’s target group • Constructs may not be measured adequately or in a way that fits the researcher’s goals • For administrative data, the definition of community is problematic (e.g., county level data) • For administrative data, only service recipients are included in the system

10. Children’s Bureau Data Collection • Administrative Data • National Child Abuse and Neglect Data System (NCANDS) • Adoption and Foster Care Analysis and Reporting System (AFCARS) • National and Cross-Site Surveys • National Survey of Child and Adolescent Well-Being (NSCAW) • Longitudinal Studies of Child Abuse and Neglect (LONGSCAN) • National Incidence Studies of Child Abuse and Neglect (NIS)

11. NCANDS Child File

12. NCANDS Description • National effort to collect detailed data about all maltreatment investigations conducted by CPS • Case-level data collected since 1993

13. NCANDS is the data source for Child Maltreatment

14. NCANDS: Data File Structure • The dataset is one LARGE rectangular file (3.6 million records) • Each record is a report-child pair of reports disposed in the Federal fiscal year • For some states, it is possible to link adjacent years by the encrypted Child ID

15. NCANDS: Child File Content • Report Data • Child Data • Maltreatment Data • Child Risk Factors • Caregiver Risk Factors • Services Provided • Staff Data • Perpetrators Data

16. Child File: Report Data • Encrypted IDs for report and child • State • County of report (limited) • Report date, investigation start date • Report source • Report disposition, disposition date • Agency notifications

17. Child File: Child Data • Age at report • Sex • Race and Hispanic origin • Living arrangement • Family member in the military • Prior victim

18. Child File: Maltreatment Data • Maltreatment type (up to 4 types) • Physical Abuse • Sexual Abuse • Neglect • Medical Neglect • Psychological Maltreatment • Disposition of each allegation • Substantiated • Indicated • Alternative response (victim or not victim) • Unsubstantiated • Closed – No finding

19. Child File: Child Risk Factors • Alcohol and drug abuse • Mental retardation • Emotional disturbance • Visually or hearing impaired • Learning disabled • Physically disabled • Behavior problem • Other medical condition

20. Child File: Caregiver Risk Factors • Alcohol and drug abuse • Mental retardation • Emotional disturbance • Visually or hearing impaired • Learning disabled • Physically disabled • Behavior problem • Other medical condition • Domestic violence • Inadequate housing • Financial problem • Public assistance

21. Child File: Services Provided • Any post investigation services • Family support services • Family preservation services • Foster care services, removal date • Juvenile court petition, petition date • Court-appointed representative

22. Services Provided (cont.)Availability varies by state • Adoption • Case management • Counseling • Daycare • Educational & training • Employment • Family planning • Health-related & home health • Home-based • Housing • Independent & transitional living • Information & referral • Legal • Mental health • Pregnancy & parenting • Respite care • Special-disabled • Special-juvenile delinquent • Substance abuse • Transportation

23. Child File: Perpetrator Data • Up to 3 perpetrators • Relationship • Parental status • Caregiver status • Age at report • Sex • Race and Hispanic origin • Military member • Prior abuser • Maltreatment outcome

24. AFCARS

25. AFCARS: Description • National effort to collect child level information regarding every child in the U.S. foster care system and every child adopted through public child welfare agencies • Detailed data regarding the child’s most recent removal • Significant state participation since 1998

26. AFCARS Data File Structure • Each record corresponds to an individual child • NDACAN distributes an annual file • Each annual file consists of about 800,000 records • For some states, it is possible to link adjacent years by the encrypted Record Number

27. Is AFCARS a longitudinal database? • Not really • Detailed case information for the most recent removal • However, 95% of foster care children have 1 or 2 removals, allowing a complete timeline

28. AFCARS: Foster Care File Content • Child demographics (Age, Sex, Race) • Child disabilities • Total number of removals • Case relevant dates (first removal, latest removal, discharge, latest placement, TPR) • Removal reasons • Placement setting • Foster parent demographics • Discharge reason • Various subsidies and support

29. AFCARS: Adoption File Content • Child demographics (Age, Sex, Race) • Child disabilities • Date of adoption, dates parental rights terminated • Adoptive parent demographics • Relation to child prior to adoption • Adoption subsidy

30. NSCAW

31. NSCAW: Description • Federally mandated national survey of children investigated following a maltreatment allegation • First national study of child welfare services to collect data from children and caregivers and detail about the home environment • First study able to produce national estimates of well-being, safety and permanency for children involved with CPS Adapted from Webb et al. (2007)

32. Two-Stage Sampling Design • Stage 1: Selection of PSUs (counties, mainly), with probability of selection proportionate to size, based on estimates of size of child welfare population in each U.S. county. Public child welfare agencies in selected counties were recruited for participation • Stage 2: Selection of children within PSUs • children were ages 0-14 at time of sampling • All children who were investigated during 16 month baseline period were eligible for sample Adapted from Webb et al. (2007)

33. NSCAW Child Sampling • Oversampled on the basis of: • Children/Families Receiving Services • Infants • Sexually Abused Children • Not Sampled on the basis of: • Substantiated Reports (cases are included whether substantiated or not) Adapted from Webb et al. (2007)

34. NSCAW Cohort Total 6,228 Longer-term foster care 727 Enter through investigation 5,501 No services 1,421 Ongoing services 4,080 In home 2,803 Out-of-home 1,277 Adapted from Webb et al. (2007)

35. Data Collection Timeframe Wave 1: Baseline Nov, 1999 – Apr, 2001 Target population: Children involved in investigations closed between October 1, 1999 and December 31, 2000 Wave 2: 12 Month Follow-up Oct , 2000 – Apr, 2002 Wave 3: 18 Month Follow-up Apr, 2001 – Sept, 2002 Wave 4: 36 Month Follow-up Oct, 2002 – Apr 30, 2004 Wave 5: ~6-7 Year Follow-up Sept, 2005 – Feb. 2007 1999 ‘ 2000 ‘ ‘ ‘ ‘ 2001 ‘ ‘ ‘ ‘ 2002 ‘ ‘ ‘ ‘ 2003 ‘ ‘ ‘ ‘ 2004 ‘ ‘ ‘ ‘ 2005‘ ‘ ‘ ‘ 2006 Adapted from Webb et al. (2007)

36. Data Sources • Children • Assessments by Field Representatives (all children) • Interviews (children 7 and older) • Caregiver (parent) interviews • Caseworker interviews • Teacher questionnaires (children 5 or older) • Agency and PSU data for context Adapted from Webb et al. (2007)

37. Measurement Domains • Child functioning and well-being • Services to children and families • Family and caregiver functioning • Community environment • Agency and system level factors Adapted from Webb et al. (2007)

38. NSCAW Child Interview • FE. Future expectations • CD. Depression • TR. Trauma Symptom Checklist • EV. Exposure to violence (VEX-R) • YA. Youth Activities • YB. Child Behavior Checklist Self Report • SA. Substance abuse • SX. Sexual activity • DE. Delinquency • IJ. Injuries • CM. Child maltreatment • BD. Battelle Developmental Inventory (BDI) • NI. Bayley Neurodevelopmental Infant Screener (BINS) • CO. Preschool Language Scales • KB. Cognitive Status (K-BIT) • AH Mini-Battery of Achievement • SE. School engagement • RP. Relationship with peers • PF. Protective factors • RC. Relationship with caregiver(s) • CL. Closeness to caregiver(s) • PM. Parental monitoring Adapted from Webb et al. (2007)

39. NSCAW Caregiver Interview • HH. Household roster • LE/LV. Child living environment • CE. Community environment • HS. Child Health & Services • VI/VN/VE. Adaptive Behavior • PS/PT/PU. Prosocial skills • TE. Emotional regulation - Temperament • TB. Emotional regulation - Toddler behavior • TC/BC - Child Behavior Checklist Parent Report • IN. Income • SH. Services - baseline • SR. Services - post-baseline • SS. Social support • PH. Physical health • DP. Depression • Audio-CASI • AD. Alcohol dependence • DD. Drug Dependence • IL. Involvement with law • DS. Discipline and child maltreatment • SF. Satisfaction w/caseworker • HM. HOME Scales Adapted from Webb et al. (2007)

40. NSCAW Foster Caregiver Interview • HH. Household roster • LE/LV. Child living environment • CE. Community environment • HS. Child Health & Services • VI/VN/VE. Vineland Adaptive Behavior • PS/PT/PU. Prosocial skills • TE. Emotional regulation - Temperament • TB. Emotional regulation - Toddler behavior • TC/BC - Child Behavior Checklist Parent Report • IN. Income • Foster caregiver services • HM. HOME Scales • AM. Adoption module Adapted from Webb et al. (2007)

41. NSCAW Caseworker Interview • CI. Case investigation • AA. Alleged abuse • RA. Risk assessment • SP. Services to parents • SC. Services to child • HB. History before case report • HR. History since case report • LN. Living environments • IV. Caseworker involvement • CP. Family compliance and progress • DM. Decision-making • CT. Court hearings • CB. Caseworker background Adapted from Webb et al. (2007)

42. LONGSCAN

43. LONGSCAN: Description • A consortium of five different studies of the causes and consequences of child maltreatment • While the sites have different samples, they have common measures, data collection methods and schedules • Data are collected longitudinally at ages 4, 6, 8, 12, 14, 16, and 18. • Wide array of measures at multiple levels of the child’s ecology

44. LONGSCAN: Measurement • Guided by Social-Developmental-Ecological Theory (NRC, 1993; Bronfenbrenner, 1989; Hawkins & Catalano, 1996). • Domains assessed: • Child/Youth • Caregiver • Family microsystem: Home environment, • functioning • Macrosystem: Neighborhood, school, support • Review of official CPS case reports Slide provided by Desmond Runyan 44

45. LONGSCAN: Domains • Child/Youth • Demographics • Physical health and medical history • Child care and separations from caregiver • Social competence • Temperament • Cognitive function and academic achievement • Socioemotional function • Developmental status, adaptive behavior, living skills • Problem behavior and risky behavior • Exposure to violence, maltreatment, alcohol and drugs

46. LONGSCAN: Domains • Caregiver • Demographics • Physical health • History of loss and victimization • Parenting attitudes • Substance use • Mental health

47. LONGSCAN: Domains • Family • Household composition • Income and poverty • Service use • Family function and satisfaction • Daily stress • Quality of partner relationship, domestic violence • Quality of relationship with the child, physical discipline, parental monitoring • Home environment

48. LONGSCAN: Domains • Macrosystem • Community income and employment • Neighborhood characteristics • Social support for the parents and the child • School safety

49. Knowledge creation is iterative

50. The Evaluation Pyramid Assess Efficiency Assess Outcomes Assess Program Process Assess the Program’s Theory of Change Assess Need for the Program Adapted from Rossi, Lipsey, & Freeman (2004)