Indicated Truancy Interventions: Effects on School Attendance Among Chronic Truant Students

1. Indicated Truancy Interventions: Effects on School Attendance Among Chronic Truant Students By Brandy R. Maynard, PhD, LMSW Campbell Colloquium Copenhagen, Denmark May 29, 2012

2. Acknowledgements Funding provided by the Campbell Collaboration, the Arthur J. Schmitt Foundation and the Institute of Education Sciences (Postdoctoral Training Grant # R324B080008) My team/co-authors: Terri Pigott, PhD, Loyola University Chicago Katherine Tyson-McCrea, PhD, Loyola University Chicago Michael Kelly, PhD, Loyola University Chicago

3. Purpose of the Study • Examine the extant body of truancy intervention research • Examine the effects of indicated intervention programs on attendance outcomes for students with attendance problems • Provide evidence-based recommendations to inform policy and practice • Recommend priorities for future research

4. Study Eligibility Criteria • Type of Studies • RCTs and QEDs • Single group pre-post test • Included as it was anticipated from the literature review that many program evaluations are likely utilizing this design • Analyzed separately from RCTs and QEDs • Types of Participants • Students attending primary or secondary educational institutions in the US or equivalent in other countries • Have an identified problem with school attendance • Types of Settings • Any setting that serves primary or secondary school students • Exclusion- residential facilities

5. Criteria (cont) • Types of Interventions • Stated primary goal of increasing student attendance (or decreasing absenteeism/truancy/school refusal) • Tier 3, indicated interventions targeting students who have been identified prior to treatment as having an attendance problem • Types of Outcome Measures • School attendance • Too few studies reported on same secondary outcome measures (i.e. student achievement, recidivism) to conduct meaningful analysis • Geographical Context • United States, Canada, United Kingdom and Australia • Time Period • Studies published between 1990 and 2009 (research may have been conducted prior to 1990)

6. Search Strategy Methods for locating Studies • Electronic databases (n=19) • Internet/website searches • Personal contacts with research centers, organizations/programs listed in NDPC and NSCE, and researchers • Bibliographies of previous reviews and retrieved studies Search Terms • Outcome: Attendance OR Absen* AND • Intervention: Evaluation OR Intervention OR Treatment OR Outcome OR Program AND • Target behavior/problem: Truancy OR “school refusal” OR absen* OR attendance OR “school phobia” OR school anxiety OR dropout OR expulsion OR suspension AND • Targeted population: Students OR Schools • The search strategy was intentionally broad to locate studies of universal and selective programs for use in future reviews.

7. Study Search and Selection Flow Chart

8. Coding Studies • All studies meeting inclusion criteria were coded using an instrument developed by the PI • Includes items related to: • Bibliographic information and source descriptors • Methods and procedures • Context, nature and implementation of the intervention • Sample characteristics • Outcome data needed to calculate effect sizes • 20% of included studies were independently coded by a second coder • > 80% inter-coder agreement

9. Descriptive Findings Study Characteristics Participant Characteristics Intervention Characteristics

10. Study Characteristics (RCT/QED) Publication Status n % Journal4 25% Dissertation /Thesis 10 63% Other Report 2 12% Sample Size (tx group) <30 2 13% 30-59 5 31% 60-99 4 25% 100-199 2 13% 200+ 3 19% Country US 100% Discipline of 1st author Education 7 44% Psychology 3 19% Social work 3 19% Criminal Justice 0 0% Nursing 1 6% Unknown 2 13% Comparison Condition Nothing or Tx as Usual 14 88% Alt. Intervention 2 13%

11. Participant Characteristics Mean Age13.73 Grade Leveln % Elementary 2 20% Middle School 5 31% High School 5 31% Mixed 3 19% SES Low 3 19% Working class 1 6% Not given 11 69% Predominant Racen % Caucasian 5 31% African Am. 3 19% Hispanic 3 19% Not given 5 31% Pre-test mean rates of absenteeism <10% 1 6% 11-20% 1 6% 21-30% 3 25% 31-40% 3 25% 41%+ 4 33% Not given 4 33%

12. Intervention Characteristics Program Typesn % School-based 1275% Court-based3 19% Community-based 1 6% Collaborative Yes 3 19% No 13 81% Multiple Modalities Yes 9 56% No 7 44% Focal Modality Group 5 31% Family 4 46% Mentoring 3 19% Alt. Ed. 3 19% Contracting 1 6% Treatment Duration (weeks) One event 1 6% 1-4 0 0% 5-9 3 19% 10-18 6 38% 19+ 3 19% Unknown 3 19%

13. Intervention Characteristics ** Categories are not mutually exclusive

14. Meta-Analytic Findings

15. Findings Do programs with a goal of increasing student attendance affect school attendance behaviors of elementary and secondary school students? Results: YES!, but…

16. How Effective are Indicated Attendance Interventions overall? • Mean ES- positive, moderate, significant • Heterogeneity was significant • Moderator analyses were conducted to examine between study differences of effects associated with study, participant and intervention characteristics

17. Forest Plot of Mean Effects

18. Moderator Analyses Study Characteristics Participant Characteristics Intervention Characteristics Random effects models assumed Analog to the ANOVA for categorical variables Bivariate Meta-Regression for interval variables None of the moderators tested demonstrated a significant relationship with treatment effect.

19. Moderator Analysis- Study and Participant Characteristics Study Characteristics Participant Characteristics

20. Moderator Analysis- Intervention Characteristics

21. Clinical Significance If the goal of attendance interventions is to improve student attendance, then the interventions have been successful overall. However, if the goal is for students to be attending school regularly, at least at the 90% level or better, the interventions are failing.

22. Discussion • Relatively few studies of indicated attendance interventions found in search process • There is little evidence to support the vast majority of interventions in existence and “best practices” being recommended • Interventions demonstrated a moderate, positive effect on attendance • Large amount of heterogeneity • Methodological deficiencies of included studies • Most did not demonstrate clinical significance • Post-test absence rates > 10% in most studies • Some meta-analytic results did not support recommended practices in the literature over other practices • Collaborative interventions • Multi-modal interventions

23. Implications for Practice/Policy • Chronic truant students benefit from interventions targeting attendance; thus it is important and worthwhile to intervene. • No type, modality or length of intervention was found to be more effective than others. • The available evidence did not support the use of multi-modal or collaborative interventions over simple, non-collaborative interventions • Interventions did not result in improving attendance to acceptable levels- need for more effective interventions

24. Implications for Research We need to do more…and better!

25. Implications for Research Gaps

26. Limitations • Relatively small number of studies found in search process • Not representative of interventions in existence • Not all potential studies may have been identified or were able to be included • Significant heterogeneity • Interventions may be too diverse to be pooled • Methodological shortcomings of included studies • Definitions of attendance/absence

27. Thoughts on Conducting a Campbell Review (& for a Dissertation) • Challenges/Difficult Choices • Narrowing down question • Search/selection of studies • Gathering data • Areas of support that would be helpful • Benefits