Responding to Nonresponders: An Experimental Field Trial of Identification and Intervention Methods

1. Responding to Nonresponders: An Experimental Field Trial of Identification and Intervention Methods Kristen L. McMaster University of Minnesota Douglas Fuchs, Lynn S. Fuchs, and Donald L. Compton Peabody College of Vanderbilt University Responsiveness-to-Intervention Symposium December 4-5, 2003 • Kansas City, Missouri The National Research Center on Learning Disabilities, a collaborative project of staff at Vanderbilt University and the University of Kansas, sponsored this two-day symposium focusing on responsiveness-to-intervention (RTI) issues. The symposium was made possible by the support of the U.S. Department of Education Office of Special Education Programs. Renee Bradley, Project Officer. Opinions expressed herein are those of the authors and do not necessarily represent the position of the U.S. Department of Education. When citing materials presented during the symposium, please use the following: “McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.”

2. Recognized Facts • Over the past 20 years, we’ve learned that most poor readers have phonological processing problems and poor word recognition skills. • Programs emphasizing phonological awareness and decoding can greatly improve many young children’s reading performance. McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

3. Yet, A Persistent Problem • 20 – 30% of children at risk • 50% or more among children with high-incidence disabilities A sizable minority of students remain unresponsive to generally effective reading programs McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

4. Two General Approaches to Unresponsive Students • Special-education-like approach: Students temporarily removed from the classroom to receive focused supplemental reading instruction from well-trained teachers. • General-education approach: Begins with classroom teacher adapting instruction, curricula, and materials. Important because many students’ reading problems are due to poor classroom instruction and because current reforms emphasize the classroom as a first step in addressing students’ academic problems. McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

5. Purpose • Individualized adaptations of classroom instruction. • More intensive one-to-one tutoring. Experimental study of two alternative interventions for students unresponsive to generally effective classroom instruction: McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

6. Participants • 8 Metro-Nashville Public Schools: 4 high poverty Title I; 4 middle class • 33 first-grade teacher volunteers, stratified by Title I vs. middle class schools, randomly assigned within school to one of 3 conditions: Standard PALS (n = 11), PALS + Fluency (n = 11), and no-treatment controls. • No-treatment controls did not participate in present study and Standard PALS and PALS + Fluency classes combined to create 22 “PALS” study classes. McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

7. Identifying Nonresponders Risk Pool • 8 students in each of 22 PALS classes chosen as “lowest-achieving” based on RLN scores and teacher judgment (N = 176; 22 classes x 8 students). Nonresponders • Monitoring of risk pool for first 7 weeks of PALS on: • PALS unit tests (percentage of items correct) • Dolch Word List (number of words read correctly in 1 min.) • Nonword Fluency Test (number of phonemes read correctly in 1 min.) McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

8. Identifying Nonresponders(Cont’d) • Z-scores computed for risk pool’s level of performance and rate of growth on both Dolch and NWF using 88 average achievers’ performance (4 average achievers x 22 classrooms). • Students identified as nonresponders who scored (a) < 90% on the last PALS unit test, and (b) scored at least .5 SDs below average achievers’ on Dolch Word List and NWF test. • 66 nonresponders identified; or 38% of risk pool and 14% of general population (N = 484; 22 classes x 22 students per class) McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

9. Treatment Levels Control PALS • Students continue with PALS program in their classrooms, 35 min. per day, 3 x per wk. Adapted PALS • Students continue working with a partner in their classrooms during PALS • “Coach” is trained to provide more modeling and feedback • Lessons introduce sounds and words at a slower pace • 35 min. per day, 3 x per wk. Tutoring • Students work with a trained adult tutor outside the classroom during PALS • Students are trained to mastery on sounds and words • 35 min. per day, 3 x per week Treatment Duration = 12 weeks McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

10. Fidelity of Treatment Control PALS • Checklist of teacher and student behavior • Fidelity checked in December and March in each of 22 classes • Mean fidelity across teachers and students = 92% Adapted PALS • All students and their partners observed once in April • Mean fidelity = 86%; range = 49% – 100% Tutoring • Each of 8 tutors participated in simulated tutoring sessions • Mean fidelity = 97% • One tutoring session per student audiotaped • Mean fidelity = 96% McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

11. Findings • Were the interventions insufficiently long or intense? • Was the nature of the interventions too similar? • Was our statistical power too low? No statistically significant differences in reading among students in PALS, Adapted PALS, and Tutoring McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

12. Effect Sizes (ESs) • ESs indicated greatest growth among Tutored students on: • Word attack • Comprehension • Greatest responsiveness among Tutored students on growth criteria. • When considering prior evidence of tutoring effectiveness, our results can be interpreted as indicating the superiority of tutoring over classroom adaptations. (The joint p-value of our findings + others’ findings is low.) McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

13. Effect Sizes by Secondary Intervention Option McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

14. Proportion Nonresponders by Secondary Intervention Option and Criterion Final Status McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

15. Adapted PALS • Wide range of fidelity (given teachers’ inadequate monitoring). • For students demonstrating poor responsiveness to generally effective classroom instruction, adaptations of that instruction may be inadequate. • However, don’t forget the ES for Word ID (.44). Classroom adaptations may adequately address some dimensions of reading development more than others. Repeated exposure to words, supervised by capable peers, may benefit Word ID development. By contrast, Word Attack may require trained adult tutors. McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

16. Criteria for Identifying Nonresponders • Final status percentile rank and growth produced similar rates of LD as traditional methods (7 – 8%). • Final status benchmark appears too stringent. Students grow well without achieving benchmark; prevalence rate = 17%. McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.