Nonie K. Lesaux, PhD Michael J. Kieffer, EdD Joan G. Kelley, EdM S. E. Faller, EdM June 30, 2010

1. Language Diversity & Literacy Development: Increasing Opportunities-to-Learn in Urban Middle Schools Nonie K. Lesaux, PhD Michael J. Kieffer, EdD Joan G. Kelley, EdM S. E. Faller, EdM June 30, 2010 IES Annual Meeting National Harbor, MD

2. Funders & Partners San Diego Unified School District Participating Schools, Teachers, & Students Superintendent’s Office Office of Language Acquisition Curriculum & Instruction Literacy Department Standards, Assessment, and Accountability Division Funding Sources Institute of Education Sciences, U.S. Department of Education Eunice Kennedy Shriver National Institute of Child Health and Human Development Harvard Medical School William and Flora Hewlett Foundation WT Grant Foundation

3. Large numbers of struggling readers, often from low-income, diverse linguistic backgrounds In feeder high schools, the majority of incoming 9th graders read at or below the 7th grade level Biancarosa & Snow, 2004; Lutkus, Grigg, & Donahue, 2007 With increasing grade levels, the sources of reading difficulties become increasingly diverse Catts et al., 2003; Lesaux & Kieffer, in press; Buly & Valencia, 2004 Literacy in the Urban Middle School Developmental Factors Organizational Factors

4. Research Framework • Two developmental studies to inform instruction: • 5-year longitudinal study of ELLs in 4th - 8th grade • Comparative study of ELLs and native English speakers who struggle with reading in 6th grade • Design & evaluation of an approach to teach academic vocabulary • 2007-2008: Quasi-experimental evaluation in 21 classrooms, rich data and feedback from teachers on implementation (Lesaux, Kieffer, Faller, & Kelley, 2010) • 2008-2009: Randomized controlled trial in 14 schools (~60 classrooms)

5. The Gap between Reading Words & Comprehending Text Crosson & Lesaux, 2009; Lesaux, Crosson, Kieffer & Pierce, in press

6. Struggling Comprehenders in Middle School Lesaux & Kieffer, in press

7. Skill Profiles of Struggling Readers: 3-Class Solution

8. Research-Practice Partnership: Guiding Goals To increase vocabulary and reading comprehension levels in mainstream, middle school classrooms large numbers of struggling readers from low-income, diverse linguistic backgrounds To simultaneously address issues of middle school literacy reform Susceptibility to effective implementation of new programs Supporting teachers to meet their students’ needs

9. Three Considerations What the vocabulary & linguistics research indicates + What will engage students + What makes delivering the instruction clear and with ease

10. Framework Targeting depth of word knowledge High utility academic words Multiple, planned active encounters Anchored in text Direct instruction & word-learning strategies

11. ALIAS Elements • Ecologically valid, engaging program for teachers to use in mainstream ELA classrooms • 20-week curriculum, 45 minutes a day • 9 two-week units; 2 one-week review units • Each unit built around 8-9 target words • Starting with rich text • Building depth of word knowledge over time • Engaging students & encouraging word play • Increasing student talk in class • Gradually releasing responsibility to students

12. Building up Knowledge of a Word,Piece by Piece… How to use the word precisely in extended writing Day 8 & 9 The meaning of the word in different contexts How to use the word to write & talk about other topics Day 7 The different word parts inside the word & their meanings The different forms of the word & how they are used Day 5 & 6 Multiple meanings for the word How to represent its meaning graphically Day 4 Its meaning in the article How to use it to talk about the article Day 3 Its dictionary definition What I already know about its meaning Day 2 How to spell it What it looks like What it sounds like Day 1

13. Day 1:Words in Context of Engaging Text Target Words: affect, community, contribute, culture, establish, ethnic, resident, welfare • Platform for Discussion: • community service • tolerance

14. Days 5 & 6:Morphology Word Form Chart Lesson Framework • Introduce new word part: “-tion” • Add target word to chart: contribute/ contribution • Search article for words with “-tion” and add to chart.

15. Days 8 & 9:Writing Lesson Framework • Read and discuss the pre-writing discussion article. • Analyze the writing prompt. • Model planning writing using a graphic organizer. • Work on ideas for writing with partners.

16. Intervention Research : The ALIAS Curriculum2008-09 Randomized Controlled Trial 14 participating middle schools 51 teachers randomly assigned to treatment or control 26 Treatment (39 classes); 25 Control (41 classes) approx. 2300 participating students 70% language minority learners; 30% native English speakers Data Sources: Student assessments Implementation logs; videotaped observations Teacher survey, Teacher interviews (end-of-study) Student focus groups

17. Implementation Reported fidelity = 93% Observed fidelity = 85% ALIAS teachers differed from control teachers on techniques related to supporting reading & language development, but not on overall instructional quality Observations in the control classrooms demonstrated that about 13% of time is spent on explicit vocabulary and comprehension instruction

18. Outcome Measures • Academic Words Taught • Depth of Word Knowledge • Morphological Skills • Understanding of Academic Words in Context • Comprehension of Passages with Words Taught • Gates MacGinitie Reading Comprehension Test • Writing (Oral & Written Language Scales) • California Standards Test – English Language Arts

19. Analyses • 4-level Multilevel Modeling (aka HLM) to account for nesting of students within class periods within teachers within schools • Pretest covariates used to improve precision (i.e., statistical power) and to investigate if treatment effect differs by students’ pretest scores • Effect sizes are robust, whether we include covariates and random effects at each of the four levels • Student* treatment interaction • LM, EO; initial vocabulary level

20. Summary of Findings • Significant and meaningfully sized effects on vocabulary, morphology, and comprehension of texts that included the words • Larger effects on word-level knowledge than on text-level skills • Significant and meaningful sized effects on standardized text-level measures for some students • Effects on writing for below-average students • Effects on state-level ELA for above-average students • Larger effects for LM students on words taught

21. Effects on Vocabulary Knowledge

22. Effects on Vocabulary * Effect Size (Cohen's d)

23. Effects on Text-level Skills *

24. Student by Treatment Interaction on Words-in-Context Comprehension *

25. Effects on Text-level Skills *

26. Student by Treatment Interaction on Writing *

27. Effects on Morphological Awareness * *

28. Student by Treatment Interactionon Morphological Awareness * *

29. Effects on ELA California Standards Test *

30. What helped teachers succeed? • Program Materials “When it comes down to it, it’s you, the classroom, and the curriculum” • Other Teachers in the School Colleagues helped in “debugging” the program. • Program Specialist Newer teachers and teachers with fewer colleagues using ALIAS found specialist especially helpful (Lesaux, Kieffer, Faller & Kelley, 2010)

31. Next Steps • Delayed post-test to determine maintenance effects • Analysis of classroom talk • How much natural variation in the quality of classroom talk exists in mainstream 6th grade classrooms in urban middle schools? • In what ways, if any, does the implementation of the ALIAS alter the quality of classroom talk over the course of the 20-week program? • Influence on reading comprehension and vocabulary? • Explaining variation in implementation • School demographics, school climate, classroom instruction characteristics • Describing Standard Practice (CO, Tx baseline) • Quality of Classroom Relationships (CO, Tx)