Research and Policy Support Group

1. Evaluating the NYC Core Knowledge Language Arts Pilot: Summary of Kindergarten and Grade 1 Results HIGHLIGHTS Research and Policy Support Group

2. Summary of Kindergarten Results 2008-2009 School Year Research and Policy Support Group

3. Methodology: A multi-method, longitudinal research design Focus of the Evaluation Hypothesis:Kindergarteners taught with the Core Knowledge Language Arts (CKLA) Program will gain reading competencies at a faster rate than their peers. • YEAR 1 • Literacy Assessments (at 10 CKLA schools & 10 comparison schools, 4 of which are Reading First schools): • Pre- and post-test of literacy skills • Additional tests of literacy skills at end of each year • Periodic assessments throughout the year (DIBELS) • Teacher and Administrator Surveys (at 10 CKLA schools): • Assesses satisfaction with and impact of CKR • Case studies (at 3 CKLA schools): • Classroom observations, administrator & teacher interviews

4. Similar Demographics at CKLA and Comparison Schools Percent of Students (Number of Students for School Size) CKLA Schools (N = 584) Comparison Schools (N = 307)* * N = the number of students for whom both fall and spring data were available. Note: These and other data were used to select comparison schools (data as of 2007-08 school year).

5. Different Literacy Domains: Greater Gains & Higher Spring Scores for CKLA Students than Comparison Students in All Literacy Domains W-J Brief Reading 5

6. 5X Greater Literacy Gains for CKLA Students than Students at Demographically Similar Comparison Schools Average Fall-Spring Gain in Scale Score Points Woodcock-Johnson (Brief Reading Test) CKLA Students Significantly Higher p < .001 CKLA Comparison

7. 6X Greater Literacy Gains for CKLA Students than Students at Reading First Comparison Schools Average Fall-Spring Gain in Scale Score Points Woodcock-Johnson (Brief Reading Test) CKR Students Significantly Higher p < .001 (N = 314)* (N = 103)* * Data presented represent the four comparison schools that are Reading First schools and the four CKR schools who were statistically comparable matches. 7

8. At All Achievement Levels, Greater Literacy Gains for CKLA Students than Students at Comparison & Reading First Schools Average Fall-Spring Gain in Scale Score Points Woodcock-Johnson (Brief Reading Test) CKR & All Comparison Schools CKR & Reading First Schools CKR Schools CKR Schools Comparison Schools Reading First Schools 8

9. Summary of Findings • By nearly all measures, Core Knowledge Language Arts (CKLA) students made significantly greater gains in early literacy than peer students. • Compared to peers, kindergarteners taught with the CKLA program made more progress in all areas of reading tested: spelling, phonemic awareness, decoding, and comprehension. • These findings hold for two types of peer comparisons: 1) demographically similar schools and 2) demographically similar schools using Reading First (another intensive, focused literacy program). • Surveys and case studies indicate overall high levels of administrator and teacher satisfaction with the CKLA Pilot, while also offering guidance for year 2 implementation and evaluation work. • Administrators would recommend program to others; teachers rate CKLA more favorably than other programs. • Administrators report change in teacher practice: more data-driven instruction & teacher collaboration. • Teachers and administrators feel more positively about the Skills Strand than the Listening and Learning Strand, particularly regarding student engagement. • Teachers may need more support addressing needs of struggling readers with CKLA & managing time to complete lessons.

10. Summary of Grade 1 Results 2009-2010 School Year Research and Policy Support Group

11. Methodology: A multi-method, longitudinal research design Focus of the Evaluation Hypothesis:First Graders taught with the Core Knowledge Language Arts (CKLA) Program will gain reading competencies and content knowledge (science and social studies skills) at a faster rate than their peers. • YEAR 2 • Assessments (at 10 CKLA schools & 10 comparison schools, 3* of which are Reading First schools): • Pre- and post-test of literacy skills • Additional tests of literacy skills at end of each year • Tests of science and social studies skills at end of each year • Teacher and Administrator Surveys (at 10 CKLA schools): • Assesses satisfaction with and impact of CKLA • Case studies (at 5 CKLA schools): • Classroom observations, administrator & teacher interviews * Last year there were 4 Reading First comparison schools. This year there were 3 Reading First comparison schools: one comparison school used Reading First in Year 1 but not in Year 2.

12. Overall, CKLA and comparison schools had similar demographic profiles. However, the differences between groups, especially % of Black/ Hispanic students, was significantly larger than last year. As a result, all analyses control for student demographics. Percent of Students (Number of Students for School Size) CKLA Students (N = 781) Comparison Students (N = 343)* • N = the number of students for whom both fall and spring data were available. • A random sample of half of the students in the comparison schools were selected for testing. All students in CKR schools were tested. Note: Data from 08-09 was used to select comparison schools (data presented here are from the 2009-10 school year).

13. CKLA students displayed greater gains & higher Spring scores in nearly all literacy and content knowledge domains relative to comparison students Note: All analyses control for student demographic characteristics. 13

14. CKLA students displayed nearly 2x greater literacy gains relative to students at demographically similar comparison schools The magnitude of the group difference was smaller than last year (in Year 1, CKLA students’ gains were 5X greater than their peers). Average Fall-Spring Gain in Scale Score Points Woodcock-Johnson (Brief Reading Test) CKLA Students Significantly Higher p < .0001 (N = 758) (N = 342)

15. CKLA students displayed 6x greater literacy gains relative to students at Reading First comparison schools The magnitude of the group difference was smaller than last year (in Year 1, CKLA students’ gained 14.3 pts while Reading First schools declined 4.1 pts). Average Fall-Spring Gain in Scale Score Points Woodcock-Johnson (Brief Reading Test) CKLA Students Significantly Higher p < .0001 (N = 233)* (N = 100)* * Data presented represent the three comparison schools that are Reading First schools and the three CKR schools who were statistically comparable matches. 15

16. Measuring fidelity of implementation to the CKLA curriculum • Examining implementation fidelity allows us to better determine: • 1) whether achievement gains can be attributable to the CKR program and • 2) which components of the CKR program teachers are struggling to implement and those they are implementing successfully • We took several different approaches to measuring fidelity in the CKR Pilot schools: • Site visits with classroom observations • Site visits were conducted at 5 randomly selected Pilot schools • 15 classrooms were observed • Interviews with administrators and teachers • Interviews were conducted with 13 teachers and 7 administrators

17. Example of a school with high fidelity

18. Example of a school with high fidelity Administrators’ Views

19. Example of a school with lower fidelity

20. Example of a school with lower fidelity

21. Fidelity of implementation and teacher effectiveness made a difference: Three Site Visit Schools Did Not Significantly Gain or Declined in Performance from Fall to Spring: • One of these schools was the low fidelity school described previously. • One of these schools had a teacher new to teaching first grade who had been in and out of the classroom, and the relationship between the principal and this teacher was strained. • All schools spent less than the required 60 minutes on the Listening and Learning Strand and the Skills Strand. • One of these schools also had a very strong focus on writing, which would occasionally cut into their other strand time. Significantly Positive Gain Non-significant Gain Significantly Negative Gain

22. Executive Summary – Achievement Gains • By nearly all measures, Core Knowledge Language Arts (CKLA) students made significantly greater gains in Year 2 than their peers • Overall Achievement • When compared to peers (both peers at demographically similar schools and peers at Reading First schools), 1st graders taught with the CKLA program made more progress in reading and social studies. • Spring science scores were significantly greater for students in pilot schools than comparisons schools, but not significantly different from students in Reading First comparison schools. • High vs. Low Performers • High and low performing CK students made larger gains than comparison students • Year 2 vs. Year 1 of the Program. • Both students with only one year of CKLA instruction and those with two years of CKLA instruction made greater gains than their peers at comparison schools.

23. Executive Summary – Surveys & Case Studies • Surveys and case studies indicate overall high levels of administrator and teacher satisfaction with the CKLA Program and high levels of fidelity in program implementation. • While survey respondents had favorable reactions to CKR, they indicated that they felt CKLA rated similarly to other programs they had used. • Surveys and case studies revealed that teachers are struggling with how to differentiate instruction with CKR, in particular for special populations. • Case study schools’ fidelity to the program aligned with their achievement gains: schools with low fidelity demonstrated less gains