Assessment, Research, & Accountability Update

1. Assessment, Research, & Accountability Update David Abrams Assistant Commissioner for Standards, Assessment, and Reporting 2008 Teachers’ Institute

2. Performance of ELL Students on 2008 Grade 3-8 ELA Tests • This analysis summarizes the performance of NYS public and charter school ELLs on the Grades 3-8 English Language Arts tests in 2008. The ELA performance of former ELLs who exited in 2006 and 2007 is also presented. • The analyses are based on 72,698 ELLs who took the State Grades 3-8 ELA tests in January 2008 and the NYSESLAT in May 2008. The analyses of the former ELLs are based on the 30,564 ELLs who exited in 2006 and 2007 and whose 2008 Grades 3-8 ELA scores are available. • The ELL population has significantly improved their performance on the Grades 3-8 ELA tests between 2007 and 2008. The percentage of ELLs scoring at Level 3 and above has increased and the percentage of ELLs showing serious academic deficiencies has decreased. • Former ELLs had comparable chance to score proficient (Level 3 or above) on the Grades 3-8 ELA tests as the English proficient students.

3. Number of ELLs Tested On Grades 3-8 ELA Tests

4. Performance of ELLs On 2008 Grades 3-8 ELA Tests

5. The Mean Scale Score increased for ELLs in every grade, though the smallest increase is seen in Grade 8

6. There was an Increase in the Percentage of ELLs Scoring At Levels 3 and 4 except for Grade 8

7. There was an Increase in Percentage of ELLs Scoring At Level 2 and Above

8. There was a Noticeable Decrease in Percentage of ELLs Scoring at Level 1

9. There was an Increase in Percentage of Proficient ELLs Scoring at Levels 3 and 4, but no gain in Grade 8

10. Performance of ELLs on Grades 3-8 ELA Tests by NYSESLAT Proficiency Level: As Student Performance on NYSESLAT increases, there is a greater chance that students will score at Levels 2 and/or Levels 3 & 4

11. Grade 3 ELL Performance By NYSESLAT Proficiency Level

12. Grade 4 ELL Performance By NYSESLAT Proficiency Level

13. Grade 5 ELL Performance By NYSESLAT Proficiency Level

14. Grade 6 ELL Performance By NYSESLAT Proficiency Level

15. Grade 7 ELL Performance By NYSESLAT Proficiency Level

16. Grade 8 ELL Performance By NYSESLAT Proficiency Level

17. ELL Performance By Number of Years of ESL Services

18. ELL Performance By Need/Resource Category

19. ELL Performance By Major Home Language Group

20. Performance of ELLs With Disabilities

21. Performance of ELLs Without Disabilities

22. Performance of ELLs With Disabilities By Disability Type

23. Percentage of ELLs at Level 3-4 onby Disability Status

24. Percentage of ELLs scoring at Level 1 by Disability Status

25. Comparison of Non-ELL, ELL and Former ELL Students’ Performance on Grade 3-8 ELA Tests

26. Comparison of Non-ELL, ELL and Former ELL Students’ Performance: Percentage of Students at Level 3 & 4

27. Percentage of ELLs Scoring at Each of the NYSESLAT Proficiency Levels Passing/Failing the Regents English Exam in Grade 11

28. Percentage of ELLs Scoring at Each of the NYSESLAT Proficiency Levels Passing/Failing the Regents English Exam in Grade 12

29. NYSESLAT Scores Predict ELA Performance Well *Dependent Variable: ELA scale score *Predictors: NYSESLAT Listening, Speaking, Reading, Writing scores

30. Summary • In 2008, the percentage of ELLs meeting the ELA standards increased from 18% in 2007 to 25% in 2008; the percentage of ELLs showing serious academic deficiencies in ELA deceased from 29% in 2007 to 18% in 2008. • NYSESLAT scores are good predictors of the ELA performance, accounting for 43% to 53% of the variance in ELA scale scores across grades. ELLs who scored at the proficient level on the NYSESLAT had a much better chance to meet the ELA standards than those who scored below proficient.

31. Summary • Former ELLs in grades 3 to 5 had comparable chance as English proficient students to meet the ELA standards, the percentage of former ELLs in grades 6 to 8 ranged from 43% to 62%. • ELLs in the elementary grades, with 3-6 years of services, and those from schools outside the Big 5 cities were more likely than their counterparts to score proficient on the Grade 3-8 ELA tests. • ELLs who scored at Level 1 on the 3-8 ELA tests were more likely to be in the middle school grades, with less than 3 years services.

32. SED Academic Language Analysis Research Project • Department is working with Charlene Rivera from the George Washington University: Center for Equity and Excellence in Education (GW-CEEE). • Purpose is to provide a description of academic language associated with the Living Environment Regents Exam and Commencement Level Core Curriculum. • Goal is to pilot an approach for describing the academic language demands. • GW-CEE staff involved in 2 year project with a selected NYS Analysis Team.

33. SED Academic Language Analysis Research Project Goals of NYS Analysis Team: • Describe and articulate the vocabulary, grammar, and academic language functions associated with NYS Living Environment course; • Articulate their expectations for the Living Environment Academic Language Framework; & • Review and make recommendations to improve the usability of the Content Standards Language Analysis Tool.

34. SED Academic Language Analysis Research Project Major Outcomes Include: • Draft list and categorization of academic vocabulary associated with classroom texts and Living Environment Regents Exam; • Draft list, examples, and explanations of grammatical structures that cause difficulty in comprehending Living Environment texts & assessment items; • Preliminary identification and descriptions of the most prevalent academic language functions for Core Curriculum; • Draft outline for the Living Environment Academic Language Framework; & • Design Professional development modules for teachers.

35. SED Academic Language Analysis Research Project • Academic Language: language used in the learning of academic subject matter in formal schooling context; aspects of language strongly associated with literacy and academic achievement, including specific academic terms or technical language, & speech registers related to each field of study (TESOL ESL Standards 4/97). • Academic Vocabulary: includes both specialized academic vocabulary, (e.g. organism, linear equation) and general academic vocabulary (e.g. produce, examine, & specify). Specialized academic vocabulary is associated with concepts of a discipline and general academic vocabulary cuts across academic disciplines.

36. SED Academic Language Analysis Research Project • Grammatical Structures: refers to the structure and arrangement of words in phrases and sentences with in written discourse. The following were selected for this project: compound & complex sentences; nominalization, long noun phrases, passive voice, and long or multiple prepositional phrases. • Academic Language Functions: language performances expected or realized in the doing of an academic tasks either through the production and/or comprehension of written or oral texts (Bailey et al, 2007).

37. SED Academic Language Analysis Research Project Next Steps: • Complete analysis, identification, and refinement of the framework; • Revise Content Standards Language Analysis Tool to provide user-friendly tool for the field; • Develop a draft Living Environment Academic Language Framework • Gather input/feedback; • Finalize Framework; & • Develop professional development modules.

38. New York State NCLB Title III Accountability:Proposed Revisions to Title III AMAOs David Abrams Office of Standards, Assessment & Reporting Dr. Pedro J. Ruiz Office of Bilingual Education and Foreign Language Studies

39. NCLB Title III Requirements • Standards for English Language Proficiency • Conduct an annual, standards-based assessment of English Language Proficiency: New York State English as a Second Language Achievement Test (NYSESLAT) • Define Annual Measurable Achievement Objectives (AMAOs) for increasing percentage of LEP/ELLs progressing toward and attaining English Language proficiency and for meeting academic achievement standards • Hold Local Educational Agencies (LEAs) and Consortia accountable for meeting the AMAOs Source: No Child Left Behind Act (NCLB) of 2001, Title III: Language Instruction for Limited English Proficient and Immigrant Students. Public Law 107-110, January 8, 2002.

40. Accountability Issues Regarding Title III Consortia • A Title III Consortium consists of a group of LEAs that join together as one eligible entity in order to qualify for the $10,000 minimum requirement and jointly apply to the State for a Title III subgrant. • Component districts that form one consortium are held accountable as one entity for meeting all three AMAOs. • AMAO determinations are made for the consortium as a whole by aggregating the data from the component districts to the consortium level (rather than the district level).

41. Title III Annual Measurable Achievement Objectives (AMAOs): Definitions • AMAO 1: Annual increases in the number or percentage of LEP/ELLs making progress in learning English • AMAO 2: Annual increases in the number or percentage of LEP/ELLs attaining English language proficiency • AMAO 3: Adequate yearly progress (AYP) for LEP/ELL subgroup in meeting grade-level academic achievement standards in English Language Arts (ELA) and mathematics Source: No Child Left Behind Act of 2001, Title III: Language Instruction for Limited English Proficient and Immigrant Students. Public Law 107-110, January 8, 2002.

42. NYS Title III AMAO 1: Making Progress • Proposed Revised Definition: 1) Advance one overall proficiency level on the NYSESLAT between two consecutive years; 2) Maintain the same proficiency level and make a total scale score gain of 43 points between two years; or 3) Score at Level 2 (Intermediate Level) or above on the NYSESLAT for ELLs with one data point only. • Unit of Accountability: Title III LEAand Consortia • ELLs Included in AMAO 1 Determination: All identified ELLs in the current school year

43. NYS Title III AMAO 2:Attaining English Proficiency • No proposed change from previous definition • Definition: Scoring at the proficient level (Level 4) on both Listening & Speaking (L/S) and Reading & Writing (R/W) modality combinations of the NYSESLAT • Unit of Accountability: Title III LEAand Consortia • ELLs Included in AMAO 2 Determination: All identified ELLs in the current school year

44. Title III AMAO 3: Making AYP • Definition: LEP/ELL subgroup must make AYP at the district level in meeting grade-level academic achievement standards in ELA and mathematics. • Unit of Accountability: Title III LEA and Consortia • Data source: Title I District AYP determination for the LEP/ELL subgroup; aggregated district data for Consortia.

45. For Title III accountability purposes, all Title III LEAs, including Consortia, must meet all three AMAO targets each year to be considered making AMAOs. Final Determination of Title III LEA AMAO Status

46. Accountability Count of Title III LEAs and Consortia in NYS • 2004-05, N=184 • 2005-06, N=191 • 2006-07, N=208 (including 10 Consortia)

47. Current Title III AMAO Targets e.g.For 2005-06, 60 percent of LEP/ELLs in each LEA must make AMAO 1 and 10 percent of LEP/ELLs must make AMAO 2.

48. Percent of Title III LEAs Meeting Current AMAOs Projected

49. Rationale for Revising Title III AMAOs • The current AMAO targets and increments were established in 2003 on estimated projections; longitudinal empirical data were not then available. • The current AMAO targets and annual increments were set with the assumption that 100 percent of LEP/ELLs must meet the AMAO 1 and AMAO 2 targets by 2013-14 as per Title I accountability requirements.

50. Rationale for Revising Title III AMAOs (Cont.) • Based on technical assistance from USED, that assumption has been revised. Although annual increases in AMAO targets are required, 100 percent attainment by 2013-14 is not. • The revised NYSESLAT (2005) differs from the original NYSESLAT in number of items and total score points available. The original AMAOs require revision due to change in test design.