August 2012

1. New York State Education DepartmentInterpreting and Using Your New York State-Provided Growth Scores August 2012

2. By the End of This Presentation…. • You should be able to: • Explain what information is included in a teacher growth report • Answer some common questions about the information • Describe the differences between a teacher-level, school-level, and district-level report • Understand how districts, principals and teachers can use the information on the reports as one source of data that can help improve instruction.

3. 4-8 grade ELA and Math Teachers & their Principals Evaluating Educator Effectiveness 2011-12

4. Key Points about NYS Growth Measures • We are measuring student growth and not achievement • Allow teachers to achieve high ratings regardless of incoming levels of achievement of their students • We are measuring growth compared to similar students • Similar students: Up to three years of the same prior achievement, three student-level characteristics (economic disadvantage, SWD, and ELL status) Every educator has a fair chance to demonstrate effectiveness on these measures regardless of the composition of his/her class or school.

5. Review of Terms • SGP (student growth percentile): • the result of a statistical model that calculates each student’s change in achievement between two or more points in time on a State assessment or other comparable measure and compares each student’s performance to that of similarly achieving students • Unadjusted and adjusted MGP (mean growth percentile): • the average of the student growth percentiles attributed to a given educator • For evaluation purposes, the overall adjusted MGP is used. This is the MGP that includes all a teacher or principal’s students and takes into account student demographics (ELL, SWD, and economic disadvantage status).

6. MGPs and Statistical Confidence MGP 87 Lower Limit Upper Limit Confidence Range • NYSED will provide a 95% confidence range, meaning we can be 95% confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be provided. • An educator’s confidence range depends on a number of factors, including the number of student scores included in his or her MGP and the variability of student performance in the classroom.

7. Growth Ratings and Score Ranges 2011-12 The growth scores and ratings are based on an educator’s combined MGP. For detailed information see the webinar posted here:http://engageny.org/resource/using-growth-measures-for-educator-evaluation-in-2011-2012/

8. First let’s look at a growth report about a teacher… Jane Eyre

9. Teacher-level Report Teacher 1D’s Growth Score and Growth Rating are listed here Jane’s MGP = 47 (this is what is used to determine the growth score and growth rating) Teacher 1D does not have any growth data reported for any of the subgroups because 16 student scores are required to report any data Jane’s Upper Limit = 55 and Lower Limit = 39 Teacher 1D has a higher adjusted MGP in Math than ELA District X School #1 Jane Eyre Jane Eyre

10. Common Questions about Teacher-level Reports Will an educator’s adjusted MGP always be higher than the unadjusted since the adjusted MGP takes into account not just prior student achievement but also economic disadvantage, ELL, and SWD status? Division B Percentile Division B Teams Not necessarily. Let’s take the following simplified example of a relay race.

11. Common Questions about Teacher-level Reports What is the difference between an educator’s MGP and the percent of an educator’s students above the state median? Let’s take the following simplified example:

12. Common Questions about Teacher-level Reports What is the difference between an educator’s MGP and the percent of an educator’s students above the state median? Let’s take the following simplified example:

13. Common Questions about Teacher-level Reports What is the difference between an educator’s MGP and the percent of an educator’s students above the state median? Note: in NYSED reports, no MGP is calculated for less than 16 student scores. Let’s take the following simplified example:

14. School-level Report The Growth Score and Growth Rating for the Principal of School #1 are listed here An adjusted MGP and associated confidence range will be reported for each subject and grade level within the school. 49 % of students at School #1 scored above the State median. 36% of the student scores are from economically disadvantaged students, and no scores from English language learners. School #1 has scores broken out by subject for grades 4-6. District X School #1 Summary of Revised APPR Provisions Memo: http://engageny.org/wp-content/uploads/2012/03/nys-evaluation-plans-guidance-memo.pdf

15. School-level Report—Detailed View Each teacher receives an adjusted MGP and associated confidence range that are used to determine the growth rating and growth score Teachers 1E and 1G did not receive any growth data because they are linked to less than 16 student scores Teacher 1B has the most student scores linked to him (43 scores) 43 student scores could not be linked to any of the teachers District X School #1 School #1 has 12 teachers who teach grades 4-8 ELA and Math Teacher 1A Teacher 1B Teacher 1C Teacher 1D Teacher 1E Teacher 1F Teacher 1G Teacher 1H Teacher 1I Teacher 1J Teacher 1K Teacher 1L

16. Frequently Asked Questions Why do some teachers have a combined MGP but no subject-level MGPs? • Minimum of 16 student scores • Student must be continuously enrolled in a course that leads to an assessment for 195 calendar days for ELA or 203 calendar days for Math.

17. Frequently Asked Questions Why are some students unassigned? How did the State define “unassigned”? Students are considered unassigned if the district did not provide a valid teacher-of-record for that student, or if the student-teacher linkage relationship did not meet the continuous enrollment guidelines set forth in the APPR guidance http://engageny.org/resource/guidance-on-new-yorks-annual-professional-performance-review-law-and-regulations/. To meet the continuous enrollment guidelines, a student needed to be linked to a teacher for 195 calendar days for ELA, or 203 calendar days for Math in 2011-12.

18. District-level View—Page 1 NY State Summary Number of student scores included in calculation of State MGP NYS Summary Data—Included on ALL District reports NY Statewide Adjusted MGP = 52 State Median = 50 Statewide about 50% of ELL, SWD, and economically disadvantaged students scored above the State median. District X

19. District-level View—Page 1-2 District Summary District-wide Adjusted MGP Number of student scores included in calculation of district-wide MGP District X Summary Data District X District X District X Summary Data—continued on next page of report

20. District-level View—Page 3 List of Schools District X has two schools that have grades 4-8 ELA and Math scores Principal of School #1 Growth Score = 14 Growth Rating = Effective Principal of School #2 Growth Score = 6 Growth Rating = Developing District X School #1 School #2

21. Using Growth Score results • Beyond evaluation, growth score information can provide additional information to help teachers, principals and districts with instructional improvement. • Of course, these measures are only one of multiple sources of evidence to use for this purpose • The best insight comes from considering the results in the context of other information about a teacher, group of teachers, principal or group of schools.

22. Districts may want to: Analyze district-level information using these reflective questions: How much did our students grow, on average, compared to similar students? Is this higher, lower, or about what we would have expected? Why? How do our MGPs for each reported subgroup (ELL, SWD, economically disadvantaged students, high- and low-achieving students) compare to each other and to our overall MGP? Are there any patterns? Are the MGPs higher, lower, or about what we would have expected? Why? How do the MGPs compare by subject and across grade levels? Why might they be similar or different? What should we do to understand any surprises using other information and evidence? Do we have the right plans in place to aid in professional growth and learning for our educators?

23. Districts may want to: Convene principals to reflect upon their school growth results in context of other information about student learning and teacher effectiveness in their schools: • Use BOCES trainers and/or SED online resources to ensure basic understanding of the measures and what information is found on reports • Engage principals individually or in a group to reflect on questions about their school information in the context of other evidence of teacher effectiveness: • How much did the students of my teachers grow, on average, compared to similar students and how does this differ across teachers? Are there differences across grades or subjects? • How do my teachers’ MGPs differ across each reported subgroup? Do I see any patterns?

24. Districts may want to: Plan for communicating with teachers about their results: • How will teachers get general information about the growth measures? • District or school-level training or self-directed use of SED resources • How and when will teachers receive their individual reports? • Remember SED will have online access for individual educators later in the fall • Use BOCES trainers and/or SED online resources to ensure basic understanding of the measures and what information is found on reports

25. Principals may want to: • Consider the reflective questions in their school-level reports: • See the Principal’s Guide to Interpreting Growth Scores: http://engageny.org/wp-content/uploads/2012/06/Principals_Guide_to_Interpreting_Your_Growth_Score.pdf • See the Sample Principal Report—Annotated: http://engageny.org/wp-content/uploads/2012/06/Principal_Sample_Growth_Report.pdf • Plan how teachers will get the information they need to understand their own growth reports

26. Teachers may want to: • Review materials from SED about growth measures • View the “Growth Model for Educator Evaluation 2011-12” Webinar: http://engageny.org/resource/growth-model-for-educator-evaluation-in-2011-2012/ • View the “Using Growth Measures for Educator Evaluation in 2011-12” Webinar: http://engageny.org/resource/using-growth-measures-for-educator-evaluation-in-2011-2012/ • See the Teacher’s Guide to Interpreting Growth Scores: http://engageny.org/wp-content/uploads/2012/06/Teachers_Guide_to_Interpreting_Your_Growth_Score.pdf • See the Sample Teacher Report—Annotated: http://engageny.org/wp-content/uploads/2012/06/Teacher_Sample_Growth_Report.pdf • Consider the following reflective questions: • How much did my students grow, on average, compared to similar students? Is this higher, lower, or about what I would have expected? Why? • How does this information about student growth align with information about my instructional practice received through observations or other measures? Why might this be?

27. For More Information… Please review our posted Guides for Interpreting Your Growth Scores: http://engageny.org/resource/resources-about-state-growth-measures/ And the guidance on NYS’s APPR Law and Regulations: http://engageny.org/resource/guidance-on-new-yorks-annual-professional-performance-review-law-and-regulations/