Implications of Teacher-Level Reports

1. Race to the Top Implications of Teacher-Level Reports Ohio RttT Webinar Presented by Battelle for Kids June, 2011

2. Race to the Top Current National Perspective on Measures of Teacher Quality and Evaluation Current Ohio Context -RttT, HB 153, Ohio TIF Value-Added and Teacher-Level Reports of Classroom Effects Getting Ready for Teacher-Level VA Reports District, School and Teacher Levels Challenges as we move forward Political Cultural Practical

3. National Perspective- Measures of Teacher Quality and Evaluation • Race to the Top Assurances • Gates Initiatives • Teacher Evaluation & Performance Pay • Recent State’s initiatives • Ranking Teachers by Value-Added Effects • LA Times • Resource Suggestions • Video – LA Times’ Reporters on Understanding VA (http://www.latimes.com/videobeta/?watchId=820c2ee2-9652-44a8-8981-bbbc8c901bfa)

4. Race to the Top • Rollout Schedule • 30% of LEAs Link in Year 1 RttT (reports received fall 2011) • 60% of all RttT LEAs in Year 2 • 100% of all LEAs in Ohio in Years 3 & 4 • Requirements—Accuracy of Reporting • Must conduct linkage • Minimum number of students and time enrolled • Access to Reporting • Online via EVAAS® accounts • Password protected • Grades/Subjects Available • ODE: grades 4-8, math & reading • BFK: grade 3, math & reading; grades 3-8, science & social studies; high school—algebra I & II, geometry, pre-calculus, biology, chemistry, English 9, 10 & 11 Teacher-Level Value-Added Reporting

5. Race to the Top Requires school districts/LEAs to establish new policies relative to teacher evaluation that include these requirements: • At least 50% of each evaluation is to be based on measures of student academic growth • Evidence-based and use multiple measures of a teacher's use of knowledge and skills and of students' academic progress • Alignment with the standards for teachers • Statements of expectations for professional performance • Observation of the teacher on at least two occasions for not less than thirty minutes on each occasion • Each teacher be provided with a written report of the results of the teacher's evaluation and recommendations 2011 Budget Bill Discussions

6. Race to the Top 2011 Budget Bill Discussions (cont’d) • When applicable to a teacher, those measures shall include student performance on the assessments and the value-added progress metric • For teachers of grade levels and subjects for which those measures are not applicable, student assessments that measure mastery of the course content for the appropriate grade level, which may include nationally normed standardized assessments, industry certification examinations, end-of-course examinations developed or selected by the board, or assessments may be used • Shall include student growth data for students assigned to the teacher during the three most recent school years • If less than three years of growth data are available, the board shall use the growth data for all of the school years that are available and the student academic growth portion of the teacher evaluation may be reduced to 40%

7. Race to the Top Teacher Incentive Fund (TIF) • Value-added use in 24 Ohio TIF districts/LEAs as one part of multiple measures within award designs (in grades/subjects available) • Participating districts/LEAs will: • Define what it means to be an effective educator • Utilize quality, reliable data to inform instructional practices • Enhance the evaluation systems for principals and teachers to improve student achievement • Design award programs that recognize educators for their contributions to student achievement and leadership in the district/LEA • Make data-driven decisions about professional development opportunities

8. Race to the Top Multiple Measures • MET resources (www.metproject.org) • White paper: Working with Teachers to Develop Fair and Reliable Measures of Effective Teaching • Teachers Talk About Multiple Measures Video • MET Project Preliminary Findings Research Report • Home Runs, RBIs and Batting Averages: How Today's Educators Measure Up • http://www.battelleforkids.org/About_Us/Publications

9. What Value-Added Results Reveal • Is a group statistic – measures the impact schools and teachers have on a group of students • It’s about us, the adults • Tells us: Is this “program” working? For whom? • My classroom • My team • My school • My district • It’s about the past Remember: Value-Added results reveal if your ‘instruction has been working

10. What reports MAY look like

11. What reports MAY look like

12. Getting Ready: Lessons Learned from BFK • What District-Level Teams Should Know • What Building- Level Teams Should Know • What Teachers Say They Need

13. What District Level Teams Should Know • Don’t assume building leaders/principals know how to interpret value-added reports. • Provide tangible action steps and expectations for BLT’s to follow for preparing and sharing value-added data with teachers. • Expect to use value-added information to inform improvement plans. • BLT’s may need support to translate data into goals or action steps. • Your public is becoming more interested in your value-added results. Anticipate and create a communication plan.

14. Fact or Fiction? • Teachers from high poverty schools generally attain less high growth. • Value-Added results discriminate best at the tails of the distribution. • Teachers with smaller class sizes will always fall in the ‘at expected’ category. • There is likely to be greater discrimination of results in reading than in mathematics. • Results at the tails of the distribution tend to remain relatively stable. • Use 2 and three-year averages before rushing to judgment. • Teachers with a homogeneous class of very high performing students may be less likely to attain growth at the same level of magnitude as teachers who have more heterogenous classes . • Value-Added measures alone are sufficient to determine teacher quality. (F) (T) (F) (F) (T) (T) (T*) (F)

15. Fact or Fiction Answers • Teachers from high poverty schools generally attain less high growth. This is false because there is an Inverse relationship between poverty and ACHIEVEMENT. The greater the poverty the lower the achievement levels. This is not true when we look at the relationship between poverty and growth where no clear relationship is evident. • Value-Added results discriminate best at the tails of the distribution. This is true and as such you are best to reserve judgments about teacher quality expect for those who are consistently very high or very low, • Teachers with smaller class sizes will always fall in the ‘at expected’ category. This is false. In our experience we have identified many highly effective teachers who have had relatively smaller numbers of students • There is likely to be greater discrimination of results in reading than in mathematics.This is false because there is likely to be greater discrimination of results in mathematicsthan in reading. This may be to due to the fact that reading instruction occurs across content areas. • Results at the tails of the distribution tend to remain relatively stable. This is true and as such you are likely to see those who are consistently very high or very low remain in those categories over time. • Use 2 and three-year averages before rushing to judgment. This is true and as such you are best to reserve judgments about teacher quality until clear patterns emerge. • Teachers with a homogeneous class of very high performing students may be less likely to attain the same magnitude of growth.This is true BUT you must consider the stem of this statement which overstates most classroom conditions. Statistically, if all the students in a class are extremely and consistently high achievers, a teacher may not be able to achieve as high a level of teacher effects as teachers with more typical classrooms. • Value-Added measures alone are sufficient to determine teacher quality. This is true and as such you are best to make judgments about teacher quality when using multiple and reliable measures.

16. What Building Level Leaders/Teams Should Know • Use this information to: • Pair teachers with students with whom they are most successful. • Partner teachers with other teachers who may complement their strengths. • Determine if the program is effective. • Identify students who are not making sufficient progress and design intervention plans. • Customize professional development based on student growth patterns. • Stimulate discussions during the school year about ongoing measures of student growth.

17. What Building Level Teams Should Know Value-added is actionable. When supported with diagnostic information, teachers are able to describe specific strategies used to modify instruction. Focus attention on those teachers who remain, over time, at either end of the performance spectrum—teachers who consistently score significantly below or above.

18. What Principals Should Know • The easiest and least intrusive way to leverage effective teachers is to simply tell them that they are extraordinarily good at what they do. By communicating genuine admiration and appreciation directly to these teachers, principals will be repaid many-fold. • Spend at least part of your valuable time in the classrooms of highly effective teachers. In these classrooms, you will learn a lot about leadership—great teachers are great classroom leaders—and a lot about what it is that effective teachers do to be effective.

19. What Teachers Reported They Need Opportunities for ongoing collaboration among teachers to discuss data and actionable strategies. Tools and examples of what to do next based on various patterns in value-added scores. Information gleaned from HET shared with them on an ongoing basis. Discussions that address specific questions related to the “fairness” of classroom-level value-added. To share examples of teacher/classroom data systems, such as notebooks, that track various data on individual students including value-added, achievement and formative assessment data.

20. Consider: How can the lessons welearned help you to successfully introduce teacher-level reporting?

21. Challenges As We Move Forward • Political • Cultural • Practical

22. Political Context • Heightened national attention to compensation reform may contribute to teacher consternation about the use of classroom-level reports for diagnostic purposes. • Union concerns may serve to alienate teachers from accepting new evaluation tools. • Expedient use of the information for political purposes may contribute to hard to implement policies.

23. School Cultures • The prevailing cultural norms generally view teachers as interchangeable and equal contributors to the school community • The introduction of teacher-level reports begins to differentiate teachers using quantifiable measures that have the potential to be made public • These data represent a fundamental shift in how teacher quality is measured and teachers are judged. Teachers have historically been judged not on the effects of their instruction, but on measures such as classroom management, student engagement, lesson planning and delivery.

24. School Cultures • School cultures often do not support the open and routine sharing of classroom-level reports in the service of team decision-making and goal setting. • School cultures are ill-prepared to recognize the real differences that exist among teachers. • Effective teachers are often not recognized based on their students’ growth.

25. School Cultures • Teachers are concerned that value-added information will be used to judge them unfairly and in ways that may affect their assignments, tenure, and potentially their compensation expectations. • Educators fear that value-added information at the teacher-level will foster competition for rewards and recognition . • “People are down on what they are not up on”. • Unless school leaders, teachers and the public are well-informed about what value-added results are saying and how they should and should not be used, concerns and recalcitrance will be heightened.

26. School Cultures • How do you deal with the cultural and political pressures that surround the use of these metrics? • Be aware of and responsive to the new demands and responsibilities that teacher-level value-added information brings. Consider the nature of parent demands. • Find appropriate ways to share and discuss teacher’s report with them discreetly. • Be sensitive to the impact teacher reports have on a teacher’s sense of self- worth. • Build an environment where teacher-level data can be routinely shared openly among colleagues in a manner that is safe and respectful. • Help teachers respond to the information they receive by guiding them towards reflective thinking and taking productive action.

27. Practical Context • Misunderstandings about what value-added analysis and what reports reveal • Misuse of the information • What to do with the information • The use of reliable multiple measures

28. Practical: Responsibility for Improving Practice • What should YOU be doing? What are you going to do to help me? Whose responsibility is it?

29. Practical: Responsibility for Improving Practice What do I need to do to help change practice? What do I need to get better? Whose responsibility is it?

30. What leaders must consider Value-Added results reveal if the ‘program’ has been working • Begin with your strengths • How can you leverage effective programs? • What patterns do you see? • How can you help to protect successful programs? • How can you leverage teachers’ strengths? • Who is getting strong results? • With which student achievement groups? • How can you create opportunities to facilitate discussions about what is working and why?

31. Leaders: How Have Your Decisions Influenced Teacher Effectiveness? • Time • Instructional time • Focused Collaborative Time • Assignments • Student assignments to teachers • Teacher assignments to courses • Focused professional development • Data-driven • Embedded • Focused on’ vital’ behaviors • Sustainable Value-Added results reveal if the ‘program’ has been working

32. What teachers must consider Value-Added results reveal if your ‘instruction has been working • Begin with your strengths • How can you leverage effective practices? • What patterns do you see in your data and over time? • How can you leverage your strengths? • Who is getting strong results? • With which student achievement groups? • How can you leverage the strengths of your team? • Who is getting strong results? • With which student achievement groups? • What can you learn from each other?

33. Teacher: How Have Your Decisions Influenced Your Effectiveness? • Instructional Practices • Did you Measure student progress ‘in time’? • Is there clarity of the learning; focused feedback, and focused assessments? • Instructional Arrangements • How do you use data to determine student assignments to courses and to create flexible groupings? • Focused professional development • Is it data-driven? • Embedded: collaborative time centered around student learning? • Focused on ‘vital’ behaviors? • Sustainable?

34. Improving Practice: • Top Ten FIP Practices 10. True formative instruction implementation is about deliberate, intentional practice. 9. Put learning targets first. 8. Collaborative, job-embedded learning is the way to move this work forward. 7. Students are at the heart of formative instructional practices.   6. Formative instruction is about developing assessment literate educators and students.

35. Improving Practice: • Top Ten FIP Practices 5. Formative instruction is the right initiative to focus on. 4. Formative instruction begins with the standards. 3. School and district leadership must “know and be able to do” formative instruction AND make this work a priority 2. Formative instruction is about daily teacher and student practices. 1. Formative instruction is a process—not a test.