Comments on: “New Research on Training, Growing and Evaluating Teachers”

1. Comments on:“New Research on Training, Growing and Evaluating Teachers” 6th Annual CALDER Conference February 21, 2013

2. How Can We Improve Teacher Quality? • Improve the pool of applicants • Alternative routes into teaching • Improve traditional teacher preparation programs • Increase starting salaries • Raise entry requirements • Do a better job of selecting the best candidates • Observable characteristics generally not related to future performance • Improve the human capital of current teachers • Studies of various forms of PD/mentoring typically find no impact on student achievement • Get existing teachers to work harder/better • Nashville and similar merit-pay experiments find little/no impact on teacher performance • Get rid of low-performing teachers and hold on to high-performing teachers • Simulations suggest gains could be large; don’t not yet know how this will pan out in practice

3. Teacher Preparation Programs and Teacher Quality: Are There Real Differences Across Programs? • Short Answer: No (at least in Missouri) • Few programs stand out in other states either • Florida, Louisiana, New York, Texas • Why Aren’t There Differences? • Uniform negative selection • Education majors at more selective universities aren’t much better than those at less selective universities • Maybe teacher prep. programs don’t differ much across institutions • Or maybe they don’t matter at all

4. Do First Impressions Matter? Improvements in Early Career Teacher Effectiveness • Short Answer: Yes • Value of teacher retention/”de-selection” based on value-added (VA) depends on how well early-career VA predicts long-run performance • Although there is substantial variability in the payoff to early career experience, VA in first two years does a reasonably good job at predicting later VA, particularly if you want to identify high and low performers • Still only about 28% of variation in future VA is “explained” by VA in first two years • If dismiss teachers in lowest 10% of VA in first two years, would eliminate 30% of teachers who are later in bottom 10% of VA and none of those who are later in the top 10% based on VA

5. Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter? • Consider three models • Student Growth Percentiles • Account for differences in potential achievement solely with prior achievement • One-Step Value Added • Use both prior test scores and student demographics to account for differences in potential achievement • Simultaneously estimate impacts of student characteristics and schools/teachers by looking at within-school/within-teacher variation in student achievement (i.e. fixed effects) • Two-Step Value Added • First step estimates achievement as a function of a student’s prior scores and characteristics, plus observable school/teacher characteristics • Second step takes the difference between the actual score and the predicted score for each student in step one and compares the averages across schools/teachers

6. Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter? • Student Growth Percentiles • Advantages • Doesn’t explicitly set lower expectations for disadvantaged students • Disadvantages • If disadvantaged students score lower (conditional on prior test scores) than non-disadvantaged students, then penalizing teachers/schools who serve them

7. Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter? • One-Step Value Added • Advantages • Takes into account that some types of students will likely score lower than others, even if they have the same initial score • Disadvantages • Assumes effects of student characteristics within-school (or within-teacher) are the same as the effects across schools (or teachers) • Effects of small differences in student characteristics within schools/teachers extrapolated to effects of large differences in characteristics across schools/teachers • Likely much bigger deal for ranking schools than for ranking teachers • Measurement error a bigger problem since less true variation within schools/teachers

8. Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter? • Two-Step Value Added • Advantages • Does not penalize schools for factors outside their control • Schools serving disadvantaged students may get worse pool of teacher candidates • Disadvantages • If school/teacher quality is correlated with student characteristics, will underestimate differences in quality • Falsely attribute differences in school/teacher quality to student characteristics • If disadvantaged students tend to be assigned low-quality teachers, then end up penalizing high-quality teachers

9. Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter? • What’s the right model? • Ehler, et al. argue for the two-step VA model • Choice not so clear • SGPs appear problematic because they do not account for student characteristics (conditional on test scores) but with enough prior scores, may not be so bad • One-step VAM may over-penalize schools/teachers serving disadvantaged students • Two-step VAM may under-reward schools/teachers serving advantaged students

10. Implications • Improving the pool of potential teachers important, but tweaking traditional teacher preparation programs not likely to get us there • Need to find was to get higher quality potential teachers by lowering entry barriers, increasing incentives or (maybe) radically changing traditional teacher preparation programs • Although far from a perfect instrument, incorporating early-career value added into retention/dismissal decisions will help • While there is no single “right” value added model, states ought to be paying more attention about the consequences of selecting a particular model and consider how they intend to use the results when decided which model they choose