NASDAP Conference 2013 Prof . Eileen Piggot-Irvine Eileen.Piggotirvine@royalroads.ca

1. NASDAP Conference 2013 Prof. Eileen Piggot-Irvine Eileen.Piggotirvine@royalroads.ca Appraisal: We Always Were Ahead of the play and the World is Just Catching Up!

2. Session outline • My own rationale for appraisal! • What’s happening in varied countries? • Barriers to effectiveness • A model for effective appraisal that ensures a ‘productive approach’ • Distilling the most important principles • Collaborative re-design of own school appraisal systems with a view to incorporating the prioritised principles

3. The Case for recognising Effective Teaching • From Looney (2012 – a superb paper): • “Students from low-income families may benefit the most from learning with very effective teachers (Nye et al., 2004; Sanders & Rivers, 1996) and those with a succession of high-quality teachers may potentially achieve the same levels as their peers from high-income families (Motoko et al., 2007). On the other hand, Sanders and Rivers (1996) found that the residual effects of learning with poor quality teachers were long lasting, and that students assigned to more effective teachers in later grades were unable to compensate for earlier gaps.” (p.440)

4. Being honest about my stand • We need great teachers and we need to know that the best teaching is happening in our schools. • In my opinion, a leader who does not want to know whether great teaching is happening, or not, is potentially robbing our children of learning.

5. USA • The US uses the term ‘evaluation’ as the description for overall appraisal activity. Approaches vary for individual states.

6. Francine Edwards, American Federation of Teachers, 2012 Keynote Address to AERA …. • In US overall, worrying trends, standardised testing, firing and moving teachers in poorly performing schools instead of looking at the context. In a low performing Ohio school, all teachers were requested to re-sit their teacher entrance exam. • 40% of teacher evaluation evidence is based on test score evidence. • Inadequate evaluation systems miss a prime opportunity to improve practice. • Evaluation (appraisal) systems need more than student achievement scores – need information on how to further improve (feedback). • An example effective system is in Toledo - based on continuous improvement, has professional standards, measures student achievement (but not only achievement).

7. From USA Researchers • “… a technically valid and reliable assessment framework will not by itself ensure the improvement of teaching… In our experience, the assessment of teacher quality fails more often because of organizational neglect than because of technical deficiencies. In particular, school districts have typically not done a good job of managing the relational and political aspects of the process.” (Donaldson & Donaldson, 2012)

8. The Move towards performance pay in The US • Widescale incentives (Teacher Incentive Fund, TIF) to implement performance pay. Extensively evaluated (Humphreys et al., 2012), but: • “… existing knowledge about how to make performance pay a transformative reform is relatively sparse.” (p.61) • General advice and long checklists of • components that must be in place was • not enough to overcome the uneven • implementation from one school to the next, • even within the same TIF grant holder.

9. Movement towards use of Value-Added measures (vam) • VAM are designed to evaluate student test score gains from one year to the next. • A cautionary tale from Darling-Hammond et al. (2012): • “Research indicates that value-added measures of student achievement tied to individual teachers should not be used for high-stakes, individual-level decisions, or comparisons across highly dissimilar schools or student populations.” (p. 14) • Extensive research has shown: • 1. Value-added models of teacher effectiveness are • inconsistent. • 2. Teachers’ value-added performance is affected • by the students assigned to them. • 3. Value-added ratings can’t disentangle the many • influences on student progress.

10. And more on VAM • Lissitz (2012), an excellent summary paper qualified with strong statistical analysis. • Lissitz is skeptical, pessimistic and advising caution about VAM.

11. BUT there is Hope and progress beyond VAM! • National Board for Professional Teaching Standards (NBPTS) – created standards that are grounded in student learning. • Interstate New Teacher Assessment and Support Consortium (INTASC) translated NBPTS for beginning teachers - adopted by over 40 states for initial teacher licensing and have become basis for evaluation that is much • more stable than VAM. • The hope is that these will extend to the • evaluation process for all teachers.

12. Hopeful systems cont. • The Measures of Effective Teaching (MET) Project - developed a number of tools (observations, video of teachers) supplemented with other artifacts of practice (lesson plans, assignments, etc.) – evidence is scored against standards of effective teaching. • Teacher Advancement Program (TAP), based on NBPTS and INTASC standards as well as the Conneticut standards-based assessment rubrics: evaluate teachers 4-6 times a year using master/mentor teachers or principals certified in a rigorous four-day training. Teachers look at and evaluate videotaped teaching episodes using the rubric, then dialogue with mentor and plan ongoing growth. Schools provide professional development, mentoring, and classroom support to help teachers meet these standards.

13. So… a mix of good and not so good exists in the US. It is highly variable, but there is hope in these latest trends.

14. UK • In the UK ‘performance management’ is the term applied to appraisal

15. In Britain a national system exists and there are direct links to salary (Bubb & Hoare, 2001). • Forrester (2011) - movement from appraisal (backward looking) to goal setting focused performance management (forward looking) with a performance related pay (PRP) component. The intent of the movement to PRP was to incentivise teachers. Forrester suggests rather that it has been a millstone.

16. Recent calls in the UK for strengthening the emphasis on student learning outcomeseg Moreland (2011): • What we should be doing, in my view, is to base judgements on the responsibility the individual teacher takes for her/his own professional development and the learning and progress made within lessons… In this way, we may begin to move away from the concept of performance management as a once a year, paper-chasing exercise and move towards a process of managing performance aimed at improving the classroom experience for both teachers and students alike. (p.24)

17. Canada - BC

18. “Teacher evaluations a hot-button topic” Geoff Johnson, The Victoria Times Colonist August 28, 2012 • “If you want to press a hot button inside or outside the hallowed halls of public education, teacher evaluation would be the one almost guaranteed to light up the "Emergency!" display on every teachers' union office desk.”

19. A Topic filled with Politician and Union Clash • British Columbia Liberal leadership candidate Kevin Falcon proposed the idea of merit pay for • educators. The B.C. Teachers’ Federation – predictably - was not in favour. • The current system gives most responsibility for teacher evaluationto principals, who are guided by language in collective agreements. • But that's not working, George Abbott, Education Minister (2011) said: • "The assessment process, I hear from most parties, • is not a satisfactory one. Most assessment is done • by principals and vice-principals and I believe • that they feel there's much room for improvement."

20. In BC - A Twin process • Teacher evaluationis a formal cyclical review (3 yearly, Christian schools 4 yearly) conducted by a principal or vice principal. The collective agreement includes well-defined procedures for this retrospective model. Evaluation generally is not seen as an opportunity for reflection and growth, but as a potential impingement on teachers’ professional competence. • Teacher assessment is a more collaborative • process that can include self-assessment, • goalsetting and planning as well as • external feedback.

21. Teacher Evaluation is almost totally summative • The guidelines for evaluation include: • 1. Evaluations shall take place: • - At the request of the teacher (T) • - When initiated by the Administrator (A) • - At a mutually agreeable time (M) • 2. The teacher shall have the opportunity to select the observation times. • 3. At least X days prior to evaluation, the teacher and the evaluator will meet and discuss evaluation criteria, timelines, etc. • 4. Reports shall focus mainly on the teacher’s area of expertise • 5. The teacher will receive a draft copy of the report and have the opportunity to propose changes prior to filing the final report • 6. The teacher will have an opportunity to submit written commentary of the final report, which will be filed along with all copies of the final report a report written by the administrator that includes a rating of either satisfactory or unsatisfactory.

22. Teacher assessment is almost totally formative • A minority of school districts in BC use a professional growth plan model as part of their assessment process. • However, this process and use of a growth plan model, is not supported by the BCTF (the union). • Because growth plans are developed in consultation with a principal and because they may link to school or district goals, they are seen to • impinge on professional autonomy. • You can’t win!!

23. Appraisal effectiveness generally • Donaldson & Donaldson (2012, p. 77-), suggest the five steps to effective systems include: • Step 1: Include teachers in designing the performance evaluation system. • Step 2: Protect opportunities to learn and grow. • Step 3: Hone principals’ and coaches’ skills at observing and consulting with teachers. • Step 4: Build time for teacher evaluation into principals’ workloads. • Step 5: Make instructional improvement a district priority. • See how this was done in NZ in Piggot-Irvine (2010).

24. NZ

25. An activity “… intended to benefit both the individual and organisation by leading to affirmation that performance expectations are being met, and to the identification of areas for improvement” • (Piggot-Irvine & Cardno, 2005, p. 12)

26. Appraisal fits within a larger performance management cycle. Performance management (a macro-descriptor) - encompasses all of the micro-processes associated with personnel. • Performance management covers the functions that begin when a staff member enters the organisation through to their exit. (Piggot-Irvine, 2003)

27. Vacancy Recruitment selection Induction Exit Interview Performance Management Mentoring Resignation Appraisal Development Discipline and Competence

28. Barriers to effectiveness overall in appraisal abound. Perillo (2006) sums up many of these when stating: • “… employee conflict and competition, the undermining of teamwork, disproportionate employee responsibility for poor performance and perceptions of bureaucratic control … poor recall of development goals, confusion and doubts about the purpose and value of the process, criticism about the time lost by participating in it …” (p.365)

29. Just one example of barriers: • Mather and Siefert (2011) think a strong focus on accountability (tighter control) has created an environment where: • “… senior managers have employed … close supervision of task content and its execution. As a result the professionals have resisted collectively and formally through unions, informally in the common rooms and individually through grievance, absenteeism, increased instrumentalism and dull compliance in the job.” (p.26) • Mather and Siefert continue - such activity displaces professional self-regulation, undermines high trust relationsand • educational autonomy.

30. Performance pay in nz • “To hold schools and teachers accountable for differences in attainment of social groups is particularly unfair and unreasonable” (Ivan Snook, in Hislop, 2012, p.1) • “… but the general policy approach continues to emphasise that it is underperforming teachers and schools that are at fault” (Hislop, 2012, p.1) • Hislop believes that the Minister’s agenda with improving the appraisal system is to link the accountability component to pay.

31. But there is great hope! • Although Forrester (2011) reminds us that: • “… little is known of actual costs let alone the extent to which performance management has contributed to ‘improvement’, ‘efficiency’ and ‘excellence’...” (p.8) • … it is my experience that appraisal can • achieve multiple advances in clarity, focus • and improved performance and relationships • as positive outcomes if a developmental and • accountability balance is maintained.

32. What is needed? what is effective? • Key elements of an effective systems • Strengthening student learning outcomes via ‘deep’ goal setting. • Establishing and maintaining trust.

34. PLUS ….. making links to enhanced student learning! Sinnemaand Robinson (2007) - appraisal role in promoting teacher inquiry focused on teaching (an input) that strengthens the student learning focus (an output). Has been a weak focus on student learning in appraisal due to: • Low priority placed on student learning or achievement nationally in statements of teaching standards; • Limited examination in appraisal of how teaching is impacting on students; • A compliance orientation predominating in appraisal with a lack of specific/challenging goals being set; and • The belief that appraisal is widely seen as an opportunity for celebration and support may prevent confronting of assumptions and requests for evidence in the process.

35. The ‘deep’ goal setting that has strongly been promoted previously by Piggot-Irvine (2003) and subsequently Piggot-Irvine and Cardno (2005), supports the strengthening of the student learning outcomes focus that Sinemma and Robinson have subsequently emphasised. • The goal setting session addresses this issue.

36. Deep goal setting should cascade from the strategic objectives of the school Cascadingalignment between school wide vision and goals, team and individual goals

37. 2013 ERO Report • In one-third of schools, the principal’s appraisal was linked to targets or strategic goals. • In 21 percent of schools, both the principals’ and the teachers’ appraisals were linked to targets or achievement goals. • Many schools did not recognise that appraisal is a key part of a systematic approach to improving the effectiveness of teaching and outcomes for students. Improvements are more likely to occur when key elements are connected and support each other. Appraisal provides an opportunity for leaders and teachers to inquire together into the impact of teaching on student learning.

38. ERO (2013) • Many of the schools that linked school and appraisal goals demonstrated consistently high performance in the six dimensions of a successful school. These included a commitment to all students achieving, and a sense of shared responsibility for student learning, as well as a systematic approach including: • using data to identify student and staff needs • setting school goals • linking staff goals to the school goals • planning and accessing related professional learning • establishing processes where teachers discuss strategies to accelerate progress • monitoring the impact on student learning to identify effective practices.

39. My Thinking! • The Appraisal Inquiry Model (AIM) • An ongoing, inclusive (strongly so with the ‘productive’ intent), situationally sensitive, teacher and student learning oriented Model.

41. A Further and Perhaps most important key feature of Effectiveness • If a national appraisal system is to proceed … it must be about teacher development leading to more effective in-class performance … must have a basis of peer review in a climate of trust …properly resourced … Brian McEntee (Listener, 9 June, 2012: p.9)

42. How Do WE Develop Trust? • -Non-defensive (non avoiding, non-controlling) openness and honesty are the underpinning values needed throughout appraisal (Argyris, 2003; Piggot-Irvine, 2012). These LEAD TO trust development. • - Openness and trust do not result from an emphasis on needing to be nice or liked (this creates avoidance). • Non-defensive strategies are easy to employ in non-contentious situations but this is not the case when there are difficult messages to be delivered, as is sometimes the case in appraisal. • In such tricky situations we often unknowingly resort to defensive behaviours including the avoidance of confronting issues.

43. Two Dominant Defensive StrategiesAvoidanceControl- mixing messages - Predetermining outcomes- Soft-selling - Leaping the Ladder of Inference- easing-in - Making decisions without checking- not revealing thinking - Offering solutions- Engaging in SLL - Over-advocating own position

44. Non-Defensive = Productive Stance • Employing: • DLL • Surfacing underlying issues and dealing with them • Reflection in Action (rather than just on Action) • Reflecting and responding during rather than after action • Staying low on the Ladder of Inference • Withholding judgments until after dialogue • Engaging in ‘Dialogue’ based conversations

45. A Productive Stance – Dialogue Steps • Advocacy • Stating the issues/recommendations, backed by evidence • Inquiry • Inviting challenge of position, views, evidence • Checking what views others hold • Receiving others’ views non-defensively • Summarizing mutual understanding of issues • Gaining Internal Commitment • Seeking bi or multi-lateral (genuine two or multi-sided) solutions • Monitoring • Taking joint responsibility for planning, implementation and monitoring

46. Distilling recurring Principles • Group Task: • From what that you have heard that is happening in in varied countries, collectively brainstorm what would you suggest are the 8 most important principles (eg high ownership by appraisees) that need to be present for an appraisal system to be effective. • Next, prioritise the principles in terms of how much they would influence appraisal effectiveness, with the first in the list being most influential. • Record the prioritised list on the large sheets of paper provided.

47. A Little more about the top priorities

48. re-design of own school appraisal systems • Now that you have a reasonably clear idea about what is important for effectiveness, its time to think about how to apply/incorporate these principles into your own school appraisal system. • Complete the following activities in your group: • Create a model that would fit your own school. • Outline the values that underpin the model. • Describe the key elements (e.g. 2 sentences per element) • You will be presenting your model to the whole group.

49. References • Bubb, S. & Hoare, P. (2001). Performance management: Monitoring teachers in the primary school. London: David Fulton. • Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation. Phi Delta Kappan, 93(6), 8-15. • Donaldson, M..L., & Donaldson, G.A. (2012). Strengthening Teacher Evaluation: What District Leaders Can Do. To cultivate high-quality teaching, school leaders need to focus on the human side of teacher assessment. Educational Leadership, May, 78-82. • Education Review Office (2013). Board Employment Responsibilities: Linking Charter Targets to Appraisal in Primary Schools (February 2013), http://www.ero.govt.nz/National-Reports/Board-Employment-Responsibilities-Linking-Charter-Targets-to-Appraisal-in-Primary-Schools-February-2013/Overview • Forrester, G. (2011). Performance management in education: milestone or millstone? Management in Education 25(1), 5–9. DOI: 10.1177/0892020610383902 • Hislop, D. (2012). Is performance pay just over the horizon? Education Today, 2, 1. • Humphrey, D.C., Gallagher, H.A., Yee, K.M., Gross, K., Campbell, A.Z., Cassidy, L.J., & Mitchell, N.M. (2012). Teacher Incentive Fund: First Implementation Report. U.S. Department of Education Office of Planning, Evaluation and Policy Development Policy and Program Studies Service. • Lissitz, R.W. (2012). The Evaluation of Teacher and School Effectiveness Using Growth Models and Value Added Modeling: • Hope Versus Reality. AERA Presentation, Vancouver, May. • Looney, J. (2011) Developing High-Quality Teachers: Teacher evaluation for improvement European Journal of Education, 46(4), 440-455. • Mather, K. & Siefert, R. (2011). Teacher, lecturer or labourer? Performance management issues in education. Management in Education 25(1), 26-31. DOI: 10.1177/0892020610383902 • Moreland, J. (2011). Managing performance for effective classrooms. Management in Education 25(1), 21-25. DOI: 10.1177/0892020610383902 • Perillo, S. (2006). Practice enhancement: Optimising teaching performance in schools. International Journal of Educational Management, 20(5), 365-379. • Piggot-Irvine, E. (2003). Key features of appraisal effectiveness. International Journal of Educational Management,17(4), 170-178. • Piggot-Irvine, E. & Cardno, C. (2005). Appraising performance productively: Integrating accountability and development. Auckland: Eversleigh Publishing. • Piggot-Irvine, E. (2010). One school’s approach to overcoming resistance and improving appraisal: Organizational learning in action. Educational Management Administration & Leadership 38(2) 229–245. • Piggot-Irvine, E. (2010). Confronting evaluation blindness: Evidence of influence of action science based feedback. American Journal of Evaluation, 31(3), 314-325, DOI: 10.1177/1098214010369251. • Piggot-Irvine, E. (2012). Creating authentic collaboration: A central feature of effectiveness. In O. Zuber-Skerritt, Action research for sustainable development in a turbulent world (pp.89-107). Bingley, UK: Emerald. • Sinemma, C.E.L. & Robinson, V.M.J. (2007). The leadership of teaching and learning: Implications for teacher evaluation. Leadership and Policy in Schools,6, 319-343.