Peer Evaluation of Teaching Portfolios

1. EDU Lunchtime Session Peer Evaluation of Teaching Portfolios 29 May 2000 John Jones, EDU x6320etjjones@ K.P. Kwan, EDU x6287 etkpkwan@

2. Session outline • Problems of evaluating teaching • Teaching portfolio: what it is and why it is preferred? • A suggested framework for peer evaluation of teaching portfolios • Components, criteria and evidence • Guidelines for peer evaluation of teaching portfolios • Suggestions and comments

3. Problem of evaluating teaching • A perfectly precise, valid and objective measure will never exist because • teaching is a complex activity • teaching has multiple goals • effectiveness of teaching depend on goals, teachers, students, and other contextual factors • there is no single ‘best’ way to teach but rather, different approaches to good teaching

4. Good teaching evaluation system • Must be seen to be fair, valid, and credible by staff to be evaluated • Provides incentives and feedback for staff that promote continuous improvements in teaching • Provides useful information for educational administrators for making informed decisions

5. Two things to note • Standardised, objective measure  fair and valid • Important to consider the consequential validity of the evaluation • Whether the evaluation leads to desirable consequences • Criteria for evaluation reflects the implicit goals/definition of good teaching of the institution • What and how to evaluate  teaching behaviours  effect on teaching that may or may not be beneficial to learning

6. Most indicators are imperfect • University-wide SFQ35.6% • Recommendations by HoD34.9% • Faculty-based SFQ 42.9% • Peer ratings of classroom teaching 32.1% • Peer review of learning materials 58.4% • External examiner’s report 35.5% • Measures of student learning outcomes 35.7% • Publication on teaching & learning 40.3% • Independent observer ratings 40.7% • Participation in LTD projects 42.3% • Self-constructed SFQ 49.8% Source: EDU survey (1999)

7. Portfolio: a preferred approach • Evaluation via peer assessment of teaching portfolio is preferred by most PolyU staff • 74.3% favoured “a portfolio of evidence from various sources compiled by the staff member” (Source: EDU survey, 1999) • Triangulation of evidence from multiple sources enables a more valid evaluation

8. Objectives Innovations & changes Student feedback Selection of evidence, Reflection, Action plan. Students’ work Assessmentguide Syllabus Teaching duties LTD Projects Peer feedback Personal teaching practice Teaching portfolio The teaching portfolio Generate in normal course of teaching According to specified criteria Database of evidence and examples

9. Components • A short reflective document (5-10 pages) created by the staff member which presents evidence from different sources on different aspects of his/her teaching • Common components • Teaching goals, objectives or philosophy • Approaches & strategies for teaching, advising and supervision • Evidence of teaching • preparing for teaching; interacting with studentsassessing students; other teaching-related activities • Reflection on evidence and planning for improvement

10. Providers of information • Self • Samples of learning materials developed for students • Participation in ongoing professional development, teaching development projects, innovative teaching, etc. • Students (past and present) • Feedback on teaching, supervision, consultation and learning • Peers • Feedback on curriculum, teaching, assessment, and materials • Otherparties • External examiner’s report on student performance • Employer’s satisfaction • Public recognition / awards

11. Evaluators of teaching • Self • Self evaluation of the teacher is the basis of any attempt to improve • HoD / Peers / Personnel committees • Evaluation by peers is generally more credible especially when • the peers are respected and trusted • criteria & standards are shared and clearly-defined • the process is transparent and understood by all

12. Need for a clear criteria • A clear and commonly agreed-upon set of criteria is needed: • For staff members to select evidence and produce their portfolio • For evaluators to ‘assess’ the portfolio

13. Criteria for evaluating portfolios • Examine the suggested framework and discuss: • Are the criteria for each of the components clear, reasonable, appropriate? • Are there any needs for any of the criteria to be added, modified, or discarded? • What evidence would you include/look for to support/verify the claims for having excelled in the criteria for each of the components?

14. Some issues for further discussion • To what extent can we trust what has been put down in the portfolio as true representation of competence, efforts, and commitments? • What should be the focus: • Competence or continuous improvement? • Evidence of success or commitment to teaching? • How should the various components and evidence be ‘weighed’ to arrive at the overall evaluation? • Can, and should, the evidences enable one to make fine distinctions among teachers in their teaching performance?

15. Some guidelines • Negotiate and agree on criteria and weighting within department • Use multiple judges ( 3) for important personnel decisions • Make holistic judgements in terms of broad categories, avoid numbers or ranking (e.g. outstanding, very good, basically competent, needs much improvement) • Examine consistency between components, and if there are evidences to support claims • Emphasise achievement as well as commitment; competence as well as continuous effort to improve • Consider stage of career and contexts (students, level, class size, subject nature, etc.)

16. Comments & suggestions • Send further comments and suggestions to: • John Jones: etjjones@polyu.edu.hk (ext. 6320) or • K. P. Kwan: etkpkwan@polyu.edu.hk (ext. 6287)