Using student evaluations to improve individual and department teaching qualities

1. Using student evaluations to improve individual and department teaching qualities Mary Hedges (Auckland University, NZ) Don Webber (University of the West of England, Bristol, UK)

2. Background • Student evaluations can be seen as an opportunity for students to vent their views on the quality of teaching that they receive • Some lecturers trivialise the importance of this information exchange opportunity • Doubts over the use of student evaluations per se • Doubts over the use of performance measures in general • Student evaluations and performance measures are here • Why not use the information as constructively as we can? • Can we use this information effectively?

3. Two themes of literature: • An academic job is becoming more complex and comes with more pressure • More measurements, such as • Research: RAE / ERA / PBRF / … • Teaching: QAA / AACSB / … • Critical reflection by individual teachers • Most reflective teachers consider student evaluations • Do student evaluations encourage reflective practice? (Brookfield, 1995)

4. Getting good student evaluations… • Is the student’s interpretation of what we do in class different from what we intended? (Trigwell, 2003) • Evaluations are partly about our ability to explain issues • Lower students’ expectations rather than improve performance (Edmundson, 1997) • Honesty, responsiveness, relevance, respect, openness, empowerment, … (Knight, 2002)

5. Ways of treating evaluations … • Skim read consider they’re normal file • Read take too personally believe all students a) hate me or b) like me, period. • Read identify areas of improvement / deterioration consider changes

6. Dealing with student evaluations… • Don’t do them in your own class, get an impartial and constructive colleague to distribute and collect them • BOTH of you sit down and discuss results • Incorporates peer evaluation, respect & trust • Facilitates discussion of issues, strengths, weaknesses • Use as a basis for professional development and to improve performance … but how?

7. Using evaluations to improve performance • Collect all evaluations across all papers and for each trait • The lecturer is approachable • The lecturer is organised and well prepared • The lecturer's enthusiasm helps me to learn • The lecturer helps me learn by using useful explanations and practical examples • The lecturer seeks and responds to feedback from students • Overall the lecturer is a highly effective teacher

8. Using evaluations to improve performance • Collect all evaluations across all papers and for each trait • Identify mean and standard deviations for each trait Score Trait

9. Using evaluations to improve performance • Collect all evaluations across all papers and for each trait • Identify mean and standard deviations for each trait • Identify areas of strength / weakness Individual Dept average

10. Using evaluations to improve performance • Collect all evaluations across all papers and for each trait • Identify mean and standard deviations for each trait • Identify areas of strength / weakness • Analyse relative to other Depts within Uni, or (using NES data) Depts across Unis Individual Dept Subject / Uni average

11. But also consider … • Some papers, by their nature, receive poor evaluations • Econometrics? • Some levels get particular evaluations • Level 1: is it too easy / hard? • Level 3: are we pushing them enough / too much? • Electives • Sample selection, good/poor students choose … • Servicing • Do language students like diagrams? • Different averages • Is the comparator group comprised of the same papers over time? • Timing issues • Flexibility in the ordering of papers, exogenous factors (credit crunch/finance papers), …

12. Then what …? • Head of Departments can: • Identify strengths where leadership roles could be developed and encourage star performers to guide (all?) others • Detect individuals who need further training • Expose areas where Dept could guide other Depts • Uncover areas where the whole Dept can improve • Build teaching teams based on skill mixes

13. Summary, this paper … • Illustrated how student evaluations can be used to identify: • Leadership roles • Training needs • Demonstrated how evaluations can be used to highlight areas: • Of excellence (to share across Depts / from Unis) • Of mediocrity (at risk of intervention a higher level) • Advocates greater usage of buddy schemes to share best practice to improve Dept teaching quality assessments • Paper from m.hedges@auckland.ac.nz / don.webber@uwe.ac.uk