Satisfied – but students want more feedback: Why did I get 37%?

1. Satisfied – but students want more feedback: Why did I get 37%? Professor Brenda Smith Higher Education Academy University of Exeter Learning and Teaching Conference, May 22nd 2008

2. Aims… • Emphasis assessment for learning rather than of learning • Encourage the active engagement of students in their learning • Enhance assessment and feedback process for both staff and students

3. Road Map of the Presentation • Myths about assessment • Burgess Report – a case for change • Assessment methods – have we got it right? • NSS tables in relation to Exeter • Focus on feedback

4. John Cowan, former Director, OU, Scotland Assessment is the engine that drives learning

5. Why do we assess? 2.Quality Assurance 1.Certification 3. Learning 4. Sustainability

6. What’s wrong with assessment? The evidence: • QAA Subject Reviews • National Student Survey • Rising concern about cheating and plagiarism • Student complaints Some issues: • Too much summative and insufficient formative assessment • Insufficient effective feedback that comes too late • Variations in practice • Unscholarly practices (especially with numbers) • Myths and traditions that inhibit change

7. Myths… • Students benefit from jumping assessment hurdles unaided • Experienced tutors apply consistent assessment standards • Students don’t want feedback on exams • That it is possible to distinguish the quality of work to a precision of one percentage point • Moving back to exams will solve ‘cheating’ problems • Double marking will ensure fairness and reliability • Assessment standards are consistent across an institution

8. The Burgess Report

9. The Burgess ReportA case for change • Present system cannot capture achievement in some key areas of interest to students and employers • Emphasis on the top two degree classes wrongly reinforces an impression that a Lower Second or Third Class Degree is not an achievement • Institutional methods for calculating the degree classification could be clearer

10. The Burgess ReportA case for change • A summative system – is at odds with lifelong learning • There is a need for greater emphasis on additional information that is currently contained in the European Diploma Supplement and academic transcript (an official record of a learner’s programme of study, grades achieved and credit gained) • Institutional practices – complex and inconsistent

11. Complexity, variety and inconsistency of classification assessment regulations • (SACWG) surveyed 35 HEIs • 25 used percentage marks; eight used grades; and two used both percentages and grades • 18 institutions aggregated or averaged percentage marks to arrive at the honours degree classification; 4 used a ‘profiling’ approach; and 13 used both or either SACWG: Student Assessment and Classification Working Group

12. Complexity, variety and inconsistency ofclassification assessment regulations Variations in degree classification can be caused by: • Differing institutions’ award algorithms • ‘Dropping’ a no of modules when calculating the degree classification • A shift towards course work and away from formal examinations • Institutions diverged on their approaches to borderline performances • Variations in the weightings given to the penultimate and final years of full-time study Ref: SACWG and Mantz Yorke et al

13. Relative weightings given to results at Levels 2 and 3 Ref: Mantz Yorke et al

14. Issues involving student trajectory A single index of performance, whether it is a grade-point average or an honours degree classification, tells nothing on its own about the trajectory of a student’s development. It does not differentiate between the student who progresses steadily over the course of a programme and the student who starts slowly but in the final year produces work of an outstanding standard

15. The student voice - what they would like… For the assessment load to be spread out More advice & support I never realised that assessment was for learning Clear criteria Relevant & real-life topics Choice of assessment topics Feedback to help me learn Variety of assessment methods MESA Project

16. Department A

17. Department B

18. Department C

19. Department D

20. Assessment Methods at Exeter? University of Hertfordshire analysed assessment methods across the institution

21. Tasks Multiple choice Open book IT based Interactive Group element Role play Medium Oral Numeric Diagram/Pictorial Cognitive skills Analytic Evaluative Skills focus Primary research Theory focus Time-span Reflective Process/Periodic Portfolio University of Hertfordshire Who assesses? Self assess Peer assess Self set element Work-related Practice focus Case Study

23. National Student Survey(Full-time students UK)

27. Video Man Friday • Please watch the video and ask yourself are you always clear about the criteria you set for students? • How do you ensure that they understand what is required of them? • Could you give students practice in setting and then marking against criteria?

31. Comparison of courses within ‘x’ University in terms of students’ use of feedback

32. Everyone can make a difference.. What are the positive features of your module/course? How do you: • Identify them? • Disseminate them within departments? • Disseminate across the institution? • Disseminate externally? • Draw on effective practices in learning and teaching from external sources? • Embed that practice?

33. Dissemination 1. For awareness 2. For understanding 3. For action. By ‘action’ here we mean some change of practice inspired by, and making use of, the materials or processes produced by your initiative 4. Evaluate the success of your dissemination

34. Conditions necessary for innovations to improve Learning, Teaching & Assessment • Implement in student focused ways • Departments that value teaching • Workload that are not too high • Effective leadership of learning, teaching & assessment • Perceived recognition and rewards

35. Recommendations in relation to support, adaption & implementation • Recognise in workload allocation formulae • Recognise scholarship of teaching outcomes – publications etc • Provide small scale grants • Provide sources of institutional support and advice • Encourage development of cross institutional networks & events • For academic developers to work alongside ‘project teams’ • Ensure time release as appropriate Ref: Carrick Institute, Australia

36. Managing the Assessment process FEEDBACK TO HELP STUDENTS TO LEARN

40. Why feedback? • What kind of feedback do learners need? • How much is enough? • How often is often enough? • How can we make feedback efficient and effective? • How can we ensure feedback is taken seriously? • Should feedback be one way? • Can students give each other feedback?

41. Why give feedback? Increase motivation and interest (Give choice) Improve performance Eliminate misconceptions Develop independence Raise confidence Create trust Individualise learning Develop a learning community

42. Paradoxes of feedback • Teachers complain about it • Students ignore it • They get it too late • When they get it it’s not useful

43. Questions….. • How many hours do you spend giving feedback? • At what time in the semester do you give most feedback? • Could you give more feedback upfront? • Could feedback be spread out throughout the year? • Could you encourage peer feedback? • Could you give feedback in different ways (MP3)?

44. Staff comments on feedback and marking “They often don’t even collect their essays…..Some of them are more ready to argue about their mark than they are to collect their essay” “They came up to me afterwards…and they said well we didn’t really understand what you meant by evaluate. And I kind of thought well isn’t it obvious, but the more I think about it I wonder if it is actually obvious to students, what the word means” Ref:www.assessmentplus.net

45. Do we try and trip our students up?

46. Evaluation of tutor feedback 30% 20% 17% 0% 0% 11.5% Encouragement Feed forward 10.5% Ask for clarification Use of English Give clarification 3.1% omission 1.5% Praise -ive Error OU/SHU Improving the Effectiveness of Formative Assessment in Science

47. Do you know what your students think about feedback? A study (Carless 2006) reported a substantial difference in the perceptions of tutors and their students on feedback on assessed work A study at the University of Sydney, ran focus groups with 5,000 students to gain an in-depth understanding of student perceptions of feedback • 3 dimensions emerged • Differing perceptions (most recognisable if written) • Impact of feedback – timeliness, significance & 1st year experience (difference between University and School) • Credibility of feedback – this was related to the students’ perceptions of the lecturers themselves

48. Students comments They are not very specific when they mark. I got “some good evaluation”, what does that mean? You put so much effort into this work that you really want feedback and to know they have read it properly. If you just get a mark you think, “I’ve worked for days and that’s it?” I think it would be helpful as well if when we had our feedback we could relate it to the assessment criteria Ref:www.assessmentplus.net

50. Do these comments aid learning? A Excellent C-Fine C/C- Interesting C Opening lousy - main part OK C- Your writing is appalling D Terrible - didn’t you learn anything at school about essay writing!