e-Feedback Who’s using it and why? Simon Starr Canterbury Christ Church University

1. e-Feedback Who’s using it and why? Simon Starr Canterbury Christ Church University LTEU Learning and Teaching Enhancement Unit

2. Session Outline • Context • what’s the deal with feedback? • e-feedback research • Approaches to e-Feedback at Christ Church • Faculty of Business and Management • Department of Geographical and Life Science • Department of Computing • Developing University Strategy • Turnitin GradeMark • Questions & Discussion

3. Context – National Students’ Voice “Feedback: Should be for learning, not just of learning … Should relate to clear criteria … Should be legible and clear” “tutors and students have differing impressions about what is expected from assessment and how a student can achieve a high mark” “Objectives for assessment and grade criteria need to be clearly communicated to, and fully understood by, student. Subsequent feedback should be provided primarily in relation to this.” NUS Feedback Amnesty Briefing Paper http://www.nus.org.uk/en/Campaigns/Higher-Education/Assessment-feedback-/

4. Context – Local Students’ Voice “[Arts & Humanities] students were in favour of electronic feedback, yet would like to see it customised for each assignment” “[Business & Management] students found that they were often unable to read tutor’s [sic] handwriting” “[Education] students understood that electronic feedback would be easier to read and more structured” “Students highlighted that feedback would be especially useful if staff used a marking grid to show the students [sic] competence in various areas, 100% of the [Health & Social Care] Reps present agreed that typed and electronic feedback would be preferable” “[Social & Applied Sciences] students agreed that typed comments were preferable” Minutes of Student UnionFaculty Council Meeting 4th March 2009

5. Context – University Policy “Students have an entitlement to timely and constructive feedback” “The language of feedback should be accessible, constructive and relate directly to the criteria against which the assessment is conducted” “The use of statement banks and electronic templates can speed up writing feedback and contribute to the quality of feedback” Principles for Effective Feedback Accepted by Academic Board 2009/10

6. Context – Research at Liverpool John Moores • 198 L4 Pharmaceutical Science and Science students (2007) • Feedback template inc. assessment criteria and statement banks • 48 e-feedback (e), 150 traditional (t) feedback • Survey (n=169, e=40, t=129) • Avg. 0.9 Likert point higher for e-feedback compared to traditional • Amount of feedback I received: e=4.0, t=2.6 • Information on where I went wrong: e=3.8, t=2.8 • Identified aspects I did well: e=3.9, t=2.7 • Markers support further use • 2-3 mins avg. time saving/script = 2 hrs for 40 scripts • Offset against programming criteria/statement banks … • Denton, P., Madden, J., Roberts, M & Rowe, P. (2008) ‘Students’ response to traditional and computer-based formative feedback: A comparative case study’, British Journal of Educational Technology, 39 (3) pp.486-500

7. Context – Research at University of Wales • 200 L4/5/6 undergrad. criminology essays (2004/5) • Feedback template inc. assessment criteria and statement banks • Survey (n=95) • 79% more aware of assessment criteria • 69% more “able to identify ways to improve in the future” • 67% more “motivated to improve in the future” • Average 4% improvement in performance per student L1-2 • Marking time reduced, student queries reduced • Adopted by department as standard • Case, S. (2007) ‘Reconfiguring and realigning the assessment feedback processes for an undergraduate criminology degree’, Assessment & Evaluation in Higher Education, 32 (3) pp.285-299

8. @Christ Church 1: Business & Management • Module: Introduction to Financial Accounting (L4) • Assignment: Financial Report; ~400 students in 113 self-selecting groups • Marking Arrangements: 5 markers feeding back grade, summary- and criteria-related feedback • assessment grid “plugged in” • Technology: Feedback 15 software (LJMU) • e-Feedback Driver: time –vs- amount and quality of feedback

9. @Christ Church 1: Business & Management Module Lead: Nicky Leatherbarrow

10. @Christ Church 1: Business & Management Criteria-related feedback …

11. @Christ Church 1: Business & Management … enabled using Feedback 15 software

12. @Christ Church 1: Business & Management Example Feedback 15 Report

13. @ Christ Church 2: Geographical & Life Sciences • Module: Organic/Environmental Chemistry ~25 students (L5) • Module: Pollution Science ~15 students (L6) • Assignments: lab reports; research tasks; numerical problems • Marking Arrangements: single marker - criteria-related feedback, summary key points and stock phrases e.g. • "the axis in the graph should be labelled“ • "references should be given in the Harvard format“ • Technology: FormFeed (Heriot-Watt) – similar to Feedback 15 • e-Feedback Driver: try out new technology - early adopter

14. @ Christ Church 2: Geographical & Life Sciences Module Leader: Emilia Bertolo “I preferred this method to conventional marking: the time is spent more creatively … with almost no time wasted in generating generic feedback again and again.” “nice to be able to use office to write the feedback, since it catches spelling errors (as a non-native English speaker, that was great)” “modifying the marking grid to make sure I was not leaving any irrelevant feedback took me longer than what I had expected” No formal evaluation with students but asked for comments – no negative feedback … “Informally, a couple of students told me that they liked it, and another mentioned how nice it was that the feedback was typed …” “One of the external examiners really liked, he thought the feedback was very meaningful … I consider this the greatest success of the experience”

15. @Christ Church 3: Department of Computing Modules/Assignments: various Marking Arrangements: since 2002 feeding back criteria-related marks, stock phrases and overall comment, auto sum mark, e-mail reports to students Technology: In-house Excel-Word-Outlook e-mail e-Feedback Driver: R&D interest

16. @Christ Church 3: Department of Computing

17. @Christ Church 3: Department of Computing • Module Leader: Dave Bennett • Used by almost all academic staff • most use for simple addition • some made enhancements e.g. automatic comment based on grade and standard marking grid • No formal evaluation with students but … • “No major issues encountered so far…” • “[students] like getting it by email” • Paper on criteria for developing e-Feedback systems in progress …

18. Summary • Experience@Christ Church echoes research • Students like: • clarity – type-written • structure & consistency – criteria driven • “feedforward” • potential for electronic return of feedback? • Tutors like: • stock phrases to save time • criteria-related feedback to guide students’ learning • potential for electronic return of feedback? • Several technologies available – key is “plugging in” assessment grids; stock phrases; electronic return?

19. Where Next? Developing University Strategy … • Strategic assessment enhancement 2010/11: • 3 week turnaround • institutionally supported e-Submission and e-Feedback systems • Piloting Turnitin GradeMark April/May • quality of feedback: criteria-related feedback and detailed stock phrases • ease: web-based, access anywhere; intuitively programme in and share assessment grids • lesser focus on speed … • … e-submission; e-return of feedback? • GradeMark Demo • Marking exercises …?

20. Questions? • Are there any opportunities for e-feedback for your students? • What might some of the drawbacks be? • Wait a minute, can’t this be done on paper anyway?