TMA feedback: can we do better? Mirabelle Walker

1. TMA feedback: can we do better?Mirabelle Walker Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT)

2. FAST project findings (1) • Quantity and timing of feedback • Sufficient feedback is provided, often enough and in enough detail • The feedback is provided quickly enough to be useful to students • Quality of feedback • Feedback focuses on students’ performance and learning, and on actions under their control • Feedback is appropriate to the purpose of the assignment and to its criteria for success • Feedback is appropriate in relation to students’ understanding of what they are supposed to be doing • Student response to feedback • Feedback is received and attended to • Feedback is acted upon by the student Gibbs & Simpson (2004–5)

3. FAST project findings (2) What students said about their TMA feedback • Received plenty of it • Motivated by praise and encouragement • Mainly received within three weeks • Read feedback but rarely acted on it

4. FAST project findings (3)

5. FAST project findings (4)

6. Idea of ‘depth’ of comment • Depth 1 – acknowledges e.g. ‘more needed here’; ‘good’ • Depth 2 – corrects / amplifies e.g. ‘you needed to mention xxxx’; ‘a good introduction’ • Depth 3 – explains e.g. ‘you needed to mention xxxx because …’; ‘a good introduction because …’

7. FAST project findings (5)

8. FAST project findings (6) Significant weaknesses in current practice: • Too much emphasis on justifying the grade • Lack of shared understanding of assessment criteria (students & CT) • ALs good at articulating students’ weaknesses; explaining strengths problematic • Lack of holistic assessment of students’ work

9. My project • Replicate some of FAST investigations in Technology – results similar? • Courses chosen: T173, T209 & T224 • Analysis of feedback on sample TMAs • Telephone interviews with students • Follow-up action with ALs, monitors, CTs, STs as appropriate – and review

10. Progress so far • Feedback analysed on all three courses • Some results circulated to T209 & T224 ALs • Some comparisons with Science made • Telephone interviews conducted on T209 & T224 – T173 next month • Some analysis of telephone feedback done – awaiting T173 to finalise

11. Category percentages compared – Technology & Science

12. Category percentages compared – Technology courses & Science

13. Category percentages compared – Technology (not T209) & Science

14. Type of content comment compared – Technology & Science

15. Type of content comment compared – Technology courses & Science

16. Type of content commentcompared within T173

17. Depth percentages compared – Technology & Science

18. Depth percentages compared – Technology courses & Science

19. Further comparison of depths

20. Number of comments per student

21. Action taken on T209 and T224 • Document ‘Using TMA comments to good effect’ prepared for each course • Explains ideas of ‘depth’ and ‘feed forward’ • Contains course-specific examples of depth 2 & depth 3 comments • Sent out in first tutor mailing (2006) with commendation from Course Chair • Monitors briefed accordingly

22. Telephone surveys • T209 and T224 complete – carried out immediately after end of course • Examined • student’s perception of usefulness/helpfulness of feedback • whether (& how) student had used the feedback in any future TMA / the ECA / the exam • student’s preferences regarding placing of comments: on PT3, script, (pro-forma)

23. Preliminary findings (1) • Students are eager to receive their marked TMAs and do read the feedback … • … but they do not necessarily use the feedback again in the rest of the course (approx 20% said they never used it) • T209 students were likely to make more use of the feedback later in the course than T224 students • T209 students particularly mentioned using skills development feedback

24. Preliminary findings (2) • Not all students found that tackling the TMA, and the feedback they subsequently received, encouraged them to study the rest of the course • A small number of students said they were disappointed with the quality of the feedback (but some were surprisingly accepting)

25. Preliminary findings (3) • Overwhelming majority of students value comments on the script the most: • ‘where lost marks made clear’ • ‘tells me exactly where the mistake is’ • ‘very specific’ • ‘more evidenced against actual text’ • ‘easier seeing my work with comments relating to it’

26. Aside about feedback and eTMAs • We can’t assume that students read the PT3 first – or even at all • We can’t assume that students find and read a separate marking document sent back with the marked TMA • Some (most?) students find juggling documents on the screen awkward/difficult • It’s easy for tutors to place comments exactly where they apply • Turn-around times longer: symptom of a problem?

27. Action taken on T209 and T224 • T209 and T224:Students reminded to look for and read the PT3 • T209:Separate pro-forma dropped for eTMAs; tutors asked to copy and paste grids at end of questions

28. FAST project findings in Technology? Significant weaknesses in current practice: • Too much emphasis on justifying the grade • Lack of shared understanding of assessment criteria (students & CT) varied • ALs good at articulating students’ weaknesses; explaining strengths problematic • Lack of holistic assessment of students’ work

29. Good feedback in Technology? • Quantity and timing of feedback • Sufficient feedback is provided, often enough and in enough detail In general • The feedback is provided quickly enough to be useful to students In general • Quality of feedback • Feedback focuses on students’ performance and learning Too much biased towards performance on this TMA • Feedback is appropriate to the purpose of the assignment and to its criteria Purpose and criteria often implicit • Feedback is appropriate in relation to students’ understanding of what they are supposed to be doing Sometimes, but more explanations would be helpful • Student response to feedback • Feedback is received by students, and attended to Yes • Feedback is acted upon by the student Not sufficiently

30. Implications for Course Teams • Different sorts of questions and criteria elicit different types of feedback • (or) To elicit particular feedback, write the question and criteria accordingly – maybe even write the course material accordingly

31. More implications for CTs • Be explicit with ourselves and with ALs and students about what an assignment’s purpose and criteria are • Don’t assume that ALs will instinctively know what sort of feedback we’re hoping for – be explicit in the marking guide (or elsewhere)

32. Implications for Associate Lecturers • Need for shift towards emphasis on supporting student’s learning and progress through course, rather than just explaining what was wrong in this particular TMA • That implies more student-centred feedback – and more holistic feedback on PT3s • It may also imply giving as much (more?) emphasis to feedback as to marks

33. Implications for monitors • Need to shift emphasis from ‘Was the mark OK?’ towards ‘Was the feedback OK?’ • Need to encourage appropriate forms of feedback (and discourage non-appropriate ones?)

34. References • FAST presentations given at Open University, 10 February 2005 • Gibbs, G & Simpson, C (2004–5) ‘Conditions under which assessment supports students’ learning’ Learning and teaching in higher education 1(1) pp 3–31; available via http://www.open.ac.uk/science/fdtl/pub.htm

35. Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT) The Open UniversityWalton HallMilton KeynesMK7 6AA http://cetl.open.ac.uk/colmsct