CETL Workshop, 17 December 2007

1. CETL Workshop, 17 December 2007 S207 eTMA projects Stuart Freake*, Craig McFarlane** and Jimena Gorfinkiel*, *Department of Physics and Astronomy **S207 Associate Lecturer

2. S207 eTMA projects Stuart: Background to the S207 eTMA projects The S207 Tablet PC trial and its evaluation Craig: An AL’s experience of marking S207 eTMAs Jimena: Enabling S207 students to submit eTMAs

3. eTMAs • Use of eTMAs has grown rapidly over last few years Proportion of courses allowing eTMAs in 2008: All courses S M T eTMA expected 40 % 28 % 2 % 58 % eTMA encouraged 31 % 23 % 45 % 16 % eTMA compulsory 5 % 1 % 21 % paper compulsory 22 % 43 % 53 % 5 % mixed submission 4 % 4 %

4. eTMAs • Use of eTMAs has grown rapidly over last few years Proportion of courses allowing eTMAs in 2008: All courses S M T eTMA expected 40 % 28 % 2 % 58 % eTMA encouraged 31 % 23 % 45 % 16 % eTMA compulsory 5 % 1 % 21 % paper compulsory 22 % 43 % 53 % 5 % mixed submission 4 % 4 %

5. Science and Maths eTMAs • Many Science and Maths students could submit electronically • some students submit paper TMAs produced with a word processor, including equations and diagrams • students can scan handwritten TMA and submit electronically • Students who have used the eTMA system for previous courses may be unhappy about reverting to paper TMAs for later courses • We should allow the option of eTMA submission, if possible • But allowing eTMAs in Science and Maths requires straightforward methods for ALs to mark and annotate eTMA files, including equations and diagrams

6. Marking physics eTMAs • One focus of p CETL is e-teaching, e-learning and e-assesssment • πCETL approved eTMA marking project in early 2006 • course selected: S207 The Physical World • 60 pts, 500 students, 25 ALs • 7 TMAs – with lots of equations and diagrams • ALs provided with Tablet PCs (Toshiba Tecra M4) • enables handwritten annotation of eTMAs • ALs can mark in similar way to marking paper TMAs

7. Criteria for software to be used • Students should be able to prepare eTMA using software they find easiest • no requirement to purchase software • scanned handwritten eTMAs should be accepted • ALs should be able to use a single application for marking • application must allow straightforward insertion of equations and diagrams • no increase in AL workload • usable whatever file format student uses • Annotated file returned by AL must be easily readable by student • No loss of information – including maths content – at any stage

8. Software solution adopted • Students are allowed to submit in any file format that their tutor can open • Student’s eTMA file is converted to pdf format • using PDF Creator, freeware from http://www.pdfforge.org • AL marks and annotates the pdf file • using PDF Annotator, £25 from http://www.GRAHL-software.com • AL ‘melts’ annotation into pdf file so it cannot be changed • saves annotated script in pdf format • returns pdf file with embedded annotation to student • Student reads annotated pdf file using Acrobat Reader

9. S207 eTMA submission data • 2006: optional participation by ALs; 19 of the 23 ALs agreed to take part • Tablet PCs sent out in early April, some ALs ready to go with TMA02 • 2007: ‘eTMA encouraged’ course; all 24 ALs participated

10. S207 eTMA submission data

11. S207 eTMA submission data

12. Evaluation • Questionnaire emailed to ALs at end of each presentation, plus review of messages on AL forum In 2007 • 70% of eTMAs word processed, including equations and diagrams • 20% mainly word processed, but some equations/figures scanned • 10% hand written and then scanned • 83% of eTMAs submitted as Word .doc files • 3% in other word processor formats • 7% .pdf files • 6% image files

13. Marking procedures used in 2007 ALs fell into two groups • About half converted to pdf format, then marked with PDF Annotator • The other half marked and commented using MS Word • they only received Word files – perhaps they explicitly requested this format? • most of them used both keyboard and tablet pen entry • a couple of ALs opted not to use the Tablet PC – used a conventional computer to insert text boxes and comments • None of the ALs used Windows Journal • but they weren’t provided with information about this software

14. Problems and disadvantages • No major problems learning how to use system • Some ergonomic issues • Comfortable writing position • Viewing the screen • Some ALs took longer to mark eTMAs than paper TMAs • Difficult to get an overview or to compare scripts; difficult to mark all Q1s as a batch

15. Advantages Advantages of using the Tablet PC • Flexible annotation – can reproduce anything done on paper, plus …. • Ease of erasing or modifying or moving comments, inserting extra pages, inserting appropriate parts of model answers • Ability to store and insert frequent comments • Time taken – eTMAs quicker to mark for some ALs … plus all of the advantages of the standard eTMA system • Ease of navigation of file – can’t lose pages or get them out of order! • Boxes on PT3 forms pre-filled • Can’t add up question scores wrongly • Can write as much as you like on PT3 • Retain a copy of TMA for reference • Improved turnaround time, particularly for students outside the UK

16. AL preferences • In 2007, • 11 ALs expressed preference for marking eTMAs • 8 were either equally happy with eTMAs or paper TMAs, or expressed no preference • 4 preferred paper marking (of whom 3 had received very few eTMAs)

17. Tablet PCs for P&A courses in 2008 • S207 ALs will continue to use Tablet PCs for marking eTMAs • all ALs are now involved, all students have the option to submit eTMAs • Tablet PCs will be provided to enable ALs tutoring SMT359 Electromagnetism and S381 The Energetic Universe to mark eTMAs • participation by ALs optional in 2008, required in subsequent presentations

18. The S207 eTMA projects An AL’s experience of marking S207 eTMAs Craig McFarlane

19. S207 eTMA hardware and software • a Tablet PC – to simulate “paper marking” • PDF Creator – to convert Word files etc into pdf files • PDF Annotator – to write directly on pdf files • OU eTMA Filehandler: (i) location for downloaded files (ii) source of electronic PT3 form (iii) zips marked files for upload • a wireless router – to download eTMAs direct to PC (not essential but preferable) • A USB keyboard to enable typed input in tablet mode (optional) S207 eTMA marking needs:

20. File conversion using PDF Creator • Student can submit eTMA in a variety of file formats • AL opens student’s eTMA file and converts it to pdf format using PDF Creator

21. Marking pdf file using PDF Annotator • AL opens pdf file with PDF Annotator • marks script by writing on screen with tablet pen

22. Sample marked eTMA

23. Sample marked eTMA

24. Advantages of Tablet PC - AL viewpoint • Mimics experience of paper marking (most ALs write faster than they type)… • …but with additional advantages: • Comments can easily be deleted/revised • Extra space can be generated for extended comments • Model answers can be pasted in at appropriate place • Equations and sketches can be inserted rapidly (and copied for further use) • Time taken about the same as paper TMAs (not all ALs agree with this)

25. Disadvantages of Tablet PC • Awkward to enter text while in tablet mode, so… • …spend time toggling between tablet/keyboard modes to fill in PT3s (This can be overcome by using a USB keyboard at the cost of “ergonomic restriction”) • Have to convert most student files to pdf form • Is it a “Rolls-Royce” solution – practicable for all science courses?

26. Student comments (Although most of these comments refer to general advantages of the eTMA system, for S207 students it is the Tablet PC which has made eTMA submission possible) • “One benefit is getting the TMAs back so quickly. It really helps to get them back and realise I am doing OK. I find the system really quick and easy to use to submit the TMAs but I have had trouble getting them back on occasion.” • “I have had no problems with the system, it has been 100% reliable and the speed of the system means that I can work up to a few minutes before the deadline and be certain that it will be with the OU in time.” • “As you know I've been doing a bit of travelling this year, America first and now Afghanistan. The eTMA service for this reason has been very useful… The only down side is the time it takes to draw diagrams and type the equations out… At the beginning we were offered scanners; this in hindsight was a good idea and I should have taken it.” • “Although it did take me ages to work out how to use Equation Editor, I am glad I got the hang of it in the end.”

27. Thoughts on two years’ eTMA marking • Using the Tablet PC, it’s possible to mark eTMAs as rapidly as paper TMAs, but… • ….speed comes with practice, and practice requires that a significant fraction of TMAs arrive in this form • This is more likely to happen if the AL actively promotes the advantages of eTMA submission • Resistance to the system from ALs exists, but would be much greater in the absence of Tablet PCs

28. How do we get more students to submit eTMAs? • A positive attitude to eTMAs on the part of the AL can help, but… • Fear of MS Word Equation Editor is the biggest barrier to eTMA submission • Graphs and diagrams are an additional problem • Jimena has been running a project with S207 students to enable some of them to surmount these hurdles……

29. The S207 eTMA experience Enabling students to submit eTMAs Jimena Gorfinkiel

30. Enabling students to submit eTMAs • How to encourage electronic submission? • How to make it easier for students? • incorporate equations and figures • produce whole TMA with a single device • Ensure good quality for tutors to read and mark • Trial of electronic equipment to produce eTMAs • asked tutors to participate in trial and contact students from early April • Most students received equipment in May (in time for TMA04)and some after residential school (August)

31. Equipment • PC Notes Taker • uses special pen and clip base unit attached to USB port • write on normal paper (produce paper and electronic version at the same time) • files for each page can be collated into Word document • jpeg files of figures and equations can be created • Logitech io2 Digital Writing System • uses special pen and special notebook • write on notebook (produce paper and electronic version at the same time) • files stored in pen and saved into computer by docking pen into penholder plugged into USB port • files for each page (converted into .gif) can be collated into Word document • files of figures and equations can be created

32. Results of evaluation • 31 students requested equipment (not all of them used it) • 14 students answered the questionnaire (8 Logitech) • 5 did whole TMA and 5 used pens for equations and figure only • 7 had submitted eTMAs before (2 did all eTMA with pen, 3 did EQ/F, 2 nothing) • Those who did 2 or more eTMAs used pens for figures and equations only • 8 students were overall happy with the pens (5 Logitech) • 6 would use the pen again (5 Logitech); 3 may use it (2 Logitech). • Most students had no problems with the installation (1 couldn’t get pen to work, 1 couldn’t save files)

33. Results of evaluation Comments: • no problems mentioned with producing single document (but did they?) • low accuracy of diagrams and difficulty with drawing lines with a ruler (both pens) • saves time over using equation editor and word processing package • would prefer to type/use word processor • would prefer to scan (Logitech) • it is difficult to erase / difficult to edit a downloaded page • mistakes in written work can be erased (Logitech) • process introduces mistakes • if more time available, better use of equipment could be made • pen is resource hungry (Logitech) • fast and easy to use • used for note taking (Logitech) • conversion to text not good (not recommended)

34. Future plans • Run project again in 2008, this time starting earlier (TMA02) • Allow students more time to familiarise themselves with the equipment • Are pens really a better option than using a scanner? • Canon CanoScan LIDE 25 • write TMA normally • scan pages directly into pdf format • produce single file easily • More guidance on do’s and don’ts? • Guide students who can type to use them only for equations and figures? • More help with deleting? • More guidance on how to erase?

35. Acknowledgements • Jonathan Underwood who set up the project • 24 S207 ALs who participated • Mike Watkins (Course Manager) and Tracey Woodcraft (Course Assistant) • Rukhsana Malik, Una Lawson and Asvin Savjani (Student Services) • Kevin Mayles (Centre Manager) and Diane Ford (Centre Assistant) of p CETL • Funding from p CETL