Enhancing Undergraduate Assessment with Multimedia: Insights from Dr. Paul Rogers & Dr. Debbie Abdel-Nabi

1. Show & Tell (& Learn) for the Point & Click Generation:Multimedia assessment of undergraduate knowledge Drs Paul Rogers & Debbie Abdel-Nabi

2. Screen recording software • What does it do? • Examples – Camtasia, CamStudio, WME. • Previous uses: • Software demonstration • Essay / coursework feedback • www.russellstannard.com/ Dr Paul Rogers

3. Use in student assessment • Practical benefits • Simple to use; • more practical than viva; • Simple submission / marking process. • Educational benefits • Increased rehearsal; • Verbalisation of knowledge requires understanding; • Use of own words shown to aid comprehension. Dr Paul Rogers

4. Example 1: Statistical analysis (year 1) • Traditional method of assessment – paper copy submission of SPSS output and related questions. • Problems: • Difficult to determine depth of knowledge • Easy to copy others’ work • Many students performing very well despite subsequent evidence to contrary • Poor distribution of grades (lack of discrimination) Dr Paul Rogers

5. Example 1: Statistical analysis cont. • We wanted to provide a means of promoting answers in the students own words, i.e. wanted to recruit the truly generative processes required for oral explanation  • Better comprehension (e.g. Muller, Sharman & Reiman, 2006) • Better recall (e.g. Bekerian & Dennett, 1988; Goolkasian & Foos, 2002) • Authenticity of understanding (e.g.Zhou Qi-jia, 2006) Dr Paul Rogers

6. Example 1: Statistical analysis cont. • One solution is to use a one-to-one examination, but • Co-ordination problems (approx. 80 students per semester) • Staffing issues • Anxiety inducing situation for student • Use of screen recording Dr Paul Rogers

7. Task • Students were instructed how to create a small (5 minute) WME movie of them: • inputting data into SPSS to a specified format and • explaining why they are taking particular steps in order to perform a given analysis and; • providing a description of (i.e. interpreting) the output file • They could practice as many times as they wished before submitting file to VLE • Feedback given via VLE Dr Paul Rogers

8. Example movie Dr Paul Rogers

9. Comparison of marks Dr Paul Rogers

10. Student feedback • ‘I found this a very interesting way of establishing how much I understood about SPSS, and feel that it enabled me to engage with the material on a more practical level.’  • ‘I thought it was really good. I liked that we had chance to practice as many times as we wanted to and also hearing myself back on WME helped my to understand what I was talking about. I think it made using SPSS much easier to understand from a beginners point of view.’ • ‘Easy to use and greatly improved my understanding of SPSS. I will definitely be using WME as a revision tool in the future.’ Dr Paul Rogers

11. Example 2: Biopsychology (year 1) Dr Paul Rogers

12. Instructions • For THREE of the slides in the PowerPoint presentation, you must create a Windows Media Encoder movie containing a description (maximum 500 words each) that includes the following: • Slide 1: Functioning along the neuron (33 marks) • Your movie should: • Name the structures A to G, then; • Using the diagram, explain: • How the action potential is initiated, and; • How the action potential is propagated along the axon. Dr Paul Rogers

13. Example movie Dr Paul Rogers

14. Student feedback • ‘In written assignments I tend to ramble on sometimes and lose track of the objective but I found using this method a good way of holding on to my train of thought, although I realised that I didn't like the sound of my own voice!!’ • ‘I did not really enjoy making the movies but I have to admit it does help you to learn about the subject because you need to understand what you are actually talking about.’ • ‘I think WME is a more interesting way of completing an assignment and gaining true knowledge upon biopsychology.’ Dr Paul Rogers

15. Example 3 – Abnormal psychology (year 3) • 66 Year 3 students were instructed to use WME and PowerPoint to create and ‘informational video’ on a psychological disorder of their choice. • No difference in mean marks in comparison to previous cohort (‘informational pamphlet’ rather than movie had been required) Dr Paul Rogers

16. Example movie Dr Paul Rogers

17. Student feedback • ‘I think it was a great method of assessment. It was like doing a presentation but without the nerves factor. Most people hate presentations and lose marks because they mess up because of nerves. With WME we could re-record our video if we stumbled over words.’ • ‘Use it from 1st year! :-) It is a great 'compromise' between an essay & a presentation.’ • ‘I think that this type of assessment is much better than the ordinary written assessment, as it offers a new experience and its more enjoyable.’ Dr Paul Rogers

18. Tutor feedback (across all uses) • Perception was of increased engagement by students • Highlighted gaps in student knowledge • Exposed weaknesses in understanding • Easy to use / mark • Enabled useful feedback to be given Dr Paul Rogers

19. Future work • Longitudinal data • The degree to which this method proves, as hypothesised, to increase retention and recall of knowledge and understanding is to be assessed by tracking student progress in subsequent years. • Declarative versus procedural knowledge Dr Paul Rogers

20. Questions? Dr Paul Rogers