Aims & content of talk

1. Engaging our diverse students using assessment as learning: presentations, role play and a mobile application Peter Chalk, Dr Yanguo Jing (FoC), Peter Hodges, MazinAlabdulbaqi, (BSc Computer Science) and Fraser Hardy (MSc Mobile Computing) London Metropolitan University Learning & Teaching Conference, 28.6.11

2. Aims & content of talk • L&T activities: • presentation, role play, interview, viva , crit, performance, seminar contribution etc • How practised, described and theorised • assessment-as-learning • simulated situated learning • Example criteria, grading & feedback sheets • Student experience • Mobile app PGFT = Presentation Grading & Feedback Tool - http://goo.gl/UyFPM - pin number = 99

3. Assessment as learning • “The ultimate goal in assessment as learning is for students to acquire the skills and the habits of mind to be metacognitively aware with increasing independence. Assessment as learning focusses on the explicit fostering of students’ capacity over time to be their own best assessors, but teachers need to start by presenting and modelling external, structured opportunities for students to assess themselves.” (Manitoba, 2006)

4. Situated learning • “(1) learning is grounded in the actions of everyday situations; • (2) knowledge is acquired situationally and transfers only to similar situations; • (3) learning is the result of a social process encompassing ways of thinking, perceiving, problem solving, and interacting in addition to declarative and procedural knowledge; and • (4) learning is not separated from the world of action but exists in robust, complex, social environments made up of actors, actions, and situations.” • [Anderson, Reder, and Simon 1996; Wilson 1993 cited in http://www.games-ed.co.uk/resources/Games-Based-Situated-Learning-v1.pdf]

5. Jenny Moon - Assessing Oral Presentations • “One of the features of an oral presentation is that it does not leave a record. Unless the presentation is recorded, there is no chance for discussion of a disputed mark. For this reason, the assessment procedure must be simple so that the assessor can use it easily and fairly from the beginning. This in turn implies that the assessment should operate on the basis of very few assessment criteria – in effect those that the assessor can hold in mind during the presentation.” (Moon, undated, my emphasis)

6. Essex University Guidance 1 • http://www.essex.ac.uk/assessment/oral%20assessment_criteria.htm • “Criteria typically focus on both the content of the presentation and the delivery broken down into categories such as: • Knowledge of subject material • Ability to answer/respond to questions • Structure of presentation • Use of audio/visual material • Pace and timing • Delivery style” [Thanks to Digby for sources]

7. Essex 2 - criteria for seminar contribution • Punctual; Attentive presence • Completion of set reading • Appropriate contribution to seminar discussion • Ability to work in a group (listening, responding, collaborating with others) • Implied matrix marking system (?): • Excellent (5); Good (4); Competent/ Acceptable (3); Poor (2); Fail/ Very poor (1); No participation (0)

8. Some London Met Examples • Responses to email to staff known to be using presentations for assessment, including • work placements (2nd author) • study skills (1st author & 1st two student authors) • FDSc C level ‘Business Practices’ module • Check List • BSc H Level ‘Current Developments’ presentation

9. Work placement (H Level module) presentation (in outline) • ‘Tick the box (Very good 70%+... Very poor, 40%-) for each of the following: • Structure & flow • Visual Aids • Language – clarity and level for audience • Content – understanding of company/ tasks • Reference to learning outcomes & capabilities • Self-presentation, eg eye-contact • Handling questions • Comments......

10. Study Skills Peer Assessment (Formative) • Use the grid below as you are listening to your colleagues’ talks. Tick or cross each using the following questions: - did they stick to the point? - did you learn much from the talk? - were you interested? • Then distribute 10 points across the following criteria: - delivery/body language - content of the talk - structure of the talk • For example, if you felt content was stronger than delivery or structure you might give 5 points for content, 3 for structure and 2 for delivery.

11. BS1F01C - Business Practices - Presentation Mark Sheet • Company name: Students presenting: • Introduction to presentation out of 5 • The Current Position out of 5 • The Business out of 5 • Customer base out of 5 • The Competition out of 5 • Finishing off out of 5 • Overall comments ............... • Marks for individual presenters out of 10 • 1... 2.... 3... 4... 5....

12. Check List Approach • Was there a useful introduction? • Specific agenda given/not given? • Logical structure? • Evidence given? Discussed? • Was there a conclusion? (Main points restated? Yes/No?) • Use of appropriate AVA? • Rapport and eye contact? • Positive body language? • Use of cue sheets/cards? • Sensitive handling of Q&A? • Comments:

13. What do students get out of it? • Quotes from blogs: • “I feel the presentation was successful in the sense that we may have been able to teach our audience something new and not completely lose their attention in doing so. I definitely feel that the presentation could be improved in various areas but the experience in itself was invaluable.”

14. On interview role play... • “We were all equally impressed with the candidates and how seriously everyone interacted. It gave everyone an insight and what possible questions to expect during an interview and the different techniques and various ways to answer or interact with possible employers. It definitely gave me a whole different perspective on the interviews and techniques, I definitely know where I can improve and prepare myself more”. • ‘Metacognitively aware’ student? (Manitoba 2006)

15. Acting on feedback... • “Immediately after doing our presentation it was clear that we needed to make quite a few changes, changes that would hopefully make the presentation more fluid and relevant. [Our lecturer] also gave us advice on where we could improve, which we prioritised and pretty much implemented straight away. I also found that I needed to adjust my content and delivery to seem more like a ‘learning object’ instead of a marketing presentation.”

16. Towards a mobile app - PGFT • Why technology enhancement (blended learning)? • capturing a ‘record’ of the event (Moon, undated) • useful for summative assessment? • useful to collect and return formative feedback? • useful in ‘ask the audience’ mode? • useful for peer assessment?

17. Prototype development • Stage 1 – simulate in SurveyMonkey • Stage 2 – simulate in web page • Stage 3 – first smart phone app (iPhone) • Stage 4 – develop app functionality • Stage 5 – develop for other smart phones – Android, Windows, RIM • Throughout – evaluate in classroom settings & feedback into app development

18. SurveyMonkey mock-up • www.surveymonkey.com/s/QDFNXSC • Contains essential features: • access code • presenter name • categories • grade • feedback • But not ‘managed’

19. Mobile app design – first pass – ‘assessor’ screen

20. ‘Organiser’ screen design – set up event

21. ‘Organiser’ screen design – viewing summative results

22. ‘Administrator’ view – setting up and managing organisers

23. Conclusions • Currently up to 95% of students have a mobile device in class, and staff too – how soon before 100%? • Can we use this technology to help engage in situated learning/ assessment activities? • Will academics want this PGFT mobile app? • How can it be better designed for use? • Is it applicable across all subjects and levels? • Is it useful for summative grading, and formative feedback?

24. References • [Anderson, Reder, and Simon 1996; Wilson 1993 cited in http://www.games-ed.co.uk/resources/Games-Based-Situated-Learning-v1.pdf] • Manitoba Education, Citizenship and Youth (2006) ‘Rethinking Classroom Assessment with Purpose in Mind’, www.edu.gov.mb.ca/k12/assess/wncp/ • Moon, J (undated) Assessing Oral Presentations, www.liv.ac.uk/cepd/DOCUMENTS/AssessingOralPresentations.doc