Peer marking of simulated pharmacological experiments

1. Peer marking of simulated pharmacological experiments Ian Hughes Professor of Pharmacology Education Co-director LTSN Centre for Bioscience University of Leeds i.e.hughes@leeds.ac.uk http://bio.ltsn.leeds.ac.uk

2. Assessment Why is assessment important? • Takes a large amount of staff time • Should be integral to the learning process • Is a spur to student work • Helps to indicate progress to students • Provides contact between staff/students • Monitors student progress, identifies problems • Satisfies current grading requirements

3. Assessment Features of good assessment • Appropriate to the learning objectives and teaching methods (knowledge, skills, attitudes) • Appropriate to the individual student • ALWAYS has a formative element • Open and transparent criteria • Appeals process for those dissatisfied • Efficient and economic • Accurate, timely and reproducible • Develop self-assessment skills

4. Assessment Peer assessment • accurate • reproducible • efficient and timely • related to learning objectives • monitor student progress • provide staff-student contact • provide high quality feedback • stimulate learning • develop self-assessment abilities ?

5. Assessment Why peer assess dry-lab write-ups? • provides full explanation • requires better understanding • all get the information • develops critical evaluation • see other’s mistakes and standards • saves staff time and effort • improves learning

6. Assessment Peer assessment of dry-lab write-ups - How’s it done? • explain purpose • instructions on format; additional questions • hand-in deadline (penalty) • all in LT (350) (penalty) - distribute at random • explicit marking schedule distributed • prepared explanations + OHP; takes 50 minutes • total marks and sign (10% checked) • appeals procedure

7. Assessment Prac write-ups Peer marked 8 Mark (out of 10) 7 Acad. staff marked 6 5 Practical number 1 2 3 4

8. Assessment What are the problems? • is it reproducible and accurate? • introduction and initiation • keep silence during marking! • students don’t like it (hard work, its your job, some are unfair) • marking schedules get passed on • cheating? • scheduling; all same prac; time between prac and marking session; standard answer /data

9. Assessment What else can be peer assessed? • ‘wet’ practicals • oral communication skills • poster presentations • data interpretation exercises • information retrieval and formating • practical skills • long essays ??????

10. Assessment . Communication skills 80 . . . . Academic staff mark % . . . . . . . . . . . . . 60 . . . . . . . . y=-0.1 +1.05x r=0.88 . . . . . 40 40 60 80 Peer mark %

11. Assessment Communication skills MARKING n=44 ACADEMIC PEER Global mean 63.2+7.8 60.2+6.1 Commonality: Top quartile 11 10 mean+s.e. 77.2+4.8 74.1+5.6 Bottom quartile 11 9 mean+s.e. 48.2+3.5 44.1+3.9

12. Assessment Peer poster assessment Group A Group B Group C n=4 n=4 n=4 staff peer first poster 56+4 55+4 59+3 staff staff staff second poster 67+4 83+4 78+5 NOTE: peer process took significantly longer; small numbers; groups not isolated; 6 weeks between posters; rest of course ongoing; self selection of groups; new method effect

13. Assessment Peer assessment of long essays • Medical students • 3000-4000 words • proforma for staff and students (properly referenced, critical approach, evidence based; good presentation) • Staff mark 70.2+2.1% • Student mark 72.6+2.2% • NSD; P>0.7

14. Peer (self) mark % Academic staff mark %