Preparing for periodic review and revalidation Session 2: refreshing the curriculum

1. Preparing for periodic review and revalidationSession 2: refreshing the curriculum Sunderland University Sally Brown July 12th 2012 http://sally-brown.net Emerita Professor, Leeds Metropolitan University, Adjunct Professor, University of the Sunshine Coast, Central Queensland and James Cook University Queensland Visiting Professor: University of Plymouth and Liverpool John Moores University.

2. The purposes of this session. To... • Help you systematically review your curriculum documentation; • Encourage you to streamline curriculum descriptors, keep language simple and avoid repetition / duplication; • Enable you to think through positive changes you can make to teaching and assessment practice; • Recognise existing good practice; • Encourage appropriate innovation.

3. Variations in international approaches to teaching, learning and assessment • Teaching contexts: what kind of live and virtual spaces do we provide for lectures, tutorials, seminars, laboratory work, field work, and so on? • Cultural differences regarding citing and using others’ work, plagiarism, ‘the master’s voice’, working collaboratively, group work, and so on. • Comparative status of universities nationally and internationally: league tables, perceptions by staff, students and other stakeholders.

4. Fit-for-purpose curricula • Why: everything within the curriculum must be there for a clear purpose; • What: content should be current, relevant • How :are you using the best balance of F2f, resource-based learning, open educational resources, group work and independent study? • When: avoid periods of intense hectic activity interspersed with patches where nothing much is going on; • Who: think through who is best placed to deliver and support your curriculum.

5. Designing effective learning outcomes. They should be: • Specific, Measurable, Achievable, Realistic and Time-constrained; • Appropriate in number and level of detail; • At the right academic level: so progression can be demonstrated; • Expressed in language that students can relate to and that external examiners deem acceptable; • Reasonably robust and able to withstand external interrogation and the test of time.

6. Reviewing learning outcomes • Are assignments constructively aligned with learning outcomes? • Are your learning outcomes over-specified? Is there room for flexibility in interpretation? • Do you make space for brilliant yet unexpected outcomes? • Is the language excessively woolly? (e.g. over use of words like ‘understand’).

7. Constructive alignment (after Biggs, 2003): • Clarify what knowledge and skills your students need to be able to demonstrate by the end of the programme; • Identify content and processes for effective curriculum delivery; • Select a range of appropriate delivery means; • Identify authentic ways by which they can demonstrate this achievement; • Decide how best to evaluate how successful you have been in achieving the above.

8. Encouraging autonomous and shared learning • Only point students towards reference material at least three years old and then include an assessment criterion on using more current reference material critically; • Let students loose on source material to find their own way around it; • Welcome their findings, while encouraging them appropriately to evaluate what they find; • Acknowledge what students bring to class from their prior formal and informal learning; • Encourage them to use their peers as valuable resources from whom they can learn much.

9. The importance of assessment for learning • Assessment can be a powerful driver for learning when well designed and effectively implemented; • But “Students can escape bad teaching, they can’t escape bad assessment” (Boud, 1995); • We need good systems for the management and tracking of student assessment (e.g. Taskstream AMS); • Timely, formative feedback is crucial so students can judge the quality of their own work. (Sadler, 1998).

10. Assessment for Learning: see http://www.northumbria.ac.uk/sd/central/ar/academy/cetl_afl/ • Emphasises authenticity and complexity in the content and methods of assessment rather than reproduction of knowledge and reductive measurement. • Uses high-stakes summative assessment rigorously but sparingly rather than as the main driver for learning. • Offers students extensive opportunities to engage in the kinds of tasks that develop and demonstrate their learning, thus building their confidence and capabilities before they are summatively assessed. • Is rich in feedback derived from formal mechanisms e.g. tutor comments on assignments, student self-review logs. • Is rich in informal feedback e.g. peer review of draft writing, collaborative project work, which provides students with a continuous flow of feedback on ‘how they are doing’. • Develops students’ abilities to direct their own learning, evaluate their own progress and attainments and support the learning of others. (my emphasis)

11. Using assessment to improve engagement • Using some regular, formative computer-based assessment tasks they can undertake privately which give them feedback on why answers are right or wrong answers; • Giving them group activities where they model the real working lives of the professions/roles they are likely to enter on graduation; • Maximising the amount of guidance on the first assessed tasks with some opportunities for rehearsal and feedback before submission; • Providing information literacy training helping them locate and judge subject-relevant resources; • Monitor live and virtual engagement.

12. Good feedback practice: (after Nicol) 1. Helps clarify what good performance is (goals, criteria, expected standards); 2. Facilitates the development of self-assessment (reflection) in learning; 3. Delivers high quality information to students about their learning; 4. Encourages teacher and peer dialogue around learning; 5. Encourages positive motivational beliefs and self-esteem; 6. Provides opportunities to close the gap between current and desired performance; 7. Provides information to teachers that can be used to help shape the teaching. (Nicol and Macfarlane Dick) (my emphasis)

13. Integrating assessment into learning • Do your assignments align with learning outcomes? • Are they sensibly spaced throughout the programme? • Are they varied to allow diverse students to shine without being overwhelming in variety? • Are feedback opportunities, plentiful, timely (so students can use them) and productive? • Are you over assessing (or more rarely under-assessing?)

14. Manageability of assessment • Across a programme, have you avoided log jams for students and staff? • Have you mapped out marking time necessary (and hence costs of marking)? • Can you access the necessary rooms, equipment and software particularly for exams? • Can you comfortably meet university requirements in terms of exam boards, moderation etc?

15. Learning from Graham Gibbs • Use ‘capstone’ modules that assess higher level learning outcomes, and do this with large, complex, demanding assignments or examinations. • Increase the use of course requirements that capture and distribute student effort, without student work being marked. • Sample these requirements for marking purposes, increasing the risk to students of not taking all of them seriously. • Use frequent ‘quick and dirty’ feedback mechanisms with rapid ‘turn-round’ and discuss feedback and student work in class.

16. More advice • Use student marking exercises and exemplars to communicate standards and criteria, not ever more detailed specifications. • Reduce the variety of types of assessment so that students have some chance to come to understand, through practice and feedback, how to tackle each at a reasonably sophisticated level.

17. Differentiating knowledge and skills. It’s all in the verbs! Task: Chose from the following verbs that are likely to relate to knowledge, skills or both: Perform, demonstrate, outline, describe, show, present, make, practice, differentiate, define, argue create, design, develop effect, utilise, enact, act, carry out, differentiate, define, argue, analyse, synthesise, evaluate, articulate, specify, propose, compare, choose, challenge, identify.

18. Useful verbs: Assessing knowledge after Kennedy et al 2011 • Arrange, collect, define, describe, duplicate, enumerate, examine, • find, identify, label, list, memorise, name, order, outline, present, • quote, recall, recognise, recollect, record, recount, relate, repeat, reproduce, • show, state, tabulate, tell.

19. Assessing Comprehension • Associate, change, clarify, classify, construct, contrast, convert, decode, • defend, describe, differentiate, discriminate, discuss, distinguish, • estimate, explain, express, extend, generalise, identify, illustrate, indicate, • infer, interpret, locate, paraphrase, predict, recognise, report, • restate, rewrite, review, select, solve, translate.

20. Application • Apply, assess, calculate, change, choose, complete, compute, construct, • demonstrate, develop, discover, dramatise, employ, examine, • experiment, find, illustrate, interpret, manipulate, modify, operate, • organise, practice, predict, prepare, produce, relate, schedule, select, • show, sketch, solve, transfer, use.

21. Analysis • Analyse, appraise, arrange, break down, calculate, categorise, classify, • compare, connect, contrast, criticise, debate, deduce, determine, • differentiate, discriminate, distinguish, divide, examine, experiment, • identify, illustrate, infer, inspect, investigate, order, outline, point out, • question, relate, separate, sub-divide, test.

22. Assessing Synthesis • Argue, arrange, assemble, categorise, collect, combine, compile, compose, • construct, create, design, develop, devise, establish, explain, • formulate, generalise, generate, integrate, invent, make, manage, • modify, organise, originate, plan, prepare, propose, rearrange, reconstruct, • relate, reorganise, revise, rewrite, set up, summarise.

23. Assessing Evaluation • Appraise, ascertain, argue, assess, attach, choose, compare, conclude, • contrast, convince, criticise, decide, defend, discriminate, explain, • evaluate, grade, interpret, judge, justify, measure, predict, rate, recommend, • relate, resolve.

24. Transitions into learning and between levels • How do you use the first part of the first semester of each year to establish the standards expected, the pace required and the behaviours expected at this level? • What strategies could you employ to avoid fall-off of commitment, skills development and engagement?

25. What’s important to do in the first six week block? • Enable students to feel part of a cohort rather than a number on a list; • Help students acclimatise to the new learning context in which they find themselves; • Familiarise them with the language and culture of the subject area they are studying (Northedge, 2003); • Foster the information literacy and other skills that students will need to succeed; • Guide them on where to go for help as necessary.

26. The first six weeks: six things we can do to diminish their chances of success • Give them an exam on day one or two to work out who needs remedial help; • Avoid teaching them anything while they ‘settle in’; • Give them a group task which provides opportunities for them to make fools of themselves in front of their peers; • Load them up with masses of information/ paperwork, all of which is deemed essential; • Give them a library orientation that focuses on what they can’t do and how complicated everything is; • Avoid taking registers until the numbers have settled down.

27. The sophomore slump • It’s a long, long time, from May to September… • In former times, there were often high expectations around reading over the summer vacation; • Realistically moist students must work over this period and also want to have holidays; • Curriculum designers need to be creative about creating ways to help keep up the learning impetus.

28. What might work • Directed reading specifically linked to early tasks in the second year; • Guided wider reading around dissertation or project topics; • Guided activities within a VLE with a specific purpose; • Interactive virtual networking opportunities with a clear aim; • Computer-based learning tasks; • Written assignments…..and?

29. Enhancing lectures. The sixty minute session: a recipe for disaster • Continuous non-stop delivery from the podium (particularly if reading from a prepared script); • Picking on random students to ‘test their mettle’; • Over-use of a single style of PowerPoint delivery, with handouts that all look the same; • Each session week on week following the same format; • Tell bad/insensitive jokes; • Behave as if students aren't there.

30. What can we do in sixty minutes rather than talk flat out? • Provide three 10 minute inputs (or six four minute ones or whatever) with interactive tasks to enable them to learn by doing; • Video ourselves delivering and then edit the best bits to use in class alongside live delivery; • Use clickers, quizzes, videos, role plays, short tests, in-seat mini discussions, periods of silent reflection, questions to and from students, short reading tasks and other means of getting them to interact; • Use some classroom time to advance brief and give class feedback on assignments; • Bring material alive with passion and enthusiasm.

31. Technology enhanced learning: we can.. • Use more Computer-Based Assessment (stop marking, start assessing!), • Encourage productive course-related discussion in formal and informal social learning spaces, • Provide links to electronic readings with guidance on what to do with them, • Make better use of Open Educational Resources and Re-usable Learning Objects; • Communicate with students via mobile phones, Podcasting, SMS et al rather than just letters, university email addresses and notice boards; • Offer some level of personalised approaches to learning and assessment.

32. Constructing level outcomes • These need to be overarching guiding principles; • Focus on the broad brush: don’t over specify detail at this level; • Refer closely to subject benchmarks; • Keep in mind the question “What would we and our peers reasonably expect could be achieved by all students at this level (threshold) and what could exceptional students hope to achieve?”

33. Using your module descriptor documentation. Task: Working in pairs and using the module descriptor checklist, review each other’s documentation; • Identify three aspects that you would like to commend; • Propose up to five enhancements you would like to suggest might improve the descriptors; • Discuss with each other how many of these enhancements are practical, viable and achievable within the timescale.

34. Module descriptor checklist • Is the module accurately described? • Do students know about pre-requisites and future directions/ • Does the content sound interesting and productive? • Is your curriculum delivery well-paced? • Are your learning outcomes SMART? • Do you have sufficient/ not too many? • Is assessment fit-for-purpose, appropriate in number, designed to encourage learning?

35. Course handbook checklist • Is it available in multiple media (paper, on-line, in different formats?); • Does it confirm to agreed university formats? • Is it comprehensive without being over-whelming in its level of detail? • Is it easily navigable? (e.g. does it have a helpful contents page and index); • Does it guide students where to go for various kinds of help? (academic, personal, crisis, financial etc.).

36. Conclusions • The curriculum enhancements we make to improve the student experience need to be convincing, strategic and evidence-based; • It is possible to make significant improvements, but it needs ownership by staff at every level of the organisation; • Strategic approaches aren’t worth a fig if individual staff don’t embrace the need to improve things; • Doing the same things we have always done in the same way we have always done them is doomed to failure.

37. References & further reading Biggs, J. (2003) Teaching for Quality Learning at University, Buckingham: SRHE & Open University Press. Bristol University Characteristics of excellent teaching http://www.bris.ac.uk/esu/academicdevelopment/prizes/excellentteacher.html Brown, S. Rust, C. & Gibbs, G. (1994) Strategies for Diversifying Assessment, Oxford Centre for Staff Development. Browne, J. (2010) Securing a sustainable future for higher educationwww.independent.gov.uk/browne-report Brown, S. and Denton, S. (2010) Leading the University Beyond Bureaucracy in A practical guide to University and College management (Eds. Denton, S and Brown, S) New York and London: Routledge. Brown, S. (2012) Managing change in universities: a Sisyphean task Quality in Higher Education, Volume 18 number 1. Brown, S. (2011) Bringing about positive change in the higher education student experience: a case study, Quality Assurance in Education, Volume 19 Number 3 p 195-207. Carroll, J. and Ryan, J. (2005) Teaching International students: improving learning for allRoutledge SEDA series

38. References: 2 Kennedy, D., Hyland, A. and Ryan, N. Implementing Bologna in your institution, Using learning outcomes and competences: Writing and Using Learning Outcomes: a Practical Guide http://sss.dcu.ie/afi/docs/bologna/writing_and_using_learning_outcomes.pdf Northedge, A. (2003) Enabling participation in academic discourse Teaching in Higher Education, Vol. 8, No. 2, 2003, pp. 169–180. Newton, J. (2003) Implementing an Institution-wide learning and Teaching strategy: lessons in managing change. Studies in Higher Education Vol 28 No 4. Nicol, D. J. and Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education 31(2) 199-218. Peelo, M. and Wareham, T. (eds) (2002) Failing Students in higher education Buckingham, UK, SRHE/Open University Press. Race, P. and Pickford, R. (2007) Making Teaching work: Teaching smarter in post-compulsory education, London, Sage Race, P. et al (2009) Using peer observation to enhance teaching Leeds metropolitan Press Leeds.

39. References 3 Renfro, W. L. and Morrison, J. L. (1983) Anticipating and managing change in educational organisations, Educational Leadership Association of Supervision and Curriculum Development Roxa, T. and Martensson, K. (2009) Significant conversations and significant networks- exploring the backstage of the teaching arena Studies in Higher EducationVol 34 no 5 p547-559 Rust, C., Price, M. and O’Donovan, B. (2003). Improving students’ learning by developing their understanding of assessment criteria and processes. Assessment and Evaluation in Higher Education. 28 (2), 147-164. Ryan, J. (2000) A Guide to Teaching International Students, Oxford Centre for Staff and Learning Development. Sadler, D. R. (1989) Formative assessment and the design of instructional systems Instructional Science 18, 119-144. Yorke, M. (1999) Leaving Early: Undergraduate Non-completion in Higher Education, London: Routledge.