Engaging staff and students

1. Engaging staff and students • Facilitators: Sally Brown, Brenda Smith and Erica Morris • 26 November 2013

2. Objectives of this session • To promote exchange, collaboration and problem solving between participating institutions in relation to managing assessment change and evaluation. • To consider ways in which staff and students can be fully engaged with ensuring assessment is integrated with learning. • To discuss how institutional change agents can work with students and staff to refocus energies on productive assessment change strategies. • To maximise the opportunity for institutions to learn from each other’s ideas, methods and achievements. • To enable participants to plan changes that foster engagement.

3. Setting the scene • Assessment issues are currently foregrounded in the light of movements towards GPAs and discussions at GCSE and A level about grading, marking and standards; • The work of the ‘Marked improvement’ project aligns well with the new QAA chapter B6 on assessment and APL (unsurprisingly as there was significant overlap between this group and the QAA working group); • Engagement issues are similarly much discussed currently, with a recognition that both staff and students need to be fully engaged for there to be effective dialogic engagement.

4. UK Quality Code for Higher EducationPart B: Assuring and enhancing academic quality Chapter B6: Assessment of students and the recognition of prior learning The Indicators of Sound Practice

5. The basis for effective assessment (1) • Indicator 1 • Higher education providers operate effective policies, regulations and processes which ensure that the academic standard for each award of credit or a qualification is rigorously set and maintained at the appropriate level, and that student performance is equitably judged against this standard. • Indicator 2 • Assessment policies, regulations and processes, including those for the recognition of prior learning, are explicit, transparent and accessible to all intended audiences. • Indicator 3 • Those who might be eligible for the recognition of prior learning are made aware of the opportunities available, and are supported throughout the process of application and assessment for recognition.

6. Thebasisforeffectiveassessment (2) Indicator 4 Higher education providers assure themselves that everyone involved in the assessment of student work, including prior learning, and associated assessment processes is competent to undertake their roles and responsibilities. Indicator 5 Assessment and feedback practices are informed by reflection, consideration of professional practice, and subject-specific and educational scholarship.

7. Developingassessmentliteracy • Indicator 6 • Staff and students engage in dialogue to promote a shared understanding of the basis on which academic judgements are made. • Indicator 6 • Staff and students engage in dialogue to promote a shared understanding of the basis on which academic judgements are made. • Indicator 7 • Students are provided with opportunities to develop an understanding of, and the necessary skills to demonstrate, good academic practice.

8. Designingassessment Indicator 8 The volume, timing and nature of assessment enable students to demonstrate the extent to which they have achieved the intended learning outcomes. Indicator 9 Feedback on assessment is timely, constructive and developmental. Indicator 10 Through inclusive design wherever possible, and through individual reasonable adjustments wherever required, assessment tasks provide every student with an equal opportunity to demonstrate their achievement.

9. Conductingassessment • Indicator 11 • Assessment is carried out securely. • Indicator 12 • Degree-awarding bodies assure themselves that the standards of their awards are not compromised as a result of conducting assessment in a language other than English.

10. Markingandmoderation • Indicator 13 • Processes for marking assessments and for moderating marks are clearly articulated and consistently operated by those involved in the assessment process. • Indicator 14 • Higher education providers operate processes for preventing, identifying, investigating and responding to unacceptable academic practice.

11. Examinationboardsandassessmentpanels • Indicator 15 • Degree-awarding bodies specify clearly the membership, procedures, powers and accountability of examination boards and assessment panels, including those dealing with the recognition of prior learning; this information is available to all members of such boards. • Indicator 16 • Boards of examiners/assessment panels apply fairly and consistently regulations for progression within, and transfer between, programmes and for the award of credits and qualifications. • Indicator 17 • The decisions of examination boards and assessment panels are recorded accurately, and communicated to students promptly and in accordance with stated timescales.

12. Enhancementofassessmentprocesses Indicator 18 Degree-awarding bodies systematically evaluate and enhance their assessment policies, regulations and processes.

13. Discussion Which of the areas identified in the QAA chapter B6 do you think will impact most on engagement practices at your university?

14. Task: • To review three student profiles and three staff profiles, and in each case to: • Decide what is the key issue; • Discuss how common this issue is; • Propose some ways of supporting deeper engagement with the implied assessment issues by staff and students.

15. Engaging students with assessment and feedback

16. Some suggestions…. Encourage students to study throughout the semester

17. Figure 1: A student’s estimate of his weekly study effort (in hours) on a programme with no assignments and an exam in week 12 Total study effort = 68 hours Ref: Using assessment to support student learning (2010)By Graham Gibbs ISBN 978-1-907240-06-5

18. Figure 2: A student’s estimate of her weekly study effort (in hours) on a programme with three assignments due in weeks 4, 7 and 10 and an exam in week 12. Total study effort = 78 hours Ref: Using assessment to support student learning (2010)By Graham Gibbs ISBN 978-1-907240-06-5

19. Help your students to understand the criteria Get them to rewrite the criteria in their own words? Why not get your students to develop their own assignment brief and criteria? Students why don’t you ask if you can develop your own assignment brief?

20. All I get is a grade next to my matriculation number without any indication on how I am actually doing I got an essay back where the only comment was ‘use a bigger text size’, there was nothing on how to improve my grade For a rather lengthy scientific report, the feedback I received consisted of a mere two ticks and a question mark I have never received feedback for any of my exams and this means I don’t know how to improve for the third year when the marks really count Don’t use bullet points!

21. Thinking about the comments on the last slide Do they relate to the assessment criteria?Do they help students to learn?Do they engage students in dialogue? If we are to actively engage both staff and students more dialogue is required

22. Principles

23. Necessary dialogue and relationships? Assessment design & development of explicit criteria Active engagement with feedback Active engagement with criteria Completion and submission of work Tutor discussion of criteria Staff Assessment guidance to staff Marking and moderation Explicit Criteria Students (Rust et al, 2005) (Rust et al, 2005)

24. Help students to understand what assessment is all about • Induct students into assessment, feedback and the standards required • Explore their expectations – do they differ from school, work or their overseas experiences? • Get to know your students – if they value you they will value your feedback • Where will this take place in your course and can you plan this as a programme team?

25. Help students to understand the standards required • Divide students into small groups • Give each group one piece of assessed work covering an excellent, good, average and failed piece of work • Get them to grade the work and explain why they gave that grade • Ask them to write appropriate feedback comments • Exchange the comments with a different group and ask them to comment on the feedback. Is it helpful, would they know what to do differently, does it feed forward etc?

26. Help students develop the capacity to use feedback effectively In small groups get students to exchange their assignments and compare the comments received Do they understand the comments, if not get them to discuss what they think they mean and how they would respond to them? What would they do differently next time if they redid the same assignment and applied the learning to a new and different assignment? • Return marked work back to the individual but with no grade or feedback • Get them to mark their own work • Collect in their grades (Pass a sheet round) • 9 out of 10 students will be within 5% • Arrange to talk to those where the difference is more than one grade

28. Involve the studentsFeedback as a dialogue For Student Completion These are areas of my work that I think are good for the following reasons Please comment on the following areas of work What I want to improve or do differently next time The mark I think this piece of work deserves is For Staff Completion

29. Integrating feedback in the curriculum • Which of your references gave you the best information? • Which part of the assignment do you feel less confident about? • Give an example of how you could apply what you have learnt to the professional world? • What is the most important thing you have learnt from doing this assignment? • What was the most challenging part of this assignment? • How long did you spend editing and improving this assignment? • If you had more time, what is the most important single thing you would have done? • What advice would you give to another student if they were to do this work? • Ref: P Race 2009

30. What is the one thing you might do differently to engage students with assessment and feedback?

31. Engaging staff Ways of working at institutional, faculty, subjects group and individual levels

32. Bringing staff along with you Trowler suggested that staff’s responses too top down change directives may include ‘compliance (both enthusiastic and reluctant, with resistance, coping strategies and with attempts to reconstruct the policy during the implementation phase’, (Trowler, 1998, p. 153). For this reason, ‘the perceived profitability of an innovation for those charged with implementing it must be clear and apparent’.

33. Engaging staff in the change process “A great deal of effort needs to be put into understanding the current status and previous history of the organisation, and allocation of resources (people as well as money) to make such changes meaningful, taking full account of institutional cultures and contexts” (Kezar & Eckel, 2002). “To effect systematic change in higher education requires a sophisticated blend of management, collegiality and simple hard work over a prolonged period of time” Robertson, Robins and Cox (2009) argue.

34. Ways of working with people Trowler (1998, p. 152) described ‘the difficulty of shining visionary light from the top in large, complex institutions like universities’ and argues that a precondition for effective change in universities is to understand the multiple cultures within universities and to: ‘Conceptualise organisations as open systems, and cultural configurations within them as multiple complex and shifting’ (Trowler, 1998, p. 150). He further argued that: ‘The pre-existing values and attitudes of staff, both academics and others, need to be understood and addressed when considering change. Individuals and groups are far from ‘empty-headed’, especially those in universities.’ (Trowler, 1998, p. 151)

35. The importance of engendering trust To convince staff that change is necessary and achievable, it is essential to generate trust that managers understand the context and are well informed about possible solutions. Harvey (2005) suggested that: “Little progress is likely within the current external quality monitoring regime unless there is a radical shift to an integrated process of trust that prioritises improvement of learning” (Harvey, 2005, p. 274). ‘Institutional strategic choice and decision making should ideally come from all members of the university community, having, of course consulted appropriately outside’. Watson (2010, in Brown and Denton)

36. Engaging staff is important “In my opinion, ‘managing the future’ on the part of any university senior management team involves: understanding the present and the past condition of your institution, getting the resources right, so that there is a zone of freedom of action in which to operate, understanding the terms of trade of the business, especially its peculiar competitively cooperative nature, helping to identify a positive direction of travel for the institution, engaging progressively with that direction of travel (through what Peter Singer describes as an ‘ethical journey’) and optimistically trusting the instincts of the academic community (of students as well as staff) operating at its best.” (Watson 2010).

37. Engaging Advocates at grass-roots level can help to effect change • Use internal advocates across the organisation to promulgate institutional aims at a local level • Enable them to build on positive outcomes where an innovation or a different approach has been used to good effect by showing others what worked for them;

38. Realism about engagement • Changes in pedagogic practice will not happen in a vacuum; • Nothing will happen if directives are issued but no one involved in implementing them sees the point of compliance: academics are experts at passive resistance; • It’s important to avoid change fatigue’ where new initiatives are introduced with alarming frequency! • Maintaining the momentum for change requires both passionate and committed leadership and cross-institutional agreement on vision.

39. Making things happen • If leaders are to engage staff in institutional change processes it is necessary to: • Identify for all stakeholders what are the purposes of making changes: what is transparent to leaders may not be apparent to the individuals charged with making the changes; • Clarify mutual expectations, so all concerned know what is required of them; • Recognise that most academics have the well-being of the university and the students learning there at heart, so clarify the benefits for all concerned of changes.

40. Strategies to encourage teams to engage in enhancing students’ learning experiences • Regularly access institutional and course level data on issues such as recruitment, retention, student achievement and student satisfaction to identify where change needs to happen; • Share this data and encourage staff concerned to identify areas for local and or university-wide improvements and direction of travel; • Use the committee structure of an institution to publicise and gain consensus for change; • Build learning communities across the HEI; • Publish clear annual priorities and targets aligned fully with the HEI’s overall plan and monitor outcomes carefully and rigorously.

41. Dealing with difficult people • Direct confrontation rarely works: it’s better to appeal to people’s better instincts; • Cooperative and dialogic discussions can convince people to work with you, so invest time in talking to people informally over coffee as well as formally in meetings; • Nothing convinces like sound, evidence-based practice, so review the literature on the issue in as thorough a manner as you would apply to your research; • Model the practices you are advocating yourself.

42. Task: strategies for working at institutional, faculty, subject group and individual level • What are the barriers to engagement? • What kinds of levers and tools do you have at your disposal? • What can hinder you? • Who can help you? • How will you know when you’ve been successful?

43. Plenary What would you do differently after today’s discussion? action planning (BS)

44. Highlights and closing comments • Dialogue about dialogue: staff and student • Lever for change • Chapter B6 of the Quality Code • Student diversity (and staff) • Learning and development preferences • e.g. approaches and methods for student feedback • Group and team working, collaborative learning • Work integrated assessment • Use of social media • Twitter • Finding out, asking questions • Tools for dialogue about dialogue • Guidelines for e-assessment formats • Use of timelines

45. Useful references: 1 Brown, S. (2012) Managing change in universities: a Sisyphean task? Quality in Higher Education, Vol18 No 1 pp.139-46. Brown, S. and Denton, S. (2010) Leading the University Beyond Bureaucracy in A practical guide to University and College management (Eds. Denton, S. and Brown, S.) New York and London: Routledge. Brown, S. (2011) Bringing about positive change in higher education; a case study Quality Assurance in EducationVol 19 No 3 pp.195-207. Cuthbert, R. (2002) Constructive alignment in the world of institutional management, presentation at the Imaginative Curriculum symposium, York: Higher Education Academy, available at http://www.heacademy.ac.uk/resources/detail/resource_database/id170_constructive_alignment_in_the_world, (accessed 6 April 2013). • Harvey, L. (2005) A history and critique of quality evaluation in the United Kingdom, Quality Assurance in Education, 13(4) pp.263–76. • Henkel, M. (2000) Academic Identities and Policy Change in Higher Education, Higher Education Policy 46.

46. Useful references: 2 • Holt, D., Palmer, S. and Challis, D. (2011), Changing perspectives: teaching and learning centres’ strategic contributions to academic development in Australian higher education, International Journal for Academic Development 16(1), pp.5–17. • Jones, J. (2010) Building pedagogic excellence: learning and teaching fellowships within communities of practice at the university of Brighton, in Innovations in Education and Teaching International vol 47 No 3 p 271-82. • Kezar, A. and Eckel, P. (2002) The effect of institutional culture on change strategies in higher education: universal principles or culturally responsive concepts?, Journal of Higher Education, 73(4) pp. 435-60. • Lueddeke, G. (1999), Toward a constructivist framework for guiding change and innovation in higher education, Journal of Higher Education, 70(3), pp. 235-60. • Marshall, P. and Massy, W. (2010) Managing in turbulent times, in Forum for the Future of Higher Education, papers from the 2009 Aspen symposium, Cambridge USA: Massachusetts Institute of Technology.

47. Useful references: 3 • Milliken, J. and Colohan, G. (2000) Managing change in Higher Education: assessing staff perceptions of the impact of semesterization, Higher Education in Europe Vol XXV No 4. • Newton, J. (2003) Implementing an Institution-wide learning and Teaching strategy: lessons in managing change, Studies in Higher Education Vol 28 No 4. • Renfro, W. L. and Morrison, J. L. (1983) Anticipating and managing change in educational organisations, Beaufort, Southern Carolina: Educational Leadership, Association of Supervision and Curriculum Development. • Robertson, C., Robins, A. and Cox, R. (2009), Co-constructing an academic community ethos – challenging culture and managing change in higher education: a case study undertaken over two years, Management in Education 23(1), pp32-40. • Roxa, T. and Martensson, K. (2009) Significant conversations and significant networks –exploring the backstage of the teaching arena, Studies in Higher Education Vol 34 no 5 p547-559.

48. Useful references: 4 • Scott, P. (2004) Change matters: making a difference in higher education, keynote given at the European Universities Association Leadership Forum in Dublin, available at http://www.uws.edu.au/data/assets/pdf_file/0007/6892/AUQF_04_Paper_Scott.pdf, (accessed 6 April 2013). • Trowler, P. (1998) Academics Responding to Change: New higher education frameworks and academic cultures, Buckingham: SRHE and Open University Press. • Watson, D. (2010) Epilogue, in Kubler, J. and Sayers, N., Higher Education Futures: Key themes and implications for leadership and management, London: Learning Foundation for Higher Education, Series 2, Publication 4.1.