Transforming assessment for learning in a digital age

1. Transforming assessment for learning in a digital age David Boud, facilitated by Mark Russell Joint Information Systems Committee Supporting education and research

2. Presenter David Boud is Professor of Adult Education in the Faculty of Arts and Social Sciences at the University of Technology Sydney. Facilitator Mark Russell is Deputy Director of the Blended Learning Unit at the University of Hertfordshire Joint Information Systems Committee

3. Session practice Text-chatting Elluminate layout Audio Whiteboard Technical problems • Use the text-chat to engage with other delegates, presenter and moderators. • You can send private text-chat messages e.g. to moderators or to individuals. • You can change your Elluminate layout to “Wide layout” to make it easier to follow the text-chat (select “View … Layouts…Wide layout”). • If you are distracted by the text-chat, you can “unlock” the Elluminate layout to enable you to adjust the size and position of the text-chat sub-window (uncheck “View…Layouts …Layout locked”) • It is best to run the Audio Set-up Wizard to test your audio set-up each time you enter an Elluminate room (select “Tools…Audio… Audio setup wizard). • You must use a headset/microphone if you want to ask a question in audio. • Only use your microphone when guided by a moderator – click on the mic icon (bottom-left of screen) to turn it on and click on it again to turn it off. • Only draw on the whiteboard if guided by a moderator. • Send a private text-chat message to “moderators” and they will try to help. Joint Information Systems Committee

4. Transforming assessment for learning in a digital age David Boud University of Technology, Sydney http://www.davidboud.com http://tinyurl.com/jiscdavidboud http://www.assessmentfutures.com Other picture

5. Pre-conference activities and reading • Assessment 2020: • Seven Propositions for Assessment Reform in Higher Education • JISC Guide • Effective Assessment in a Digital Age Activities • Drawing from the JISC Guide, what is the most strategic form of digitally enabled assessment needed to foster learning? Why this one? • Drawing on the Assessment 2020 propositions, what is the single greatest contribution that digital technology can make to this reform agenda? How can it best be made? • http://www.jisc.ac.uk/digiassess • http://tinyurl.com/assessment2020

6. Where am I coming from? • Assessment is the single greatest influence on learning for good and ill • The digital environment has been frequently misused with poor forms of assessment. Nevertheless, it still offers huge potential to transform learning experiences • We need to be clear about how assessment influences learning and how it is experienced • What counts is not the technology, but what it is used for?

7. Overview of presentation • We haven’t yet transformed assessment for a pre-digital age • The trap is to use the digital environment to lock in a primitive view of assessment • Reclaiming learning as a key purpose of assessment, but in a new way • Assessment for longer term learning • Positive and negative affordances of digital technology for assessment for learning • Some key sites for development: feedback, self-assessment, collaboration

8. I. We haven’t yet transformed assessment for a pre-digital age • Let us not pretend that assessment in everyday situations is satisfactory • Often messy compromises between assessment for certification and learning

9. I. We haven’t yet transformed assessment for a pre-digital age • Let us not pretend that assessment in everyday situations is satisfactory • Often messy compromises between assessment for certification and learning Changing thinking about assessment is needed: • Assessment isn’t just about formal structured assessment activities, it is about everything that enables us (teachers and learners) to make judgements about learning • Whatever else it does, assessment should fundamentally be about building learners’ capacity to make informed judgements about their work. It always needs to do double duty—for immediate purposes and for the longer-term

10. What is the worst feature of e-assessment we need to avoid? A Excessive use of close-ended questions B Easy to over-assess with relentless testing C Students work in isolation from each other D Easy to remove tutor involvement E Lack of personal explanations • Select your response from the buttons (A B C D E) at the bottom-right of the list of participants • Type any additional comments into the text-chat

11. II. The digital environment can lock in a primitive view of assessment • Need to avoid reverting to primitive ideas about assessment when faced with an unfamiliar environment • Assessment in general may be traped in an earlier era of measurement, norms, testing, etc. • eAssessment is sometimes discussed as if it were a different beast, many papers on the topic ignore basic assessment and learning issues and non-e literature • Need to consider the human dimension—how assessment shapes students’ actions—not just more efficient testing • Students must have an active role. This become more important on the web as it is so easy to remove this.

12. Question What is the greatest untapped potential of the digital environment for assessment? • Type your responses into the text-chat

13. III. Reclaiming learning as a key purpose of assessment, but in a new way • Need to shift conceptions of assessment towards longer term learning • Assessment isn’t just about formal assessment for certification purposes • Greatest potential of digital technology is for assessment and feedback to be easily be embedded in all learning activities • eg. use of clickers in lectures and new cheaper technology for this purpose

14. Key feature of quality assessment • Addresses need for information to guide learning when needed and in a way needed • need to distinguish guidance from simply what students want and when they want it • Always promotes high quality study activities • and discourages cramming for tests • Assessment completely aligned with pedagogy and purpose • constructive alignment but with a long-term learning view

15. IV. Assessment for longer term learning Key features: • Sustainable • Develops informed judgement • Constructs reflexive learners • Forms the becoming practitioner (in each of the following slides you can vote for which topic you would like to explore in more depth)

16. Feature 1. Sustainable • Looks beyond the immediate content • To what is required beyond the end of the course • Avoids creating dependency • e.g. through pleasing the lecturer, or looking to them for judgement • Focuses on higher-order knowledge and skills in context • Memorising as such is not tested because doing so fosters bad habits • Select which topic you would like to explore in more depth – using the buttons (A B C)

17. Feature 2. Develops informed judgement • Students must develop the capacity to make judgments about their own learning • Otherwise they cannot be effective learners now or in the future • Assessment is about informing students’ own judgements as well as making judgements on their work • Summative assessment alone is to risky and does not equip students for new challenges. Assessment is more important than grading • Opportunities for developing informed judgement need to be staged across a program • Working across subjects/modules is essential • Select which topic you would like to explore in more depth – using the buttons (A B C)

18. Feature 3. Constructs reflexive learners • Students must necessarily be involved in assessment • Assessment is a key influence in their formation and they are active subjects • Positions students to see themselves as learners who are pro-active and generative • Such learners take responsibility for driving learning • Focus on fostering reflexivity and self-regulation through every aspect of a course • Not just assessment tasks. • Select which topic you would like to explore in more depth – using the buttons (A B C)

19. Feature 4. Forms the becoming practitioner • Assessment to help calibrate judgement • Learners act on their belief in their own judgements; if these are flawed it is more serious than particular knowledge gaps • Develops confidence and skills to manage their own learning and assessment • Understanding is not enough • Develops the capacity to work effectively with others to assist learning and mutually develop informed judgement • Assessment in work settings occurs with and for others • Select which topic you would like to explore in more depth – using the buttons (A B C)

20. Exploring these ideas further Examples of different kinds of assessment practice to foster learning in the longer term: http://www.assessmentfutures.com

21. Assessmentfutures splash http://www.assessmentfutures.com

22. Question What are the most challenging aspects of this view of assessment for your practice? • Type your responses into the text-chat

23. V. Affordances of digital technology for assessment for learning Learning involves students being pro-active and doingrather than listening/reading • responsiveness • extends repertoire of ways of responding to variety of students and variety of student needs • user-control • purpose, timing, involvement • collaboration • working with others doesn’t need a particular physical environment • multi-media and multi-modal

24. Negative affordances What do we need to avoid? • Lots of things are easy to do, but don’t transform assessment and don’t support learning • More testing or self-testing • More compliance and control • More dependency

25. Positive affordances • Always there, never tired • Databases can be used to support assessment tasks • Can involve others easily—they don’t have to be physically present • Keeps good records (for students as well as teachers)

26. Traps • Teachers dominate because everything can be pre-coded • Over-predetermined standards/criteria/approaches • Students not involved in key steps in the assessment process In short, we need to resist the temptations of a behaviourist technology

27. Question • What is the area in the use of digital technologies for assessment in which we need to invest effort to get the greatest payoff for learning with very little investment of resources? • Type your responses into the text-chat

28. VI. Key areas for development • Feedback • Self-judgements • Collaboration in assessment

29. 1. Feedback • Feedback in non-digital environments suffers from the misconception that the process is complete when useful information is provided for students • For feedback to be effective, the effect of information on students’ work must be identified and adjusted to maximise the effect • Digital applications need to enable us to do this and for students to access and track feedback

30. 2. Self-judgements • Self-testing with immediate guidance on all answers is easy to do and is worthwhile especially in technical areas • Self-assessment involves much more than this, particularly identifying suitable criteria and making judgements about students’ own work • Examples: e-portfolios, Re:View

31. An example—Re:View • ReView provides a step in the direction of full self-assessment • It is a web application developed to aid marking, feedback and graduate attribute development. • Student self-assessment is an option that academics can select for each task http://reviewsecure.com

32. Self Assessment Process (video) http://www.youtube.com/watch?v=6dgFSbkswks http://reviewsecure.com/

33. Close up of staff marking screen with student’s self assessment (blue triangle sliders) showing after staff click ‘Save’

34. 3. Collaboration in assessment Collaboration is already a strong feature of digital environments, so what needs to be involved for assessment: • Group tasks • Consideration of standards and criteria • Peer feedback • Moderation of marks by other students, eg. http://spark.uts.edu.au

35. VII. What is the transformation needed? • Not about technology • We will use whatever is available and user-friendly • Not about techniques and methods • More are available that we can sensible use at present • It is about our disposition • Towards learning and whatever it takes • Towards seeing assessment as a central tool in promoting learning, that should also be in the hands of students

36. Discussionto be continued in the conference discussion forum • How will digital technology otherwise change our views of assessment? • What will students expect as a normal part of any educational interaction? What expectations should we foster?

37. Some further references • Boud, D. (1995). Enhancing Learning through Self Assessment. London: Routledge • Boud, D. (2009). How can practice reshape assessment? In Joughin, G. (Ed.) Assessment, Learning and Judgement in Higher Education. Dordrecht: Springer, 29-44. ISBN 978-1-4020-8904-6, 978-1-4020-8905-3 • Boud, D. (2010) Assessment for developing practice. In Higgs, J., Fish, D. Goulter, I., Loftus, S., Reid, J-A. and Trede, F. (Eds.) Education for Future Practice. Rotterdam: Sense Publishers, 251-262 • Boud, D. & Falchikov, N. (Eds) (2007). Rethinking Assessment for Higher Education: Learning for the Longer Term. London: Routledge • Boud, D. and Falchikov, N. (2006). Aligning assessment with long-term learning, Assessment and Evaluation in Higher Education, 31, 4, 399-413