Webinar Series

1. Webinar Series This session is the firstof e-Assessment Scotland 2013 20 Aug 2013: Challenges and Opportunities Associated With Implementing eMarking Workflows in a University Setting Courtenay Harris, Connie Price and Helen Flavell (Curtin University, Australia) Your hosts Professor Geoff Crisp, Dean Learning and Teaching, RMIT University geoffrey.crisp[@]rmit.edu.au Dr Mathew Hillier, Teaching and Educational Development Institute, University of Queensland mathew.hillier[@]uq.edu.au

2. Challenges and Opportunities Associated With Implementing eMarking Workflows in a University Setting Dr Courtenay Harris Dr Connie Price Dr Helen Flavell

3. Presenters Dr Courtenay Harris School of OTSW Faculty of Health Sciences Dr Connie Price Manager of Assessment Curtin Teaching and Learning Dr Helen Flavell Coordinator of SOTL Faculty of Health Sciences

4. PERTH WESTERN AUSTRALIA

5. Assessment in the 21st Century Context HE Context Assessment Issue • Threats to existence of universities • Accrediting student learning remains key function of HE • Infiltration of technology • Decreasing government funding • Increasing accountability • Increasing student numbers • Increasing student diversity • Increased focus on role and value of assessment • What aspects can be automated or done by AI • Need to develop accurate costing models for assessment • Including different assessment types • Greater proportion of workload around/not in the marking process • Every aspect of assessment is magnified • Need for focused feedback

6. Foundations for Professional Health Practice 100 (FPHP 100) • Common first year unit for all courses in the Faculty of Health Sciences • Super large unit • 2900 students across both semesters • 40 teaching staff • Inter-professional education • staff and students • Internal (F2F) and external (online) student enrolments • Teaching model and use of class based technology • Varied assessment types

7. Drivers for eMarking within FPHP 100 • Electronic submission / return • Practicalities due to 2900 students /40 tutors • Feedback timeframes • Access to student submissions • Academic integrity • Consistent and efficient assessment practice and feedback • implementation of marking rubrics • transparency for staff and students • Change of balance of editing vs focused feedback • Efficiency for staff, while maintaining rich feedback • Students • Student evaluation of the unit indicated improvement required

8. Electronic Management of Assignments

9. Assessment Management Workflow Submission Record Grade Return Feedback Annotate Calculate Grade Grading Apply Rubric

10. Assessment Management Workflow Submission Record Grade Return Feedback Annotate Calculate Grade Grading Apply Rubric

11. Assessment Management Workflow Submission Record Grade Return Feedback Annotate Calculate Grade Grading Apply Rubric

12. Assessment Management Workflow Submission Record Grade Return Feedback Annotate Calculate Grade Grading Apply Rubric

13. Workflow Repeat for each student

15. eMarking Working Party Priorities • Workflow • Minimal steps • Minimal tools • Multiple screens • Minimise workload • Enhancing Integrity • Academic integrity (plagiarism check) • Moderation • Archiving of electronic artefacts • Assessment and Feedback • Evaluation -> Mark • Mark-up of artefact • Provide feedback in a variety of ways

16. Electronic Assignment Workflow for Blackboard Learn 9.1 Staff Workflow Student Workflow Blackboard Desktop Application Create Turnitin assignment Create Blackboard eRubric Submit assignment View attempts via Grade Centre Receive Digital Receipt View Turnitin Originality Reports View Originality Report Download submissions from Grade Centre Launch MS Word or Adobe Acrobat Annotate submission Grade Assignment Upload annotated submission View mark Download feedback Store assignment artifacts View eRubric

17. Electronic Assignment Workflow for ReMarksPDF Staff Workflow Student Workflow Turnitin ReMarksPDF Blackboard Create eSubmission Link Submit assignment as .pdf Download submissions from Grade Centre .zip file Upload .zip to Turnitin View Turnitin Originality Reports Launch ReMarksPDF App Connect to Bb database Create eRubric Grade Assignment Provide feedback View mark Upload annotated submissions to Grade Centre Download feedback View eRubric Store assignment artifacts

18. Electronic Assignment Workflow for GradeMark Staff Workflow Student Workflow Create eSubmission Link Create eRubric Submit assignment Receive Digital Receipt View Originality Report View Originality Report Complete eRubric Annotate document and provide feedback View mark View feedback Store assignment artefacts View eRubric

19. “The 2013 Horizon Project Australia Advisory Board agreed with the global group that digital media literacy is not nearly pervasive enough in faculty training. There is a need for more training before being asked to teach, and for more professional development opportunities once in the profession. This key challenge is underscored by the widespread belief that most academics are not leveraging emerging technologies for their own work, whether that be in the classroom or in support of their own research. While a lack of adequate training opportunities is a part of the challenge, ultimately a change in the mind sets of disciplines and individual faculty will be required, along with cultural shifts within institutions, before emerging tools and technologies are routinely adopted and implemented as a matter of course.”

20. “Being agile as a teacher means staying true to your purpose, fixed in your intent to provide excellence in education but always adapting and adopting new methods of achieving that goal and where prompted to by changes in the world around you: the Internet demonstrates how agility is now a key aspect of knowledge work—make it part of your teaching.” (Allen M, 2012)

21. Developing academic staff capacity for agile teaching: a case study in implementing eMarking workflows for student assessment in a large first year Health Sciences unit Office of Learning and Teaching Seeding Grant Funded Project

22. Professional Development Approach • Theories of change and change management within the context of academic leadership and current higher education change drivers • Individual reflection on self as well as social and cultural beliefs in relation to innovation, teaching and learning and change

23. Professional Development Approach • Exploration of intrapersonal skills such as self-talk to deconstruct habitual thinking and its effect on performance and acceptance of ICT innovations • Develop an understanding of ICT system components as opposed to “how to use” a program to support a shift in thinking/approach from mastery t o adaptive responses.

24. Method • Pre and post PD questionnaire combining measures relating to attitudes to teaching and technology acceptance. • Semi-structured interviews with 16 participants following the first major assessment piece. Coded and analysed by a qualitative researcher using NVivo. • Two focus groups (12 participants in total) to check themes that emerged from interviews.

25. Themes • The PD influenced them in the following ways: • Cognitive (majority of participants reported) - increased awareness by bringing new knowledge, adding to existing knowledge, refreshing knowledge. Knowledge on software applications, strategies, self awareness to create creative thinking / planning for immediate and future plans. • Emotional (majority reported) – increased confidence, excitement, enthusiasm, motivation, empowerment • Practical (majority reported)– taking action to improve teaching practice, manage both IT and teaching challenges, using IT and teaching strategies, improving skills

26. Characteristics of the participant are key to being agile in the adoption of ICT • Characteristics demonstrated by participants included: • Professional background / employment status / professional identity • Previous work and teaching experience • Family / personal responsibilities and experiences • Self efficacy, willingness to persevere • Willingness to engage with others, mindful of others in the unit / in regards to IT support • Ability to employ intrapersonal strategies to cope with changing environments • ‘Can-do’ attitude

27. Discussion • Issues of self-selection • Sessional staff • Disciplinary background • Impact of workload/personal life on uptake • Change in behaviours requires ongoing commitment/time to reflect • Structured nature of the unit

28. Sources • Slide 3 images • http://babynameobsessed.wordpress.com/2012/12/28/leilani-and-jax/the-cottesloe-beach-perth-australia/ • http://www.australia.com/explore/cities/perth/perth-surrounds.aspx • http://www.police.wa.gov.au/Aboutus/Policejurisdictions/tabid/1352/Default.aspx • Slide 12 image http://thatstotallytarot.com/wp-content/uploads/2011/01/work_stress.jpg • Slide 17 • Johnson, L., Adams Becker, S., Cummins, M., Freeman, A., Ifenthaler, D., and Vardaxis, N. (2013). Technology Outlook for Australian Tertiary Education 2013-2018: An NMC Horizon Project Regional Analysis. Austin, Texas: The New Media Consortium. • Slide 18 • Allen, M.). Move quickly to adopt new approaches. Learning in Networks of Knowledge Retrieved June 22, 2012, from http://www.knowledgenetworklearning.net/concepts/pedagogy/teachingagility/