350 likes | 451 Vues
This project aims to develop a quality assessment method for digital educational materials, focusing on professional and academic writing skills. The target audience includes students, instructors, and developers. Project partners include various universities and organizations. Research methods such as card sorting and observational analysis are used. The project phases include research, development, evaluating, testing, and dissemination. The Matrix usability criteria assess the learning environment layout, multimedia integration, and user needs. The development process involves iterative evaluation and piloting by domain experts and researchers.
E N D
Antwerp - 9 February 2007 Assessing digital educational materials
Project Aim Developing a generic method for quality assessment of digital educational materials (professional and academic writing skills). Target audience • students • Instructors • developers
Project Partners • University of Antwerp • Ghent University • Higher Institute Antwerp • Radboud University Nijmegen • Vergouwen Overduin • Stroomt • Lodz University • Zürcher Hochschule Winterthur • Swedish TelePedagogic Knowledge Center
Project Phases • Research • Development • Evaluating and testing • Dissemination
Research methods • Card sorting • Observational analysis • Semi-structured interview • Focus group
Calliope & WritingStudio • Semi-structured electronic learning environments • Multilingual • Blended learning • Professional writing and communication skills
Matrix usability • The layout of the learning environment is appealing. • The layout of the learning environment is efficient. • It is easy to find specific information. • The integration of multimedia in the learning environment is optimal. • First time users find it easy to use the learning environment. • The digital learning environment meets the needs of the user. • The digital learning environment functions as it should.
Procedures • Card sorting • Very insightful, but not for usability (transfer to content) • Task analysis • Real actions vs. Session actions • Morae recordings prove to be added value • Semi-structured interview • Offers ad hoc explanations • Focus group • Very insightful
Conclusions • Card sorting • Select research subject carefully • Task analysis • Valuable technical info • Intuitive responses • Useful input for focus group • Interview • Verbalising explicit info • Focus group • Useful roundup
Conclusions • Usability testing allows/forces us to adopt the language of the users. • Usability testing indicates that it is difficult to distinguish between different sub domains. • Example: • Do you think the learning environment is efficient? • Yes, it contains the information I’m looking for.
Program • Welcome & Introduction • Online Learning • Theoretical background to QuADEM • How does it work? • DIY • Discussion
Introduction • Stands for ‘Quality Assessment for Digital Educational Materials’ • Funded by EU and in collaboration with international partners • Method for the development & review of digital educational materials • Who are you?
Context • Socio-economic aspects • New conceptions of learning • New technology • Innovative new learning environments with computers & internet as new tools
Blended learning • Blended learning can combine • modes of web-based technology • various pedagogical approaches • any form of instructional technology • instructional technology with actual job tasks (Driscoll 2002)
Blended learning • Blended learning and • individualized learning • problem-based learning (Emig, 1977 ; Flower & Hayes, 1980) • narrative learning (Cf. Vygotsky’s zone of proximal development) • social constructivism
Blended learning • Valathian • Skill-driven learning“a combination of self-paced learning with instructor or facilitator support to develop specific knowledge and skills” • Behaviour-driven learning“a combination of various events and delivery media to develop specific behaviours” • Competency-driven learning”a combination of performance support tools with knowledge management resources and mentoring to develop workplace competencies”
Blended learning • Kerres & De Witt
Blended learning • Khan
Blended learning • Graham • Activity-level blendingis a combination of face-to-face an computer-mediated activities. • Course-level blendingis a combination of face-to-face and computer-mediated activities as part of a course. • Program-level blendingoften entails one or two models: either the participants choose a mix between face-to-face and online courses, or the combination is prescribed by the program. • Institutional-level blendingoccurs when an institution has made an organisational commitment to blend face-to-face and computer-mediated activities.
Blended learning • No prefixed blend: modular approach to criteria • Which criteria matter to you?http://www.writingstudio.eu
Development process • QuADEM objective • To develop a method for Quality Assessment of Digital Educational Materials. • Assessment based on preset criteria • Through an iterative process of evaluation and piloting • By domain experts and researchers
Development process • Delta phase • Literature study • Framework development • Gamma • Setting criteria • Determining evaluation procedure • Finetuning criteria & evaluation • Beta • Validating evaluation procedure • Finetuning evaluation procedure • Alpha • Fixing criteria
Method • Overall composition • several aspects of quality = components • to evaluate one component = unit • core ingredient of a unit = checklist • a checklist = list of criteria
Method: components • A component represents one aspects of what constitutes a successful digital learning material. • Content • Assessment • Interculturalaspects • Learning styles • Usability • Pedagogy • Style & language • Multimedia • Questionnaires
Method: unit • A unit contains all information you need to evaluate a specific component of an online learning environment • Summary • Fact sheet • Score • Preconditions • Checklist • Methods • Procedure • Manual
Evaluation • Starting situation • user is developing a new digital learning materials or revising existing materials • Steps • determine goal & focus • determine assessor • select units • check preconditions • select research methods • select respondents • design procedure • gather rating information • determine scores
DIY 1 • Usability • Focus group
DIY 2 • Assessment • Think aloudinterview
Disclaimer • The QuADEM project has been funded with support from the European Commission. • This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.