1. Graduate Learning Outcomes: The challenge of assessment Serge Desmarais University of Guelph Learning Outcomes Conference October 17, 2014

2. Fostering a learning outcomes culture • U of G established 10 graduate learning objectives in 1987: • Inspirational and lofty • “desired characteristics of graduates” • Used to guide new grad and undergrad course and new program proposals • Since 2005, we re-engaged the community in an effort to develop a learning outcomes culture • 2005: Provost’s White Paper • 2006-2007: 21st Century Curriculum Committee • 2008: UUDLEs & GDLES • 2010: IQAPs at U of G & across the province

3. Fostering the Culture • First strategic steps • Introduced “Outcomes” as a priority for governance: • First addressed at Board of Undergraduate Studies and Program Committees • Following by introduction of same strategy at Board of Graduate Studies • Created a new Senate Committee: • Quality Assurance – new Senate committee; assist departments in meeting requirements under IQAP • Engaged Associate Deans with program-level responsibility in the discussion • Increased staff support: more teaching & learning staff

4. University Learning Outcomes • Two year process of broad consultation with faculty, staff and students • New learning outcomes combine: • Bases of Competence (Evers, Rush & Bedrow) • U of G’s learning objectives (1987) + + • LEAP

5. University Learning Outcomes 5 University-wide Outcomes approved by Senate for both undergraduate (2012) and graduate (2013) programs: Critical and Creative Thinking Literacy Global Understanding Communicating Professional and Ethical Behaviour

6. Critical and Creative Thinking . . . one applies logical principles . . . to solve problems with a high degree of innovation, divergent thinking and risk taking. . . . show evidence of integrating knowledge across disciplinary boundaries. Depth and breadth of understanding of disciplines essential to this outcome.

7. Critical and Creative Thinking • Depth and Breadth of Understanding • Level 3: Compares the merits of alternate hypotheses in many different disciplines. Demonstrates mastery of a body of knowledge and critically evaluates the limits of their own knowledge and how these limits influence analyses.

8. Assessing L.O.s in Undergrad programs • Collaboration with Desire2Learn to develop the assessment tools within our current LMS • Our eventual goal: an assessment method that will be used by all departments and programs • The initial plan engaged two programs: • Bachelor of Arts and Science • Bachelor of Engineering • Now working with the full B. Eng program and beginning process with DVM (Veterinary Medicine)

9. Lessons learned… so far • Software development is not the most complex aspect of the project • Decisions re: which outcomes to assess at what point in the curriculum progression is complex • Long term, challenge is to gain full departmental and faculty engagement: • Much resistance, paranoia, and hope for continued status quo • Identify your champions and the most engaged programs, and leverage their enthusiasm and persuasiveness

10. How to proceed? • A “one size fits all” model is not the right approach • Must respect disciplinary differences • Efforts should be programmatic, not only course by course • Graduate programs must have a clear sense of when and where specific learning outcomes will be addressed and assessed • Curriculum Mapping • Assessment must be documented and evidentiary based

11. So… how do we assess in grad programs? • Moving to a culture of evidence-based continuous improvement will be hard work • Requires a re-envisioning of the entire graduate program process • Requires a commitment and broad agreement on departmental and program strategy • Let’s work on a few examples….

12. Communicating Rubric

13. Information Literacy Rubric

14. Thank You Serge Desmarais Acting Provost and Vice-President (Academic) s.desmarais@exec.uoguelph.ca www.uoguelph.ca/vpacademic/avpa/outcomes