National Learning Infrastructure Initiative Transformative Assessment Focus Session March 15, 2002

1. National Learning Infrastructure Initiative Transformative Assessment Focus Session March 15, 2002

2. Mount Royal College Context • Four year undergraduate college • Comprehensive credit program base: 11,000 credit students • Large credit-free operation: 51,000 registrations in continuing education, languages institute, international education and Conservatory of Music operations • 80% of credit students from Calgary area • 240 FT faculty; ~ 350 PT faculty

3. “Sensibilities” of Mount Royal College’s Assessment Program • Informed by contemporary views and activity that complement institutional cultures • Supported and resourced • Transformative/Inclusive (Mertens, 1998) • Utilization Focused: R2 (Patton, 1997) • Change “aware” (Rogers, 1962)

4. Fundamental Principles for Developing an Assessment Framework Assessment is: • A function of our Mission (Core Purpose): • Student Success & Satisfaction • A function of our Vision: • Canada’s leading undergraduate college educating and training individuals for success in the new economy • A function of our Stakeholders and what they Value

5. Principles for Development:Assessment is a Function of Institutional Mission MRC’s Core Purpose: Student Success and Satisfaction

6. Principles for Development: Assessment is a Function of Institutional Vision Vision StatementSpeaks to the need for: • Canada’s leading innovation/progressiveness • undergraduate college relevance of what we offer • educating & training individuals quality of how its delivered • for success in the new economy value-added benefits

7. Value-added Principles for Development:Assessment is a Function of Institutional Values MRC’s Values: • Student-centered • Progressive, innovative, leading • Personalized teaching & services of highest quality • Education as an instrument for individual, community & economic development • Open communication, honesty, integrity, respect for individuals

8. What Does This Mean? MRC’s assessment framework answers the following fundamental questions: • What do key stakeholders value from MRC? • Are MRC’s educational offerings & services relevantto the needs & values of our stakeholders? • Does ourstudent-client focusfoster student academic success and satisfaction?

9. What Does This Mean? • Do our educational & business practices meet or exceed nationally/internationally recognized standards for quality? • Do our students/clients & other stakeholders realize value-added benefitsthat enhance their success in the new economy? • Is MRC positioned as being among Canada’s most innovative and progressive undergraduate colleges?

10. Values of Primary Stakeholders JUDGED ON OUTCOMES Students, Alumni, Community, Faculty/Staff, Governments, Investors, other comparable providers. • Who are they? • What do they value? • What criteria do they use to measure value?

11. Public Accountability PERFORMANCE PROGRESS ON VISION • What must we do to achieve our vision? • What are our implementation strategies? • How will success be defined?

12. Creating Shared Goals for Institutional Transformation • Identifying/Creating the Context for Change • Major goals in strategic documents • Priorities • Issues of the day • Expectations of key stakeholders • Perceptions of the institution • Demonstrate and communicate quality and effectiveness of what is done

13. How To . . . • Institutional Research • Assessment Task Group • Key Questions: Institution-wide dialogues • Project-based work • Program-based initiatives (Academic/Service) • Collaborative • Environmental Sensitivity & Scanning • http://www.petersons.com/ugrad/consider.html

14. CASE STUDY Course Adaptation Research Project Mount Royal College 2000-2002

15. Classroom Plus …use of learning technologies to augment, enhance and extend face-to-face instruction.

16. Course Adaptation for Web-Supported Delivery Pilot Project • Over-riding goal: • use of learning technologies to foster more flexible teaching-learning environments for faculty and students • End-result: “hybrid” courses, blending face-to-face and online in various proportions • 20 projects over a two-year period • Significant research/evaluation component • Funding: Provincial grant (Learning Enhancement Envelope)

17. Course Adaptation for Web Supported Activity Project GOAL #4 • To study the impact of such course adaptation on teaching-learning environments from a variety of perspectives including • Impact on the student learning experience • Impact on student learning • Impact on faculty teaching practice • Impact on faculty role, responsibilities and workload

18. Course Adaptation Research Project: CARP Targeted Outcomes • Reality Based & Results Oriented • Instructional Technology and Pedagogical Quality Models • Cost: time is the most important currency • Professional “Location” • Institutional Diffusion • Inventive yet rigorous methods • Authentic and visible evidence of student learning

19. CARP Project Overview: Research Components • Activity-based costing • Flashlight, NCHEMS, Custom • Assessment of teaching,learning and technology • Flashlight, FAST • Evidence for student learning • Visible Knowledge Maps • Support Faculty Research Interests

20. Map Coding & Analysis (QSR Nud.ist & SPSS) • Visible Knowledge • Class Benchmarked Learning Map Individual Learning Maps • Consistency and frequency of key conceptual ideas • Accuracy and application of key conceptual ideas • Correlations between time concepts are introduced and their appearance on maps • Significance of change over time • Relationship between measures (e.g. extent to mastery) • Relationship between Flashlight results and student grades • Totality of knowledge, sources of learning