Improving Undergraduate Education through the Assessment of Student Learning

1. Improving Undergraduate Education through the Assessment of Student Learning SACS-COC Institute on Quality Enhancement and Accreditation July 30, 2006 Peter T. Ewell National Center for Higher Education Management Systems

2. Looking Back: Origins of the Assessment Movement Two Decades Ago • Undergraduate Reform Reports of 1985-86 • Internal Stimulus: Call for More Coherent Teaching/Learning Approaches and Information for Improvement • External Stimulus: Stakeholder Demands for Information on “Return on Investment” • Tensions in Motive and Message Ever Since

3. Why Didn’t Assessment Go Away? • Pressure to Produce Evidence of Student Learning Outcomes Never Let Up • By Early 1990s, Accreditors Replace States as Primary External Stimulus to Get Started • Intermittent Federal Interest in Assessment as an Element of National Accountability • But Resulting Faculty Ambivalence About a Process Seen as “External” and “Administrative”

4. Looking Back: What’s Been Accomplished? • Assessment Is for the Most Part Perceived as Inevitable and Legitimate • Vast Majority of Institutions Have Statements of Learning Outcomes (General and Programmatic) • A “Semi-Profession” of Folks Involved in Assessment • Steadily Growing Sophistication with Respect to Methods of Gathering Evidence

5. Looking Back: What Hasn’t Happened? • Authentic Integration of Assessment into Faculty Cultures and Behaviors • Assessment Activities Still Largely “Added On” to the Curriculum Instead of Being Embedded In It • Systematic and Widespread Use of Assessment Results for Institutional and Curricular Improvement • Proactive and Sincere Institutional Engagement with Accrediting Organizations Around Topics of Assessment

6. Assessment as a “Perpetual Movement?” • Most Social Movements… • Fade as Fads, or • Go Away Because Core Ideas are Mainstreamed • The “Assessment Movement” Has Done Neither... • External Requirements Keep it Alive • But External Pressures also Constrain Faculty Buy-In and Meaningful Institutional Use • Moving Beyond a “Perpetual Movement” Will Demand... • Making Assessment Real to Faculty by Connecting It to the Actual Practice of Teaching and Learning • Re-Focusing Accountability on the Authentic Student Abilities that Society Says it Needs

7. Some Prominent Changes in Higher Education’s Operating Environment • Doing More with Less • Need for Curricula that are Effective and Efficient • Need Information About Curricular Functioning to Enable Effective Action • Changes in Instructional Delivery • Competencies and “Deep Learning” • Student Engagement and Role of Technology • New Views of Accountability • “Quality”— From Inputs/Processes to Results • Stakeholder Voices — Students and Employers

8. Challenges to the Academy: The Internal Dimension • Changing Paradigm of Teaching and Learning • Resulting Changes in Academic Roles, Behaviors and Structures • Nature and Role of “Faculty” in Instruction • Patterns of Student Enrollment • [Technology as a “Wild Card”]

9. A Changing Paradigm of Teaching and Learning • From “Faculty Teaching” to “Students Learning” • Students “Make Their Own Paths” through Multiple Learning Opportunities • Explicit Designs for Learning Based on Research

10. Changing Paradigm: Possible Lines of Response • Demonstrated Achievement Becomes Paramount, not “Seat Time” • Students “Teach” One Another • Individual (and Asynchronous) Paths • Technology Seen as Opportunity to Re-Think [not as a “Solution”]

11. Some Implications for Student Assessment Processes • Assessments Reinforce Common Standards for Learning Across Curricula and Classes • “Seamless” Assessments Become an Integral Part of Curriculum and Pedagogy • Assessments Emphasize Connections and Longitudinal Development, not Just Attainment

12. Assessment Approaches: A Resulting Shift in Emphasis • Accountability-Based: Assessments Added onto Instruction to “Check Up” on the System in the Aggregate • Scholarship and Improvement: Assessments Built into the System to Simultaneously Assure Standards and Provide Feedback on Collective Performance

13. Kinds of Information Needed • Alignment of Key Learning Outcomes Across Units, Sequences, and Courses • Match Between Curricular Design, Delivery, and Student Experience • Match Between Instruction and Needs of Diverse “Student Bodies” • Effectiveness of Particular Innovations and Interventions

14. Changing Nature of the Faculty Role • Dis-aggregation of Instructional Roles • More Things for Faculty to Do • New Potential Career Patterns and Paths

15. New Faculty Roles: Possible Lines of Response • Emphasize Peer Support and Collaboration in Faculty Development • Recognize and Regularize Alternative Career Paths • Recognize “Mentorship” as the One Thing You Can’t Responsibly Outsource

16. Kinds of Information Needed • Capturing and Re-Aggregating Data About Discrete Instructional Functions • Accounting Technology-Based Costs • Tracking Faculty “Assets”

17. Changing Patterns of Student Attendance • Increased Levels of Multi-Institutional Attendance • Increased Complexity in Course-Taking Behavior within Institutions • Greater (and Unpredictable) Time Lapses Between Instructional Encounters

18. Changing Patterns of Attendance: Possible Lines of Response • Coherence Based on Common Practices [and Outcomes], not Common Content • Stress Ways for Students to Exploit and Reflect on Their Own Experiences • Establish Clear Transition Points at which to Assess Student Mastery of Key Concepts

19. Kinds of Information Needed • Relationships Between Particular Institutional Experiences and Particular Outcomes • Tracking Student Learning Styles and Individual Paths of Development • “Episode-based” [as opposed to time-based] Data Structures

20. Challenges to the Academy: The External Dimension • Increased Accountability • Changing Expectations Regarding What Students Know and Can Do • The “New Competition”

21. What Are States Doing? • Forces Influencing State Approaches • Decreased Agency Capacity Due to Funding Cuts • Momentum of K-12 Reform (NCLB) • Political Uncertainty and Instability • Types of State Policy Responses • P-16 Alignment • Performance Measures and “Report Cards” • Achievement Testing (Driven by K-12)

22. What’s Shaping Accreditation? • Forces Influencing Accreditors • Pressure for Specific Performance from Federal Government • Demands from Institutions to “Add Value” • New Models from Other Sectors and Abroad • “New Looks” in Accreditation • Focus on Outcomes and Effectiveness • Presenting Evidence [e.g. “Portfolios”] • Review Approaches [e.g. “Academic Audits”] • Connection to Institutional Planning [e.g. QEP]

23. Learning Outcomes: What Employers Expect • Higher-Order “Literacies” as Well as Specific Skills • Framed in Terms of “Practice” [not “Knowledge”] • As Much About Attitudes as Academics [“Soft Skills”]

24. External Forces: Some Resulting Influences on the Academy • Increased Emphasis on Credentialing • Modularity and Acceleration to Increase Accessibility • Pressure to Respond to “Students as Customers” • Accountability “Superstructures” that Divert Attention and Information Resources

25. Kinds of Information Needed • Outcomes and Performance Measures • Data on What Experiences/Services Students Can Expect • Peer Comparisons [Increasingly Outside the Academy] and Comparative Performance • Needs and Satisfaction of External Stakeholders

26. Attributes of a Meaningful “Culture of Evidence” • Shared Recognition That Many (But Not All) Things Are Knowable • An Accessible Store of Information About Organizational Condition and Performance • An Attitude Toward Problem-Solving that Minimizes “Finger-Pointing” • Clear Follow-Through On Decisions Made and Why They Were Taken

27. Cultures of Evidence: Success Factors • Visible Metaphor of Scholarship • Beginning with Real Problems and Processes, not with “Method” • Consistent Messages from Leadership • Periodically “Re-Socializing” the Community • “Closing the Loop” with Action

28. Cultures of Evidence: Inhibiting Factors • Either Excessive or Non-Existent Consequences • Alien Language and “Management Culture” • Excessive Complexity • Burnout and “Committee Fatigue” • Changing the Rules

29. Meaningful Assessment is More About Mindset than Method • Questions About Learning are not Just Matters of Opinion • What Information Might We Collect? • What Might We Expect to Find? • What Difference Would Finding Out Make? • Assessment is About Improving Practice • What are We Trying to Fix? • How Good is Good Enough? • What Changes are Implied?

30. The Bottom Line • For Internal Management, “Seat of the Pants” Decision-making is no Longer Sufficient • Information Used Openly, Consistently, and Continuously to Inform Academic Decisions • For External Constituencies, “Trust Me” is no Longer Sufficient • Need Clear, Understandable Evidence of Student Academic Attainment • Be Vigilant about the Information You Choose and the Signals it Sends • Make Sure that What You Measure is What You Value • Harness the Accreditation Process to Make it Happen!