A Case Study Presentation At the 2004 AAHE Assessment Conference On June 15, 2004

1. THE DIVERGENT BUT COMPLEMENTARY PATHS OF COMPREHENSIVE PROGRAM REVIEW AND STUDENT LEARNINGOUTCOMES ASSESSMENT A Case Study Presentation At the 2004 AAHE Assessment Conference On June 15, 2004 By Ed Rugg & Val Whittlesey Kennesaw State University

2. TWO MAJOR DOMAINS OF ASSESSMENT PROGRAMREVIEW LEARNINGOUTCOMES

3. FOCUS OFPROGRAM REVIEW UNIT OF ANALYSIS • A DEGREE PROGRAM • AN ORGANIZATIONAL UNIT ASSESSMENT ISSUES • INPUTS, PROCESSES, OUTPUTS • QUALITY INDICATORS • PRODUCTIVITY INDICATORS • CENTRALITY & VIABILITY • ROI & INVESTMENT POTENTIAL

4. FOCUS OF LEARNING OUTCOMES UNIT OF ANALYSIS • THE GRADUATE OF THE PROGRAM • STUDENT LEARNING ASSESSMENT ISSUES • KNOWLEDGE OF GRADUATES • SKILLS OF GRADUATES • ATTITUDES/VALUES OF GRADUATES • EDUCATIONAL GROWTH/GAINS

5. COMPLEMENTARY OVERLAP Learning Outcomes Constitute an Important, but Only One, Set of Outputs Evaluated in Program Review

6. ACCREDITATION’S QUEST FOR QUALITY ENHANCEMENT Southern Association’s (SACS/COC) PRINCIPLES OF ACCREDITATION is subtitled: FOUNDATIONS FOR QUALITY ENHANCEMENT “The concept of quality enhancement is at the heart of the Commission’s philosophy of accreditation….an institution is expected to document quality and effectiveness in all its major aspects.” (p. 5)

7. SACS CORE REQUIREMENTSFOR PROGRAM REVIEW 2.5 The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services….(Institutional Effectiveness) 2.12 The institution has developed an acceptable Quality Enhancement Plan…that …is part of an ongoing planning and evaluation process. (QEP)

8. SACS COMPREHENSIVE STANDARDS FORLEARNING OUTCOMES 3.4.1 …each educational program for which academic credit is awarded …establishes and evaluates program and learning outcomes. 3.5.1 The institution identifies college-level competencies within the general education core and provides evidence that graduates have attained those competencies.

9. SACS COMPREHENSIVE STANDARD ONINSTITUTIONAL EFFECTIVENESS 3.3 Institutional Effectiveness 3.3.1 The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.

10. IMPLEMENTING A DUAL FOCUS ON ASSESSMENT AT KSU • COMPREHENSIVE PROGRAMREVIEW • ASSURANCE OF LEARNING

11. IMPORTANCE OF A WEB-BASED ARCHIVE OF DOCUMENTATION • Managing Complex Multi-Year CPR and AOL Processes • Anytime Access, Referral & Communication on CPR & AOL • Sharing Experiences & Good Practices • Documenting Compliance & Improvement for Accreditation

12. ASSESSMENT REPORTING PRIOR TO 2003 • From 1994 to 2003, academic departments at KSU sent their annual assessment reports to the Office of Institutional Planning. • These nonstandard annual reports, ranging from 1 to 50+ pages, were simply filed away with no follow-up or feedback.

13. DISSATISFACTION WITH THE “A” WORD & REPORTS • Unfocused Reports – assessments rarely tied to learning outcomes or program goals. • Uneven Reports – from exemplary/sophisticated content to inadequate/unsophisticated. • Unappreciated – widespread aversion to and avoidance of assessment.

14. ASSURANCE OF LEARNING INITIATIVE AT KSU • Established an Assurance of Learning (AOL) Council in Spring of 2003 with High-Level Academic Leadership (AVPAA as Chair). • Council Membership Includes Two Assessment-Oriented Faculty from each College, the Director of Excellence in Teaching & Learning, and the Director of Institutional Effectiveness.

15. AOL COUNCIL FUNCTIONS • The AOL Council serves as a steering committee and advisory/consultative group to academic programs as they design, implement, and improve their assessments of student learning outcomes. • The AOL Council provides common structures and good practices for learning outcomes assessment and collegial formative feedback on annual reports at the program level.

16. EIGHT ELEMENTS OF STUDENT LEARNING OUTCOMES ASSESSMENT • Articulating Student Learning Outcomes • Connecting Outcomes to the Curriculum • Connecting Outcomes to Assessment Methods • Articulating Expected Results with Respect to Outcomes • Articulating the Assessment Plan for Collecting and Analyzing Data • Collecting and Analyzing Data • Comparing Actual and Expected Results • Using Results for Improvement

17. BEGINNING AT THE BEGINNING IN 2003-2004 • Articulating General and Specific Student Learning Outcomes for Each Degree Program and General Education • Connecting Learning Outcomes to the Curriculum

18. IMPLEMENTATION OF A COLLABORATIVE MODEL • Reporting and Evaluation Guidelines • Formal Workshops • Informal Brown Bag Consultations • Council Review of and Formative Feedback on AOL Reports • Assessment Resources • Examples of Good Practice

19. AOL COUNCIL REVIEW PROCESS IN SPRING 2004 • AOL Council used Evaluation Guidelines to provide formative feedback on the program reports. • Used subcommittee assignments for Council’s reviews. • 48 AOL program reports reviewed with their authors by the Council.

20. LESSONS LEARNED&NEXT STEPS

21. QUESTIONS?? PowerPoint & Handouts Available for Download at http://ie.kennesaw.edu