Exploring Good Practices in Outcomes-Based Assessment Program Review

1. Exploring Good Practices in Outcomes-Based Assessment Program Review Marilee J. Bresciani, Ph.D. Associate Professor, Postsecondary Education and Co-Director of the Center for Educational Leadership, Innovation, and Policy San Diego State University 3590 Camino Del Rio North San Diego, California, U.S.A. 619-594-8318 Marilee.Bresciani@mail.sdsu.edu

2. Overview of Book • Describes “good practices” in outcomes-based assessment program review that were generated from multiple case studies. • Outlines basic components of outcomes-based assessment program review. • Introduces criteria for identifying good practices within outcomes-based assessment program review. • Outlines steps for implementing sustainable outcomes-based assessment program review. Bresciani, M.J.

3. Context • Public demand for accountability in higher education • Commission on the Future of Higher Education • Professor Bensimon’s highlight of our innate intellectual inquiry, “Why is that what I am doing doesn’t work?” Bresciani, M.J.

4. Purpose • The purpose of this study is to use multiple institutional case studies to explore effective outcomes-based assessment program review practices in order to establish criteria that could be adapted at other institutions for implementing pervasive and sustainable outcomes-based assessment program review. Bresciani, M.J.

5. Framework and Epistemology • Astin’s I-E-O • Astin, A. W. (1993). Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. American Council on Education; Oryx Press. • Papert’s Situating Constructionism • Papert, S. (1991). Situating Constructionism. In Constructionism. Edited by Papert and Harel. Cambridge, MA:MIT Press. Bresciani, M.J.

6. Research Questions • What are the components of outcomes-based assessment program review at institutions where this practice is pervasive, effective, efficient, and enduring ? • What are the criteria for pervasive, effective, efficient, and enduring outcomes-based assessment program review processes? • What are the key steps in implementing pervasive, effective, efficient, and enduring outcomes-based assessment program review processes? Bresciani, M.J.

7. Methodology • The methodology for this study was multiple case studies (Stake, 1995; Merriam, 1988) • Multiple sources of information were utilized in this analysis: surveys, case study submissions, document analysis, and clarification sought through email and over phone. • 3 Year Study • Open, axial, and selective coding • Member checks Bresciani, M.J.

8. Sample • Criterion sampling • A list of 43 institutions generated based on criteria and circulated to 25 assessment scholars for further comment. No additions or deletions were made. • 18 institutions completed all phases and were included in the study Bresciani, M.J.

9. Good Practice Components 1. Program Name and Description 2. Program Mission Statement 3. Program Goals 4. Program Outcomes 5. Student Learning Outcomes 6. Evaluation Methods for Each Outcome Bresciani, M.J.

10. Good Practice Components (cont’d) 7. Results by outcomes or interpretation of results as they relate to outcomes 8. Decisions & recommendations made about the program for each outcome 9. Level of involvement of program faculty/staff in process 10. Time-frame of evaluation process Bresciani, M.J.

11. Examine the application of the template See program review guidelines from John F Kennedy University

12. Criteria for Good Practices of Outcomes-Based Assessment Program Review Criterion 1: Clear Understanding of Goals and Expectations for Program Review Criterion 2: Collaboration Criterion 3: Use of Results Criterion 4: Awards and Recognition Criterion 5: Resources to Support Program Review Bresciani, M.J.

13. Criteria for Good Practices (cont’d) Criterion 6: Coordination of the Process Criterion 7: Flexibility Criterion 8: Addressing Barriers Criterion 9: An Evaluation of the Program Review Process Bresciani, M.J.

14. Examine a sample matrix for considering effective, efficient, and enduring OBAPR

15. Suggestions for Implementing Outcomes-Based Assessment Program Review (OBAPR) • Create a Well-Represented, Well-Respected OBAPR Committee • Organize the Committee’s Role and Responsibilities • Articulate Expectations for Outcomes-Based Assessment Program Review • Plan Short-Range and Long-Range Goals Bresciani, M.J.

16. Suggestions for Implementing Outcomes-Based Assessment Program Review (OBAPR), Cont. • Identify Existing Resources and Processes and Identifying New Resources • Establish a Communication Plan • Discuss Implementation Barriers and Strategies to Overcome Them • Move Forward with Flexibility Bresciani, M.J.

17. Examine sample questions to consider when implementing OBAPR

18. Criterion Sample Selection References • American Association of Higher Education (1994). Nine principles of good practice for assessing student learning. Retrieved March 17, 2006, from http://www.aahe.org/assessment/principl.html. • Eckel, P., Green, M., and Hill, B. (2001). On change V—Riding the waves of change: Insights from transforming institutions.Washington DC: American Council of Education. • Lopez, C. (1997). Opportunities for improvement: Advice from consultant-evaluators on assessing student learning. Evidence of strong institutional support for assessing student learning. Retrieved March 16, 2006, from http://www.ncahigherlearningcommission.org/resources/assessment/index.html. Bresciani, M.J.

19. Citation • Bresciani, M.J. (2006). Outcomes-Based Academic and Co-Curricular Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing Bresciani, M.J.