Implementing Good Practices in Outcomes-Based Program Review

1. Implementing Good Practices in Outcomes-Based Program Review International Assessment & Retention Conference June 8, 2007 - St Louis, MO Marilee Bresciani, San Diego State University Cyd Jenefsky, John F. Kennedy University marilee.bresciani@mail.sdsu.edu & jenefsky@jfku.edu

2. Outcomes of Session • Evaluate good-practice principles in your institution’s program review processes • Analyze good-practice principles applied to a case study • Identify next steps for applying/ improving good-practices in your own institution

3. Inverted Funnel

4. Criteria for Good Practices for Outcomes-Based Assessment Program Review • Criterion 1: Clear understanding of goals & expectations for program review • Criterion 2: Collaboration • Criterion 3: Use of results

5. Criterion 4: Awards and recognition • Criterion 5: Resources to support program review • Criterion 6: Coordination of the process

6. Criterion 7: Flexibility • Criterion 8: Addressing barriers • Criterion 9: Evaluation of the program review process

7. What good practice criteria do you already have in place at your institution? Use the sample criteria checklist to apply to your institution.

8. Inverted FunnelReminder

9. Case Study: JFK University Illustration of implementing criteria 1, 2, 5, 6, 7 & 8

10. JFKU: Process Design Groundwork: • Univ-wide key stakeholder involvement • High-credibility, active faculty • Explicit leadership commitment • Accreditation review as lever • Point person in charge of development

11. Process Design • Research-based • Good-practice research • Models from good-practice institutions • Iterative • Start with ‘what matters’ to stakeholders • Train-the-trainer model • Clear committee goals and timeline

12. Implementation • Actively cultivate buy-in • Start with “champions” • Professional development at each step • Address fears, anxieties, concerns • inc. one-on-one conversations • Stakeholder representatives

13. Guiding Questions • What do we want our students to learn by the time they complete our program? • Intended student learning outcomes • What are they actually learning? • Actual student learning results

14. Guiding Questions (cont’d) • How well are they learning these outcomes? • Levels of achievement (based on explicit criteria) • How do we know? • Evidence of student learning

15. Guiding Questions (cont’d) Closing the loop: • How are we using the evidence to guide decisions for improvement? • Do the improvements we make work?

16. Capacity-Building • Process builds capacity for assessment • does not presume capacity • workforce retraining • Built into conceptual framework • Train-the-trainer • expert • peer-to-peer training • “Zone of proximal development”

17. Flexibility • Constantly self-assessing & adapting • Process serves “end-users” • Disparate cultural groups • Work with multiple accreditation standards • Learning as we go

18. Apply JFK’s Process What criteria can you easily identify in their program manual that you could adapt and apply to your institution?

19. How do we move our institutions toward implementation of good practice criteria?

20. Suggestions for Implementation • Create a well-represented, well-respected OBAPR committee • Organize the committee’s role and responsibilities • Articulate expectations for outcomes-based assessment program review • Plan short-range and long-range goals

21. Suggestions (cont’d) • Identify existing resources and processes and identify new resources • Establish a communication plan • Discussimplementation barriers and strategies to overcome them • Move forward with flexibility

22. What are your organization’s next steps for implementing good practices in OBAPR?

23. What other questions do you need to ask yourselves?

24. The process is not a means unto its own end; it is a way to systematically engage in daily critical inquiry about discovering what works well and what needs to be improved. (Maki, 2004).

25. References • Bresciani, M.J. (2006). Outcomes-Based Academic and Co-Curricular Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing

26. Criterion Sample Selection References • American Association of Higher Education (1994). Nine principles of good practice for assessing student learning. Retrieved March 17, 2006, from http://www.aahe.org/assessment/principl.html. • Eckel, P., Green, M., and Hill, B. (2001). On change V—Riding the waves of change: Insights from transforming institutions.Washington DC: American Council of Education. • Lopez, C. (1997). Opportunities for improvement: Advice from consultant-evaluators on assessing student learning. Evidence of strong institutional support for assessing student learning. Retrieved March 16, 2006, from http://www.ncahigherlearningcommission.org/resources/assessment/index.html.

27. One Minute Evaluation • What is the most valuable lesson that you learned from this workshop? • What is one question that you still have?

28. Contact info: marilee.bresciani@mail.sdsu.edu jenefsky@jfku.edu