Evaluator Workshop Fall Visits 2010 Tuesday, May 25, 2010

1. Evaluator WorkshopFall Visits 2010Tuesday, May 25, 2010 Please join the audio portion of this training:866-740-1260, Access Code: 7489001 ReadyTalk Help Desk: 800.843.9166 International Help: 303.209.1600

2. Announcements This presentation and the accompanying materials are available for download from: http://www.wascsenior.org/evaltrainingFall2010 For assistance with Voice and Web connections please contact: ReadyTalk Help Desk, 800.843.9166International Help: 303.209.1600 Please mute your microphone if you are not speaking. If you have questions, please enter them into the Chat window.

3. WASC Evaluator WorkshopFall Visits 2010

4. Workshop Outcomes Know how to prepare for and conduct an effective visit Be prepared to produce a useful, high-quality team report Be ready to make sound judgments about institutions under the Standards Be familiar with resources that support your work on a team

5. Agenda • Context for the Visit/Accreditation • Preparing for the Visit • Conducting the Visit • Developing Team Recommendations • Writing the Team Report

6. Context for Accreditation and Visits • The continuing evolution of the WASC process and Standards • The accountability movement • Moving from assessment for recruitment to assessment for accountability • New areas of emphasis • The Handbook revision process

7. Recent Changes in the Institutional Review Process and Standards • Changes to Institutional Review Process re: Student Success, Program Review and EE Sustainability • Changes to CFRs • Clarifying the scope of the CPR visit to review the “infrastructure” for assessment of student learning • Program Review and Program-Level Student Learning in a systematic way Tool: Table A (RB pg. 41)

8. Covering the Impact of the Financial Recession on Institutions Questions to ask the institution: • How has the financial recession affected your institution? • How has your institution responded? • What plans are in place in case the current state of affairs becomes permanent?

9. Q&A • Please feel free to type in your questions using the chat window or just chime in.

10. The Three-Stage Review Process 1. 2. 3. Capacity & Preparatory Review Educational Effectiveness Review Institutional Proposal

11. The Three-Stage Review Process • Institutional Proposal • Capacity and Preparatory Review • Educational Effectiveness Review

12. Timeline for Three-Stage Review Process

13. Institutional Self-Review • The heart of accreditation • Built upon an effective internal process of • Evaluation • Self-reflection • Recommendations from previous reviews • Plans for action

14. Stage 1:The Institutional Proposal

15. The Institutional Proposal • Guides the entire accreditation review process • Connects institution’s context and priorities with the Standards of Accreditation • Provides primary basis for both institution self-review and team evaluation • Allows alignment of accreditation activities to institutional strategic plan and key areas chosen for improvement

16. The Letter of Intent • Submitted by institutions seeking Candidacy or Initial Accreditation, the LOI serves the same purpose as the proposal • Includes suggestions from Eligibility approval letter • Submitted to assigned WASC liaison, one year in advance of CPR Review • Instructions are in “How to Become Accredited” on WASC website

17. Stage 2: The Capacity and Preparatory Review

18. Purpose of the CPR • Review and verify the information in the institutional presentation (report and data) • Evaluate key institutional resources, structures, processes in light of Standards • Evaluate institution’s infrastructure to support and assess student learning • Assess institution’s preparedness to undertake the Educational Effectiveness Review

19. Stage 3: The Educational Effectiveness Review

20. Purpose of the EER • Invite sustained engagement by the institution on the extent to which it fulfills its educational objectives • Enable the Commission to make a judgment about extent to which institution fills its Core Commitment to Educational Effectiveness

21. The Two Core Commitments

22. Core Commitment 1 “The institution functions with clear purposes, high levels of institutional integrity, fiscal stability, and organizational structures to fulfill its purposes.”

23. Core Commitment 2 “The institution evidences clear and appropriate educational objectives and design at the institutional and program level. The institution employs processes of review, including the collection and use of data, that ensure delivery of program and learner accomplishments at a level of performance appropriate for the degree or certificate awarded.”

24. The Four Standards Tool: Standards at a Glance, RB p. 37

25. Standard 1:Defining Institutional Purposes and Ensuring Educational Objectives • Institutional Purposes • Integrity

26. Standard 2:Achieving Educational Objectives Through Core Functions • Teaching and Learning • Scholarship and Creative Activity • Support for Student Learning

27. Standard 3:Developing and Applying Resources and Organizational Structures to Ensure Sustainability • Faculty and Staff • Fiscal, Physical, Information Resources • Organizational Structures & Decision Making Processes

28. Standard 4:Creating an Organization Committed to Learning and Improvement • Strategic Thinking and Planning • Commitment to Learning andImprovement

29. Expectations for Two Reviews Tool: Expectations for Two Reviews (RB pg. 20)

30. Q&A • Please feel free to type in your questions using the chat window or just chime in.

31. Preparing for the Visit(Visit Guide, Part II, pp. 29-52)

32. Roles and Responsibilities of Team Members and Staff • Role of Team Chair (RB pg. 193) • Role of Team Assistant Chair (RB pg. 195) • Role of assigned WASC staff liaison (VG pg. 7) • Team assignments Tool: Section 9 (Tips, Roles and Advice, RB p 189)

33. Timeline For CPR/EER Reviews 12 weeks 2 months Institution mails report to team and WASC Team holds conference call Site visit held and team report written Institution responds to errors of fact in team report Institution responds to final team report Commission acts at February or June meeting Tool: CPR or EER Timeline (VG, pg. 29)

34. Pre-visit Preparation • Read all the documents from WASC • Standards, CFRs, policies, visit guide, rubrics • Background documents re: institution and purpose of the visit, including Proposal and/or last action letter/team report • Read the institutional report • Review the data portfolio and exhibits • What to look for and highlight?

35. Reviewing the Exhibits • Enrollment data • Headcounts and FTE • Graduation data • Faculty data • Key financial indicators • Inventory of Educational Effectiveness Indicators • Inventory of Concurrent Accreditation and Key Performance Indicators Tool: How to Review WASC Data Exhibits(RB pg. 61)

36. Reading the Report • Has the institution done what it said it would do in its Proposal? • Has it collected and analyzed data effectively? • Are its conclusions supported by evidence? • Are there serious problems or potential areas of noncompliance? • Does the report contain recommendations for further institutional action?

37. Worksheet for Team Pre-Visit Conference Call • Organizes team’s responses to institutional materials • Helps team make preliminary evaluation under the Standards • Provides basis for team to work toward consensus • Should be submitted in advance of call Tool:CPR Conference Call Worksheet(VG pg. 42) EER Conference Call Worksheet (VG pg. 47) SV Conference Call Worksheet (SVG pg. 70)

38. Team Pre-Visit Conference Call • Evaluates quality of institutional report and alignment with Proposal and previous action letter(s) • Identifies areas of good practice, improvement, and further inquiry • Identifies issues, strategies, evidence needed • Identifies persons and entities to be interviewed • Makes or refines team assignments • Plans visit logistics

39. Off-Campus Sites and Distance Education Programs(special requirement for some visits) Prior to Visit: Sites/online programs will be identified and assignments made • Review substantive change action letters to determine if issues have been identified • Develop plan for the review of the programs and/or sites During Visit • Interview faculty, administrators and students • Evaluate facilities OR online infrastructure • Observe classes (can be done ahead for online) • Document visit and findings in the appendix • Discuss important findings with team for inclusion in report, as appropriate Tools:Protocols(RB pg. 157, RB pg. 162) Forms(RB pg. 55, RB pg. 58)

40. Compliance Audit(special requirement for some visits) • Required for: • Institutions seeking Candidacy and Initial Accreditation • Some institutions under sanction • Additional report submitted by institution in advance of the visit—with links to documents • Prepare as appendix to report Tool: Compliance Audit Checklist(RB, pg. 51)

41. Determining Strategy for CPR Visit • What evidence is provided to show capacity and readiness for EE? • What are the strengths and weaknesses of the evidence? • What other evidence do you want to review to evaluate capacity and preparation for EE? • Do any issues arise with regard to the Standards? • Meetings: format/methodologies

42. Determining Strategy for EER Visit • What evidence is provided to show EE? • Why was it chosen? • What are the strengths and weaknesses of the evidence? • What other evidence do you want to see to evaluate effectiveness? • Do any issues arise with regard to the Standards? • Meetings: format/methodologies

43. Drafting in Advance of the Visit • Assistant Chairs draft outline of team report and Section I • Team members draft outline or text for which they are responsible, using institution’s report and data portfolio, with space for additional evidence, analysis and conclusions Tool: Team Reports(VG pg. 54)

44. Q&A • Please feel free to type in your questions using the chat window or just chime in.

45. Conducting the Visit

46. Launching the Visit: Team Executive Session • Discuss preliminary findings • Identify major issues for exploration • Refine lines of inquiry for each meeting • Confirm team assignments • Discuss use of tools and rubrics • Review preliminary outline/draft team report • Discuss options for confidential team recommendation • Review schedule

47. Visit Schedule • Executive sessions and debriefings with team only • Meetings and interviews with key individuals and groups • Open meetings with students, faculty and staff • Document review • Time for drafting report sections • Final exit meeting Tool: Visit Schedule (VG pg. 40, SVG pg. 68)

48. Confidential Email Account • Set up by WASC as extension of open meetings • Checked by Assistant Chair during visit • Important emails shared with team and investigated • Comments included in team report only if the institution has a chance to address them Tool: Sample Notification re: Confidential Email Account (RB, pg. 151)

49. Approaches Used on Visits • Document review • Interviews and meetings • Mini-questionnaires • Techniques for small and large meetings • Fishbowl exercises • Audits Plan visit methodologies in advance as part of schedule.

50. Tips for Good Interviews • Decide on a protocol for interview • Prepare questions and lines of inquiry in advance • Ask questions that elicit information, stimulate discussion, or require judgment • Avoid interrogation, leading questions, or loaded language • Avoid consultation, giving solutions, or talking about your institution • Let them do the talking