Assessment 101 An Introduction for Administrative and Support Units

1. Assessment 101 An Introduction for Administrative and Support Units Presented by Horace D. (“H.D.”) Stearman, Ph.D. Director of Institutional Effectiveness August 14, 2008

2. Module One: Overview • What Is Assessment? • Why Do We Do Assessment? • What The Accreditors Say • Assessment Misconceptions • The Assessment Cycle • Step 1: Mission Statement and Core Functions/Services • Step 2: Expected Outcomes • Step 3: Measures • Step 4: Target Levels • Step 5: Findings • Step 6: Action Plans • Potential Assessment Pitfalls • Relation of Assessment to Planning

3. What Is Assessment? • Assessment is a systematic, ongoing process of collecting, reviewing, and using data about an institution’s academic, administrative, and support units to improve the effectiveness of these units. • We carry out assessment of administrative and support units by • Creating a mission statement for our unit that briefly describes our purpose; • Identifying the core functions and services we provide that enable us to meet our mission; • Determining expected outcomes to describe the results we expect to see in the way we perform our core functions and services; • Selecting measures that will quantify the achievement of our expected outcomes; • Setting target levels of performance on the expected outcomes; • Interpreting the levels of performance that we attain on our expected outcomes to judge how well our actual performance matches our target levels; and • Using the results to document, explain, and improve the effectiveness of our unit in accomplishing our mission.

4. Why Do We Do Assessment? • To improve what we do • To inform decision-making, budgeting, and planning • To focus our efforts • To show students, alumni, employers, and others that we’re doing a good job • To satisfy the expectations of our accreditors

5. What The Accreditors Say • The institution engages in ongoing planning to achieve its missions and goals. It also evaluates how well, and in what ways, it is accomplishing its mission and goals and uses the results for broad-based, continuous planning and evaluation. Through its planning process, the institution asks questions, seeks answers, analyzes itself, and revises its goals, policies, procedures, and resource allocation.Northwest Commission on Colleges and Universities “Standard 1.B – Planning and Effectiveness”

6. Assessment Misconceptions • It’s A Passing Fad • It’s Not My Job • You Can’t Assess What I Do • I Don’t Have Time For Assessment • I Don’t Have The Expertise To Do Assessment • Assessment Might Reveal Shortcomings Or Failures

7. The Assessment Cycle Outcomes Plans for ImprovementAssessment of Outcomes

8. Step 1: Mission Statement and Core Functions/Services • Write a brief mission statement that describes your unit’s purpose. • Identify the four to seven core functions or services that enable your unit to meet its mission. • Taken together, the mission statement and the list of core functions or services answer the questions “What do we do and how do we support the RMUoHP mission?”

9. Some examples of mission statements and core functions • The Department of Career Services counsels students and alumni as they explore career directions and equips them with the information they need to make good decisions.Our core functions include • Provide resume-writing workshops, mock interviews, and related training to students as they prepare for job searches • Coordinate on-campus visits by corporate recruiters and on-campus job fairs • Maintain a database of job postings and a library of employment-related books and DVDs for student use • Counsel students and alumni in best practices for job searches • The Office of the Registrar maintains and processes student academic records accurately and efficiently. Our core functions include • Maintain student satisfaction with registration functions • Process degree audits for graduating students • Fill transcript requests

10. Step 2: Expected Outcomes • Write three to five expected outcomes that describe the results you expect to see in the way you perform your core functions and/or deliver your key services over the coming year. • Expected Outcomes should be • related to something under the control of your unit • worded in terms of what your unit will accomplish or what your clients will think, know, or be able to do following the provision of services (“X will happen”) • measurable, meaningful, manageable

11. Some examples of expected outcomes • The Office of Institutional Research will collect and disseminate campus data for internal and external reporting purposes. • The Office of the Registrar will provide resources to enable students to use on-line registration successfully. • The Office of Career Services will engage students in development early and often through various outreach efforts. • The Office of Information Technology Support will provide its customers with prompt and effective assistance in resolving their technical problems related to systems, networks, and desktop applications.

12. Step 3: Measures • Select at least one method of measurement—preferably more—for each expected outcome that you created in Step 2. • Measures should be observable and, in most cases, yield quantitative data. • Types of measures include • counts/rates/percentages • questionnaires and surveys • minutes of meetings and focus groups • deadlines met

13. Step 4: Target Levels • For each of the measures selected in Step 3, set a specific goal for the “score” that you hope to achieve. • Other terms that are often used for target levels are indicators or criteria. • The target level that you set for each measure should • appear reasonably attainable • take into account your past performance in the function or service being assessed • provide the potential for showing improvement

14. Some examples of measures and target levels • Average attendance at library orientations will be 80% of all first-year students. • 90% of fifty randomly selected financial aid files will be complete and accurate when audited. • Average scores on the Student Satisfaction Survey will be 4.2 on a five-point scale. • 100% of scholarship applications will be processed within sixty days of receipt. • By December 2008, faculty using TechLink will be surveyed to determine what they see as the three most critical areas of need in upgrading the service.

15. Step 5: Findings • Briefly summarize the findings of your assessment by including for each expected outcome • the target level that you set • the actual level that you attained • a brief analysis of why you did or did not meet the target level

16. Some examples of findings • In the annual survey of faculty/staff satisfaction, 36% of our customers expressed frustration with the wait time for help desk requests. Our target level was no more than 15% for this measure. Analysis of survey data showed that 85% of those respondents expressing frustration with the wait time for help desk requests were located in the Office of Financial Affairs. • Average routing time for personnel action forms (PAF) was decreased by 21%. Our target level was 30% for this measure. Analysis of the routing pathways for a random sample of thirty PAFs showed that forms for administrative personnel required seven more hours in transit than those for support staff because of several additional sign-offs being necessary.

17. Step 6: Action Plans • Your action plan should be a brief description of what steps you will take to address any expected outcomes where you did not meet your target levels. • You also may wish to review the outcomes in which you met your target levels to see if other improvements or activities are needed in the future—especially if you anticipate conditions changing so as to pose a threat. • Action plans should be included as expected outcomes in your next assessment cycle. • Unless you “close the loop” by taking actions based on your findings, you’re not really doing assessment!

18. Some examples of action plans • To address the 15% decline in annual giving by alumni, a personal contact, mail, and email campaign emphasizing to new graduates the benefits of annual giving to the institution will be developed and implemented in July 2009. • The 20% increase in wait time for level-one response will be addressed by the hiring of four new employees for the help desk during fiscal 2009. • The 17% increase in the number of books not returned to the library on time will be addressed by increasing the fines for overdue books to $3.00 per day starting in fall semester 2009.

19. Potential Assessment Pitfalls • Taking measurements that are not related to expected outcomes • Waiting until the plan is perfect before beginning • Collecting results for so many measures that results are overwhelming • Spending more time on measuring things than on doing things

20. Relationship of Assessment to Planning

21. Module Two: Overview • Tour of Assessment Folders on R Drive • Template for Creating Assessment Plans • Sample Assessment Plan • Practice Exercises • Summary • Assignments: Complete by September 5

36. Template for Creating Assessment Plans

37. Template for Creating Assessment Plans

38. Template for Creating Assessment Plans

39. Sample Assessment Plan

40. Sample Assessment Plan

41. Practice Exercises: Generate Mission Statement and Core Functions or Services • Divide into groups of 3 • Choose 1 group member (#1) for whose unit you will write a mission statement and four core functions or services • Member #2 will ask questions, #3 will take notes on responses • 5 minutes: questions and notes • 5 minutes: all three members write mission statement and four core functions or services • 5 minutes: discussion of selected mission statements and core functions/services by all workshop members

42. Practice Exercises: Generate Expected Outcomes • Remaining in the same groups as in the previous exercise and retaining your focus on the same unit for which you created a mission statement and core functions/services, use the same questioning and note-taking process again to generate three expected outcomes for the unit • 5 minutes: questions and notes • 5 minutes: all three members write three expected outcomes • 5 minutes: discussion of selected expected outcomes by all workshop members

43. Practice Exercises: Generate Measures and Target Levels • Remaining in the same groups as in the previous exercise and retaining your focus on the same unit for which you created three expected outcomes, use the same questioning and note-taking process again to generate measures and target levels for each of the three expected outcomes • 5 minutes: questions and notes • 5 minutes: all three members write measures and target levels for the three expected outcomes • 5 minutes: discussion of selected measures and target levels by all workshop members

44. SummarySix Steps of the Assessment Cycle • Mission Statement and Core Functions or Services • Expected Outcomes • Measures • Target Levels • Findings • Action Plans Why We Do Assessment • To improve what we do • To inform decision-making, budgeting, and planning • To focus our efforts • To show students, alumni, employers, and others that we’re doing a good job • To satisfy the expectations of our accreditors

45. Assignments: Complete by September 5 • Download a copy of the assessment template for administrative and support units from the R:\Assessment Folder • Write a first draft of your unit’s 2008-2009 assessment plan that includes • Mission statement • Four core functions or services • Three to five expected outcomes • Measures and target levels for each expected outcome • Set up an appointment with the DIE to review your draft assessment plan during the period September 8 – September 19

46. Assignments: Complete by September 5 Assessment Plans Should be Created by These Individuals and Their Support Staff • Academic Dean • Director of Institutional Effectiveness • Director of Institutional Marketing • Director of Research • Executive Vice President • Human Resources Manager • IRB Manager • Medical Librarian • Marketing Manager • President • Registrar • Vice President of Academic Affairs • Vice President of Finance • Vice President of Student Services