Anne Fulkerson, Ph.D. Institutional Research

1. Developing an Assessment PlanOwens Community College Assessment Day WorkshopsNovember 13-14, 2009 Anne Fulkerson, Ph.D. Institutional Research

2. Agenda • What is assessment? • Why assess? • Types of assessment • Basic steps in the assessment planning process • Identifying outcomes • Defining measures • Evaluating measures • Other important considerations • OCC assessment plan template & rubric • Sample assessment plan • How IR can help

3. What is Assessment? “Assessment is the ongoing process of: • Establishing clear, measurable expected outcomes of student learning. • Ensuring that students have sufficient opportunities to achieve those outcomes. • Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations. • Using the resulting information to understand and improve student learning.” Suskie (2004), pg. 3

4. Why Assess? • Demonstrate effectiveness • Improve student learning Establish Learning Outcomes Provide Learning Opportunities Implement Change Measure Learning Adapted from Suskie (2004) and Maki (2004)

5. Types of Assessment Formative—assessment that takes place while learning is in progress (or while a new program is being developed) in order to provide feedback for improvement Summative—assessment that takes place at the end of a course or program to document that learning has occurred or to make judgments about the efficacy of a program

6. Establish Mission Statement

7. Identify Student Learning Outcomes Student Learning Outcomes: Specifiable activities, products, skills, abilities, behaviors, attitudes, or pieces of knowledge that students attain as a result of their involvement in a particular set of educational experiences. • Represent the most important competencies that all students should possess • Do not reflect ALL learning that might occur • Often aligned with accrediting body standards • No magic number, but keep them manageable

8. Expressing Student Learning Outcomes Too vague: Students will demonstrate information literacy skills Too specific: Students will be able to use institutional online services to retrieve information Better: Students will locate information and evaluate it critically for its validity and appropriateness

9. Expressing Student Learning Outcomes Fuzzy Terms: Action Words: Remember Identify Perform Create Define Summarize Explain Discuss Describe Solve Find/Locate Analyze Evaluate Apply Debate/Argue Synthesize Integrate Research Choose Construct Design Develop Organize Use Know Understand Become aware of Appreciate Think critically Demonstrate knowledge Learn Write proficiently

10. Bloom’s Taxonomy (Revised) Thinking Skills Based on an adaptation of Anderson & Krathwohl (2001) http://uwf.edu/cutla/assessstudent.cfm

11. Sample Student Learning Outcomes Not Measurable: Recognize a need for lifelong learning and plan for personal and professional growth Demonstrate an historical knowledge of the symphonic, string orchestra, and chamber ensemble repertoire Measurable: Describe and adopt a plan for ongoing professional development and lifelong learning Describe the historical development of the symphonic, string orchestra, and chamber ensemble repertoire http://uwf.edu/cutla/assessstudent.cfm

12. Identify Opportunities for Students to Achieve Each Learning Outcome

13. Define Measures for Each Outcome Types of Measures: Quantitative—numeric (e.g., test scores, placement rates, GPA, structured surveys) Qualitative—described by words rather than numbers (e.g., interviews, focus groups, observations, rubrics) Direct—require students to display knowledge or skill (e.g., tests, performances, assignments) Indirect—second-hand evidence (e.g., surveys, course evaluations, journal reflections)

14. Word of Caution Don’t measure everything, just because you can Pick the best measures Keep it manageable

15. Determine How Measures will be Evaluated • Criterion-based benchmarks—compares student performance with a pre-established standard. • Norm-referenced benchmarks—compares student performance with a standardized norm or a group of peers. • Value-added approach—compares student performance at two points in time to see if they have improved. • Longitudinal/historical approach—compares current students with prior students.

16. Other Things to Consider • What resources are needed? • Who’s responsible for collecting and analyzing the data? • Build support through participation in the planning process • Assessment plans are perpetual drafts

17. OCC Assessment Plan Templatehttps://intranet.owens.edu/committees/outcomes/index.html

18. OCC Assessment Plan & Report Rubrichttps://intranet.owens.edu/committees/outcomes/index.html

19. Sample Assessment Plans • Owens— https://www.owens.edu/portrait/index.html • College of Charleston, SC— http://spinner.cofc.edu/~oap/docs.html • Supplemental Instruction

20. How IR Can Help • Data support and analysis • Assessment consulting • Survey development • Rubric development • Identifying appropriate metrics • Identifying existing data sources

21. References Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing: Herndon, VA. Palomba, C. & Banta, T.W. (1999). Assessment Essentials. Planning, Implementing, and Improving Assessment in Higher Education. John Wiley & Sons: San Francisco, CA. Suskie, L. (2004). Assessing Student Learning: A Common Sense Guide. Jossey-Bass: San Francisco, CA. Owens Community College, Student Learning Assessment Committee (2009). Building your SLAC assessment plan: Instructions for the SLAC assessment plan template version #1: intranet.owens.edu/committees/outcomes/index.html. University of West Florida, Center for University Teaching, Learning, & Assessment. http://uwf.edu/cutla/Assessres.cfm.