Refining Your Assessment Plan: Session 2

1. Refining Your Assessment Plan: Session 2 Direct Evidence of Student Learning Ryan Smith and Derek Herrmann University Assessment Services (UAS)

2. Refining Your Assessment Plan: Session 2 Introductions

3. Introductions • Ryan Smith, Director, UAS • Derek Herrmann, Coordinator, UAS • Assessment Advisory Council (AAC) • Assessment Academy Team

4. Introductions • Who you are • In what Department/School you work • Why you are here

5. Refining Your Assessment Plan: Session 2 Overview of PRAAP

6. Overview of PRAAP • Process for the Review of Academic Assessment Plans • Two years before Program Review (and one year before Self-Study) • AAC members and UAS staff • Use an established rubric

7. Overview of PRAAP

8. Refining Your Assessment Plan: Session 2 Direct Evidence of Student Learning

9. Direct Evidence of Student Learning • Tangible, visible, self-explanatory, and compelling evidence of exactly what students have and have not learned

10. Direct Evidence of Student Learning • Performance assessments and traditional assessments • Traditional assessments • Multiple-choice tests, essay tests, and oral examinations • Usually designed only to collect assessment information, not give students a learning opportunity

11. Direct Evidence of Student Learning • Performance assessments and traditional assessments • Performance assessments • Ask students to demonstrate their skills rather than relate what they have learned through traditional tests • Increasingly popular because they merge learning and assessment

12. Direct Evidence of Student Learning • Embedded and add-on assessments • Embedded assessments – program, general education, or institutional assessments that are embedded into course work • Add-on assessments – beyond course requirements

13. Direct Evidence of Student Learning • Quantitative and qualitative assessments • Quantitative assessments – use structured, predetermined response options that can be summarized into meaningful numbers and analyzed statistically • Qualitative assessment – use flexible, naturalistic methods and are usually analyzed by looking for recurring patterns and themes

14. Direct Evidence of Student Learning • Effective assessment formats • Paper/thesis – students develop an argument and support it with information or data they have gathered • Project – students complete an assignment over a prolonged period of time • Development of a product – project whose focus is on the development of a tangible product

15. Direct Evidence of Student Learning • Effective assessment formats (cont’d) • Performance – students prepare and present a performance of a valued activity • Exhibition – a project, product, or performance that is presented to judges and defended or debated with them • Case study/critical incident – students are given a realistic example of an application in their field

16. Direct Evidence of Student Learning • Effective assessment formats (cont’d) • Clinical evaluation – students perform a professional service in a real-life setting • Oral exam – students answer spontaneous questions put to them by experts • Interview – similar to an oral exam but the forum in which it is carried out may not be as public or involve as many questioners as an oral exam

17. Direct Evidence of Student Learning • Effective assessment formats (cont’d) • Comprehensive exam – students complete a time-limited essay test that requires them to organize and present central ideas, facts, and concepts in response to questions • Portfolio – students gather examples of their work, write about aspects of their learning and achievement, and include their written reflections

18. Direct Evidence of Student Learning • Effective assessment formats (cont’d) • Portfolios • Showcase portfolio – document the extent of learning by featuring the student’s best work • Developmental portfolio – designed to show student progress and includes evidence of growth by comparing products from early and late stages of the student’s academic career • Collective portfolio – collections of student work that are created by faculty for assessment purposes

19. Refining Your Assessment Plan: Session 2 Curriculum Mapping (Part 2)

20. Curriculum Mapping • Analyze learning opportunities by mapping learning goals against courses

21. Curriculum Mapping

22. Curriculum Mapping

23. Curriculum Mapping • Transcript analysis – review a sample of transcripts of graduating students to learn courses they choose and when they take them

24. Curriculum Mapping • Syllabus analysis – can help determine if students have enough assignments and classwork to achieve each of the intended learning goals of the course, including the program, general education, or institutional goals that course is intended to support

25. Refining Your Assessment Plan: Session 2 Rubrics

26. Rubrics • A rubric is a scoring guide: a list or chart that describes the criteria you and perhaps your colleagues will use to evaluate or grade completed student assignments

27. Rubrics • Using a rubric to grade student assignments… • Makes your life easier • Improves student learning

28. Rubrics • A checklist rubric is a simple list indicating the presence of the things you’re looking for in a completed assignment

29. Rubrics • A rating scale rubric is a checklist with a rating scale added to show the degree to which the things you’re looking for are present in completed assignments

30. Rubrics • Descriptive rubrics replace the checkboxes of rating scale rubrics with brief descriptions of the performances that merit each possible rating

31. Rubrics • Creating effective rubrics • Look for models • List the things you’re looking for • Leave room for the ineffables and the unexpected

32. Rubrics • Creating effective rubrics (cont’d) • Create the rating scale • Create at least three levels • Label each level with names, not just numbers • If you are developing a descriptive rubric, fill in its boxes • Try out the rubric

33. Refining Your Assessment Plan: Session 2 Questions, comments, and discussion