1 / 33

Refining Your Assessment Plan: Session 4

Refining Your Assessment Plan: Session 4. Use of the Results Ryan Smith and Derek Herrmann University Assessment Services. Refining Your Assessment Plan: Session 4. Introductions. Introductions. Ryan Smith, Director, UAS Derek Herrmann, Coordinator, UAS Assessment Advisory Council (AAC)

madra
Télécharger la présentation

Refining Your Assessment Plan: Session 4

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Refining Your Assessment Plan: Session 4 Use of the Results Ryan Smith and Derek Herrmann University Assessment Services

  2. Refining Your Assessment Plan: Session 4 Introductions

  3. Introductions • Ryan Smith, Director, UAS • Derek Herrmann, Coordinator, UAS • Assessment Advisory Council (AAC) • Assessment Academy Team

  4. Introductions • Who you are • In which Department/School you work • Why you are here

  5. Refining Your Assessment Plan: Session 4 Overview of PRAAP

  6. Overview of PRAAP • Process for the Review of Academic Assessment Plans • Two years before Program Review and one year before Self-Study • AAC members and UAS staff • Use an established rubric

  7. Overview of PRAAP

  8. Refining Your Assessment Plan: Session 4 Use of the Results

  9. Use of the Results • Who is involved? • Chairperson/Director (or Assistant/Associate) • Program Coordinator • Faculty • Individuals • Committees

  10. Use of the Results • Where is it done? • Department/School meetings • Retreats (general or assessment) • Informal conversations

  11. Use of the Results • Examining the data • Rubrics • A scoring guide; a list or chart that describes the criteria used to evaluate (or grade) completed student assignments • Using a rubric to evaluate (or grade) assignments… • Makes your life easier • Improves student learning

  12. Use of the Results • Holistic rubric: short narrative descriptions of the characteristics of different levels of work

  13. Use of the Results • Checklist rubric: simple list indicating the presence of the things you’re looking for in an assignment

  14. Use of the Results • Rating scale rubric: checklist with a rating scale added to show the degree to which the things you’re looking for are present in an assignment

  15. Use of the Results • Descriptive rubric: replaces the checkboxes of rating scale rubrics with brief descriptions of the performances that merit each possible rating

  16. Use of the Results • Examining the data • Content analysis • Involves making sense of the material being reviewed • Summarize common themes and the extent of consensus concerning those themes • Should be approached with an open mind and a willingness to “hear” what respondents are saying

  17. Use of the Results • Examining the data • Content analysis • Should begin with a clear understanding of the goals associated with the review • Sometimes coding categories are predetermined; sometimes, they emerge as materials are reviewed

  18. Use of the Results • Inter-rater reliability • Indicates the extent of agreement among different reviewers • Generally want to verify that scores based on subjective judgments are reliable (consistent)

  19. Use of the Results • Inter-rater reliability • Two approaches • Have at least two people independently review data, compare notes, and come to agreement • Have reviewers work together on all decisions, collaborating as they go to resolve discrepancies

  20. Use of the Results • Establishing standards • Generally set own standards • Should take the campus and program mission into account when establishing standards

  21. Use of the Results • Establishing standards • Sometimes faculty want to know how their students compare to students on other campuses • Can set expectations based on the relative performance of their students

  22. Use of the Results • Summarizing the results • Frequency table: a visual depiction of data that shows how often each value occurred

  23. Use of the Results

  24. Use of the Results • Summarizing the results • Frequency table: a visual depiction of data that shows how often each value occurred • Measures of central tendency • Mean: arithmetic average • Median: 50th percentile • Mode: most common score

  25. Use of the Results

  26. Use of the Results • Summarizing the results • Qualitative summary: “The majority of respondents indicated that they hold positive attitudes toward their degree program.”

  27. Use of the Results • Sharing the results • Be open, honest, balanced, and fair • Understand your audiences and their needs • Help your audiences see the big picture

  28. Use of the Results • Sharing the results • Celebrate good results • Address areas needing improvement • Goals and/or outcomes? • Direct and/or indirect evidence? • Curriculumand/or instruction? • Incorporate results into planning and decision-making processes

  29. Refining Your Assessment Plan: Session 4 Final thoughts

  30. Final Thoughts • Goals and learning outcomes define how students demonstrate their mastery of what faculty want them to learn • A cohesive curriculum is aligned with goals and learning outcomes

  31. Final Thoughts • Assessment involves collecting direct and indirect evidence concerning student development • Embedding assessment in the curriculum has many advantages • Assessment should not focus on individual students or faculty

  32. Final Thoughts • A recurring theme in assessment is collaboration • Three major criteria apply to assessment; it should be meaningful, manageable, and sustainable

  33. Refining Your Assessment Plan: Session 4 Questions, comments, and discussion

More Related