1 / 10

Assessment Workshop

On February 5, 2009, Roanoke College hosted an assessment workshop aimed at evaluating intentional learning among first-semester students in pilot GST 101 sections integrated with CCLS. Supported by the U.S. Department of Education and the Teagle Foundation, the workshop focused on direct assessment methods, faculty development, and the implementation of effective rubrics. Participants engaged in scoring randomly selected student papers using established guidelines, fostering reliability through norming and paired reading processes. The initiative promotes improvement in course design and student learning outcomes.

conroy
Télécharger la présentation

Assessment Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment Workshop Roanoke College February 5, 2009 This project is supported by the U.S. Department of Education and the Teagle Foundation.

  2. Our Purpose: • To assess intentional learning in our students during their first semester in experimental sections of GST 101 combined with CCLS • 9 pilot GST 101 sections integrating advising and CCLS • Direct assessment to supplement results of indirect measures—focus groups and surveys (student and faculty) • To provide faculty development

  3. Our Tools • Learning outcomes • Rubrics • Scoring for assessment guidelines • Norming process • Paired reading process

  4. How We Got Here • Learning outcomes and rubrics developed in workshops facilitated by Mary Allen (February, June 2007) • Intentional assignment developed by pilot GST 101 faculty in workshop--July 2008 • Rubrics & procedures pilot-tested by FIPSE staff

  5. Our Task: Score 60 Papers • Intentional learning papers randomly selected from pilot GST 101/CCLS sections • IRB-approved procedures and student consent • Blind scoring—student/professor identities, sections • Workshop schedule--important to get the work done

  6. What Is Scoring for Assessment? • Gives ratings based on essay’s achievement of target learning outcomes (LOs) • Ignores minor errors, length, style, reader preferences and tastes • Readers form a community that “agrees to agree on scoring standards” for a particular artifact (White)

  7. Reading Sessions • Not gradingstudents but assessingcourses or programs • Read and score set of papers with the criteria (rubrics) AND the norming process in mind. • Papers are to be rated as FIRST-YEAR papers. • Avoid idiosyncratic reading and scoring (i.e., personal or disciplinary biases). • Beware of halo effect! (Score on rubric only; filter out other problems) • See “Guidelines” sheet.

  8. This Assessment Is Based on… • Intentional learning assignment • Integrated into GST 101/CCLS classes— graded • Same one used by all pilot sections--PISTACHIO • Based on RC Liberal Learning Goals--BLUE • Intentional learning rubrics—PISTACHIO • Study rubrics—key words, distinguishing levels? • Norming—to “calibrate” measurements & get inter-rater reliability

  9. Paired-Reader Process • Individual readers read paper and decide on an appropriate score on both LOs. • Discuss score with partner. • Ground this discussion on 1) the rubric and 2) then the writing prompt. • Apply the rubric within the parameters of the writing prompt. • Record a consensus score for paper. • Consult with us if consensus is difficult.

  10. Works Cited • “Packet for Designated Assessment Leaders in Departments (GMU).” Writing Across the Curriculum at George Mason University. Nov. 13, 2007. <http://wac.gmu.edu/program/assessing/phase4.html> • White, Edward M. Teaching and Assessing Writing: Recent Advances in Understanding, Evaluating, and Improving Student Performance. 2nd ed. San Francisco: Jossey-Bass, 1994.

More Related