850 likes | 1.32k Vues
Student Learning Outcomes. Strategies, Tips, and Tools for Facilitating Learning Outcomes Assessment in Student Services. Jerry Rudmann, Irvine Valley College February 2008. Overview - Student Services. Fine-tuning assessment Tips for writing survey items Focus groups
E N D
Student Learning Outcomes Strategies, Tips, and Tools for Facilitating Learning Outcomes Assessment in Student Services Jerry Rudmann, Irvine Valley CollegeFebruary 2008
Overview - Student Services • Fine-tuning assessment • Tips for writing survey items • Focus groups • Helpful technology tools • Clickers - promote active learning and record SLO information • Rubric generators - a way to measure most anything • PDF Acrobat forms - autoscoring and recording student input • Portfolios - making students responsible and reflective • Scanning - some ideas • Tracking software - organizing all this stuff • Several options / strategies for making SLOs meaningful • SSO versus SLO • Problem focus • Less is better • Use what you already have • Think of SLOs in the context of student development • Qualitative assessment in OK • Other…?
Some Options / Strategies for Making SLOs Meaningful • Address “robust” SLOs (overarching outcomes) • Problem focus • Less is better • Share SLOs with students • Use what you already have • Think of SLOs in the context of student development • Qualitative assessment is OK • SSOs vs. SLOs…
General Tip 1: Problem Focus Approach • What competencies do students have difficulty mastering? • Focus SLO activities on problem areas.
General Tip 2: Keep It Simple But Meaningful • Corollary - Often, less is better.
General Tip 3: Student Development Approach • Student development • Academic self-efficacy (Bandura) • Academic self-regulation • Campus involvement (Astin) • Mentoring professor studies • Student Services DO help student success
Students self-rate their competencies on program or college level learning outcomes. Students’ satisfaction with various student services. Surveys - SLO Uses
Types of Questions Open-ended – respondents answer in own words Closed-ended – respondent limited to a finite range of choices
Types of Questions Open-ended Flexible Hard to code answers Good for preliminary work to finalize a survey Closed-ended Easier to code answers, process and analyze Hard to write good closed-ended items
Item Format Visual Analogue Scale Food in the cafeteria is… Poor_ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _Excellent Likert Scale Food in the cafeteria isoutstanding! SDDN A SA (Strongly Agree) (Disagree) (Neutral) (Agree) (Strongly Agree)
Nine tips for designing and deploying a survey Don’t call it a survey Provide a carefully worded rationale or justification at the beginning Group items by common format Start with more interesting items Put demographic items last Mix in negative wording to catch acquiescence (aka “response set”) Automate scoring when possible If asking for sensitive information, use procedures designed to assure anonymity Always, always, always pilot test first
Survey Administration Methods Face to Face Written Group administration Mail Computerized http://research.ccc.cccd.edu Password protected Validation rules Branching and piping Telephone
Focus Groups Focus groups can be especially insightful and helpful for program and institutional level learning outcome assessment. Have your college researcher provide some background materials. Focus Groups: A Practical Guide for Applied Research By Richard A. Krueger, Mary Anne Casey The RP Group sponsored several “drive in” workshops over the last few years.
Goal for This Section • Technology Uses • Technology ToolsExpected Outcome: Be able to select and use technology-based approaches to assess student learning outcomes
Assessment Challenges • Engaging Students in Self-Evaluation • More Efficient Assessment
Some Technology Tools • Online Rubric Builders • eLumen (SLO Assessment/Tracking) • Classroom Responders (“Clickers”) • Scannable and Online Tests and Surveys • ePortfolios • Adobe Acrobat Forms • Excel Spreadsheets
Rubrics • Way to measure the heretofore immeasurable: products and performances. • A rubric breaks the assessment into important components. • Each component rated along a scale well-labeled scale.
Chocolate Chip Cookie Rubric Chocolate Chip Cookie Rubric
Rubrics are Good! • Facilitate staff dialogue regarding satisfactory performance. • Create a more objective assessment. • Make expectations more explicit to the student. • Encourage metacognitive skill of self-monitoring own learning. • Facilitate scoring and reporting of data.
Online Discussion Rubric http://www.uas.alaska.edu/sitka/IDC/resources/onlineDiscussionRubric.pdf
Design Your Own Rubric • Please work in groups and use the worksheet in your packet to design a scoring rubric for assessing one of the following: • Coffee shops • Syllabi • Customer service at retail stores • Grocery stores • Online courses
Online Rubric Builders • Rubrics to guide and measure learning • Tools • Rubistar http://rubistar.4teachers.org • Landmark Rubric Machine http://landmark-project.com/rubric_builder
Rubistar Art History Rubric Rubistar
Adobe Acrobat Forms • Make form using MS Word • Import form and save as PDF form • Adjust the fields • Add fields to tally sub scores and total scores
eLumen to Assess SLOs • Reduce Time Spent Creating Reports • Assess Course, Program, and/or Degree-Level Outcomes • Share Assessment Rubrics Across Classes and Programs • View Individual or Aggregated Results • Use Online or Offline http://www.elumen.info
Criterion-Based Assessment • Rubrics are attached to each SLO Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
Rubrics Describe Criteria • Writes prose clearly Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
Library of Degree-Level SLOs Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
And Rubrics Link to SLOs Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
from the Science committee from the Biology Department from the faculty committee on critical thinking from the faculty committee on communication skills Science and Gen Ed SLOs/Rubrics Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
Scorecard for All Students in the Course Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
Class Scores by Student Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
Aggregated Data for Course Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
Course Aggregates by Program Excerpted from eLumen: A Brief Introduction by David Shupe, July 2007
Classroom Responders • Engage students • Monitor student understanding • Quickly and easily collect and store assessment data • Use publisher item banks or create your own
Most valuable tip is… • Finding ways to use technology to make SLO and SSO assessment easier and more efficient • Concentrating SLO work on skills students have difficulty mastering • Building SLOs around student development (self-efficacy, goal clarity, etc.)
Renaissance Learningfor clicker training resources http://www.renlearn.com
Scanning Technology • A way to gather survey input from students • A way to test students’ knowledge http://www.scantron.com and http://www.renlearn.com
Surveys and Tests • Online or Scannable • Surveys • Pre and post surveys of student self evaluation of progress • Gather stakeholder (faculty, business community leaders, advisory groups) input on expected learning outcomes • Student satisfaction with service (SSO) • Quizzes/Tests • Practice and graded
Some Survey Software Options • Scannable surveys and quizzes - Optical Mark Reader by Remark (OMR Remark) • Need software and a Fujitsu scanner • Use word processor to create scannable bubble-in surveys or answer sheets. • Produces item analysis output. • Online survey tools • eListen (Scantron Co.) • SelectSurvey.net http://www.classapps.com/SelectSurveyNETOverview.asp
Excel Spreadsheets Example of autoscoring and record keeping in a Japanese Program.