Assessing Your Assessments: The Authentic Assessment Challenge Dr. Diane King Director, Curriculum Development School of Computer & Engineering Technologies firstname.lastname@example.org March 13, 2009
SESSION OBJECTIVES • How to identify and define learning outcomes • How to align assessment with learning outcomes • How to assess learning outcomes
What is assessment? • Why do we assess? • What do we assess? • Is it working?
What is Assessment? • Assessment is the systematic gathering and analyzing of information (excluding course grades) to inform and improve student learning or programs of student learning in light of goal-oriented expectations • Effective Grading: A Tool for Learning and Assessment Barbara Walvoord and Virginia Johnson Anderson Jossey-Bass, 1998
What is Assessment? • “Techniques used to analyze student accomplishment against specific goals and criteria” • UW-Madison Assessment Manual
Authentic Assessment • …simulate or replicate important real-world challenges (Wiggins and McTighe) • …measures a student’s ability to perform a real world task (Northern Illinois University)
Authentic Assessment • A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills. • Student performance on a task is typically scored on a rubric to determine how successfully the student has met specific standards. (Mueller)
Why Do We Assess? • To know if students can apply what they have learned in authentic situations • (Mueller)
Why Do We Assess? • To Improve • To Inform • To Prove • UW-Madison Assessment Manual: Using Assessment for Academic Program Improvement April 2000 Program Improvement Revised April 2000
What Do We Assess? • How do we know what to assess?
Where To Start? • At the END!!! • …with the OUTCOME!!!!
Outcome • Specific accomplishments to be achieved • (Hatfield)
LEARNING OUTCOME • What do I want my students to KNOW as a result of taking this course? = KNOWLEDGE • What do I want my students to be able to DO as a result of taking this course? • = SKILLS
Learning Outcome • HOW are my students going to APPLYwhat they have learned in here, out there? • = PERFORMANCE
Learning Outcomes “As a result of this program/course, students will be able to:….” • College wide • Discipline specific • Program level • Course level
Backward Design • 1. Identify desired results • 2. Determine acceptable evidence • 3. Plan learning experiences and instruction • Wiggins and McTighe
Desired Result • Participants will identify one key learning outcome for a course they teach • Participants will design one authentic assessment activity to measure
Performance Task • Real world application of knowledge or skill
Measurable Performance • What criteria will you use to measure how the student performed the task?
Rubric • A scoring scale used to assess student performance along a task-specific set of criteria – Mueller
Types of Assessment Methods Self-/peer assessments Collaborative project Performances Experiments/research studies/visual representations Case studies Service learning Internships – logs/journals/reflections • Portfolio - Digital/electronic/web-based • Special projects/capstones • Journals/learning logs/digital learning records • Conferences/interviews • Oral examinations
Types of Assessment Methods Education plan Faculty critiques Documentation of service learning experiences • Anecdotal observations • Student generated items • Industry certifications that show competencies • Conferences/interviews
Direct Assessment • Comprehensive Exam • Major Project • Student Portfolios • Pre-test/Post-test • Embedded questions • Performance assessment • Senior Portfolio • Embedded questions in exams • Senior seminar
INDIRECT ASSESSMENT • Internship Evaluation • Alumni, Employer, Graduate Exit survey • Student scholarly achievement • Examination of information contained in department's own database • Student Satisfaction Survey • Student Course Evaluation • Community perception of program • Student graduation/retention rate • Focus group discussions
Classroom Assessment Techniques Informal/Immediate Feedback • The One-Minute PaperThe minute paper is a short exercise in which you ask students to write for one minute on two questions: What was the most important thing you learned today? and, what question still remains in your mind after today's class? • The Muddiest PointThis assessment method is similar to the minute paper. Students write a one-minute essay on the muddiest point that remains in their minds after a lecture, demonstration, or presentation. • The One-Sentence SummaryIn this method, students write and then discuss a one-sentence summary that describes the content covered in class. • Directed ParaphrasingIn directed paraphrasing, students summarize a concept or procedure in two or three sentences. • Applications CardsHere, the instructor asks students to think of real-world applications of topics discussed in class.
ASSESSMENT VALIDATION • 1. Are the following elements present in the instrument? • Learning Outcome • Product or performance-based assessment • Real-world relevance • Application of knowledge • Alignment with criteria on rubric
ASSESSMENT VALIDATION • 2. Will the assessment task elicit the learning outcome(s) being assessed? • 3. Will the assessment task elicit a full expression of ability at a level appropriate to the students’ general education learning experience? • 4. Does the assessment task require students to demonstrate proficiency of the learning outcome (understanding and ability)?
ASSESSMENT VALIDATION • 5. Does the assessment task integrate knowledge and skills gained throughout the students’ general education learning experience? • 6. Does the assessment task permit students some individual difference in meeting the performance criteria? • 7. On a scale (from disconnected to fully integrated), does the assessment task encourage students to integrate competencies with each other?
ASSESSMENT VALIDATION • 8. Does the assessment task assess both knowledge and ability? • 9. Is the assessment task authentic; that is, does it involve students in issues they see as vital concerns or engage them with problems related to the real world? • 10. Will the assessment task produce results that can provide diagnostic, structured feedback on students’ attainment of the targeted learning outcome?
Workshop Applied Learning Tasks Introduction to Computers/Computer Literacy • 1. Identify One Learning Outcome • 2. Develop one authentic assessment task to measure that outcome