Download
1 / 48
480 likes | 578 Vues
Self-Assessing Locally-Designed Assessments. Jennifer Borgioli Learner-Centered Initiatives, Ltd. Handouts. qualityrubrics.pbworks.com/DATAG. Organizational Focus. Assessment to produce learning… a nd not just measure learning.
E N D
Self-Assessing Locally-Designed Assessments Jennifer Borgioli Learner-Centered Initiatives, Ltd.
Handouts qualityrubrics.pbworks.com/DATAG
Organizational Focus Assessment to produce learning… and not just measure learning.
“Less than 20% of teacher preparation programs contain higher level or advanced courses in psychometrics (assessment design) or instructional data analysis.” Inside Higher Education, April 2009
“The higher the stakes of an assessment’s results, the higher the expectation for the documentation supporting the assessment design and the decisions made based on the assessment results.”
Performance-Based Assessments (PBAs) A performance task is an assessment that requires students to demonstrate achievement by producing an extended written or spoken answer, by engaging in group or individual activities, or by creating a specific product. (Nitko, 2001)
Three Types of Measurement Error Subject effect Test effect Environmental effects
Testing Fatigue Test Familiarity Bias Score Score
Final Eyes isn’t about editing rather “is this what you want the students to see/read?”
Test from Period 1 Test from Period 2
Reliability Indication of how consistently an assessment measures its intended target and the extent to which scores are relatively free of error. Low reliability means that scores cannot be trusted for decision making. Necessary but not sufficient condition to ensure validity.
three general ways to collect evidence of reliability Stability: How consistent are the results of an assessment when given at two time-separated occasions? Alternate Form: How consistent are the results of an assessment when given in two different forms?; Internal Consistency: How consistently do the test’s items function?
Cronbach’s Alpha “In statistics, Cronbach's (alpha) is a coefficient of reliability. It is commonly used as a measure of the internal consistency or reliability of a psychometric test score for a sample of examinees. Alpha is not robust against missing data.”
Item Analysis “This isn’t familiar to me”
How do we ensure alignment and validity in assessment?Degrees of Alignment
If you want to assess your students’ ability to perform, design, apply, interpret. . . . . . then assess them with a performance or product taskthat requires them to perform, design, apply, or interpret.
How many?3-5 3 – 5 standards in a PBA (reflected in rows in the rubric) 3 – 5 items per standard on a traditional test
Fairness and Bias Fair tests are accessible and enable all students to show what they know. Bias emerges when features of the assessment itself impede students’ ability to demonstrate their knowledge or skills.
In 1876, General George Custer and his troops fought Lakota and Cheyenne warriors at the Battle of the Little Big Horn. In there had been a scoreboard on hand, at the end of that battle which of the following score-board representatives would have been most accurate? Soldiers > Indians Soldiers = Indians Soldiers < Indians All of the above scoreboards are equally accurate
Standard Error of Measurement An estimate of the consistency of a student’s score if the student had retaken the test innumerable times
WHEN DESIGNING A PRE/POST PERFORMANCE TASK • the standards and thinking demands must stay the same. • the modality that students express their thinking through must also stay the same. • the content of the baseline and post must be different. • the rubrics for the pre/post will be the same in terms of thinking and modality, but the content dimension will be different.
