Quality in formative assessment

1. Quality in formative assessment Symposium at the 2008 meeting of AERA, New York, NY Discussant: Dylan Wiliam www.dylanwiliam.net

2. Formative assessment Formative assessment involves the creation of, and the capitalization upon, moments of contingency in instruction

3. What gets formed? • Long-cycle • Span: across units, terms • Length: four weeks to one year • Impact: Student monitoring; curriculum alignment • Medium-cycle • Span: within and between teaching units • Length: one to four weeks • Impact: Improved, student-involved, assessment; teacher cognition about learning; before end-of-unit adjustment to instruction. • Short-cycle • Span: within and between lessons • Length: • day-by-day: 24 to 48 hours • minute-by-minute: 5 seconds to 2 hours • Impact: classroom practice; student engagement

4. For whom? • Individual students • Groups of students on whom data were collected • Other groups of students

5. Theory of action • Evidence of student learning is elicited (by teachers, peers, or by the learners themsleves) • This evidence is intepreted in terms of a theory of learning • Decisions about next steps in instruction are taken that are either better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited

6. Competing definitions of quality • What is effective formative assessment? • Data is generated for the purpose of improving instruction (intention) • Data is generated that is likely to improve instruction (prediction) • Under any conditions • Under commonly encountered conditions • Under specified conditions • Data is generated that actually improves instruction (action)

7. Evidence collection • Kinds of assessment in support of learning • Monitoring • Whether the required learning is taking place • Diagnostic • When the required learning is not taking place, what is not being learned • Formative • When the required learning is not taking place, what needs to be done to improve the situation • Evidence collection must be driven by a theory of learning

8. Learning progressions • “Unpacking” state standards • Horizontal unpacking—operationalization for testing • Vertical unpacking—operationalization for learning • Learning progressions • Are rarely independent of curriculum • Require both empirical and theoretical rationales

9. The role of content knowledge • Knowing what the correct answer is • Knowing what incorrect answers indicate • Knowing what instructional activities are likely to move learning on • Knowing what issues are worth bothering about

10. Specific comments • Herman & Choi • An important existence proof • A particularly well understood, and curriculum-independent, domain • How much accuracy is needed? • Heritage, Kim & Vendlinski • Importance of teacher x principle x task effects • Planning next instruction is hard • Requires an “anatomy of quality” • Wylie & Ciofalo • Competing priorities • Minimizing false positives • Interpretability of true negatives