Research on formative assessment practices

1. Research on formative assessment practices 2013 AERA Classroom Assessment SIG Symposium: Advancing Research in Classroom Assessment Dylan Wiliam

2. Section 3: Formative assessment • Four chapters • Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy (Black) • Gathering evidence of student understanding (Heritage) • Feedback and instructional correctives (Wiliam) • Examining formative feedback in the classroom context: New research perspectives (Ruiz-Primo & Li)

3. Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy Paul Black

4. Formative and summative assessment • A simple model of instruction • Clear aims • Planning activities • Interaction • Review of the learning • Summing up • Regulation of learning processes • Proactive • Interactive • Retroactive Tensions arise between summative and formative functions, especially in terms of instruments, methods, and agents

5. The role of teachers • Teachers’ exclusion from summative functions of assessment weakens the kinds of inferences that can be made • But, inclusion of teachers brings its own issues • Poor construct definition makes assessment design more of a discretionary process than it should be • Teachers model their assessments on tests • “Summative drives out formative” • Alignment of curriculum, assessment and instruction • Case studies (Australia, England, Scotland, Sweden)

6. Gathering evidence of student understanding Margaret Heritage

7. Eliciting evidence • Two fundamental purposes of assessment • provide information on current achievement • inform future instruction

8. Sources and quality of evidence • Sources of evidence • What learners say, write, make, or do (Griffin 2007) • Interactions (questions, discussions) • Tasks • Directed (model-eliciting activities) • Partially directed (activities) • Undirected (observations) • Quality of evidence • Construct underrepresentation • Construct-irrelevant variance • systematic • random (i.e., unreliability)

9. Learning progressions • “What gets better when someone gets better” • Two key aspects • progressions lay out in successive steps, increasingly more sophisticated understandings of core concepts and principles in a domain, and • progressions describe typical development over an extended period of time. • Two approaches to their development • Top-down (theoretical) • Bottom-up (empirical)

10. Feedback and instructional correctives Dylan Wiliam

11. Reviews of research on feedback • Fuchs & Fuchs (1986) • Natriello (1987) • Crooks (1988) • Bangert-Drowns et al. (1991) • Dempster (1991, 1992) • Elshout-Mohr (1994) • Kluger & DeNisi (1996) • Black & Wiliam (1998) • Nyquist (2003) • Brookhart (2004) • Allal & Lopez (2005) • Köller (2005) • Brookhart (2007) • Wiliam (2007) • Hattie & Timperley (2007) • Shute (2008)

12. Information about the learning process… … that teachers can use for instructional decisions… …and students can use to improve performance… …which motivates students Feedback: an evolving concept (Brookhart, 2007) Conceptualization Source(s) • Scriven (1967) • Bloom, Hastings and Madaus (1971) • Sadler (1983; 1989) • Natriello (1987); Crooks (1988); Black and Wiliam (1998); Brookhart (1997)

13. Back to the future “These considerations of utility and alternative interventions suggest that even [a feedback intervention (FI)] with demonstrated positive effects on performance should not be administered whenever possible. Rather, additional development of [feedback intervention theory] is needed to establish the circumstance under which positive FI effects on performance are also lasting and efficient and when these effects are transient and have questionable utility. This research must focus on the processes induced by FIs and not on the general question of whether FIs improve performance—look at how little progress 90 years of attempts to answer the latter question have yielded.” (Kluger & DeNisi, 1996 p. 278)

14. Dual pathway theory (Boekaerts) “It is assumed that students who are invited to participate in a learning activity use three sources of information to form a mental representation of the task-in-context and to appraise it: (1) current perceptions of the task and the physical, social, and instructional context within which it is embedded; (2) activated domain-specific knowledge and (meta)cognitive strategies related to the task; and (3) motivational beliefs, including domain-specific capacity, interest and effort beliefs.” (Boekaerts, 2006 p. 349)

15. As a result of the appraisal, the student activates energy and attention along one of two pathways • the growth pathway (increasing competence) • the well- being pathway (prevent harm, threat or loss) • Integration of other theories • Mindset (Dweck, 2000) • Mastery and performance goals (Dweck, 2000) • Interest (Hidi & Harackiewicz, 2000) • Self-regulated learning (Deci & Ryan, 1994)

16. Examining formative feedback in the classroom context:New research perspectives Maria Araceli Ruiz-Primo and Min Li

17. Formative assessment in the classroom • Updated review of research on feedback in classroom learning • Vast majority of studies are poorly designed • No control groups • Feedback effects evaluated in the instructional session • No evidence of pedagogical orientation (e.g., orientations to the purpose of feedback, or how it could be used)

18. A new, expanded, notion of feedback • Feedback should: • Be seen as a process guided by the learning goals towards which the teacher and students work • Actively involve students in the process • Be considered as an instructional scaffold that goes beyond written or oral comments. • Be specifically intended to improve learning outcomes • Ensure its usefulness by making feedback accessible and practical. • Consider different sources of information • Demonstrate, over time, alignment with a learning trajectory

19. Future priorities • Defining feedback • Understanding variability in feedback practices • Understanding feedback in use • Reviewing feedback research from new theoretical understandings

20. Major unresolved questions • Can teachers be involved in the summative assessment of their students without compromising the potential of formative assessment to improve achievement? • How can learning progressions be developed? • How can we better theorize the impact of feedback interventions?