Classroom Assessment: Minute-by-minute and day-by-day

1. Classroom Assessment: Minute-by-minute and day-by-day Dylan Wiliam www.dylanwiliam.net

2. Overview of presentation • Why raising achievement is important • Why investing in teachers is the answer • Why formative assessment should be the focus • Why teacher learning communities should be the mechanism • How we can put this into practice

3. Raising achievement matters • For individuals • Increased lifetime salary • Improved health • Longer life • For society • Lower criminal justice costs • Lower health-care costs • Increased economic growth

4. Where’s the solution? • Structure • Smaller high schools • K-8 schools • Alignment • Curriculum reform • Textbook replacement • Governance • Charter schools • Vouchers • Technology • Computers • Interactive white-boards

5. School effectiveness • Three generations of school effectiveness research • Raw results approaches • Different schools get different results • Conclusion: Schools make a difference • Demographic-based approaches • Demographic factors account for most of the variation • Conclusion: Schools don’t make a difference • Value-added approaches • School-level differences in value-added are relatively small • Classroom-level differences in value-added are large • Conclusion: An effective school is a school full of effective classrooms

6. How important is teacher quality? • How much progress will an average student make when taught by a great teacher (i.e., the best teacher in a group of 50)? • An extra month per year • An extra two months per year • An extra three months per year • An extra four months per year • An extra six months per year

7. Teacher quality • A labor force issue with 2 solutions • Replace existing teachers with better ones? • No evidence that more pay brings in better teachers • No evidence that there are better teachers out there deterred by burdensome certification requirements • Improve the effectiveness of existing teachers • The “love the one you’re with” strategy • It can be done • We know how to do it, but at scale? Quickly? Sustainably?

8. Advanced content matter knowledge • <5% • Pedagogical content knowledge • 10-15% • Further professional qualifications (NBPTS) • <5% • Total “explained” difference • 20-25% The ‘dark matter’ of teacher quality • Teachers make a difference • But what makes the difference in teachers?

9. Cost/effect comparisons

10. The research evidence • Several major reviews of the research • Natriello (1987) • Crooks (1988) • Kluger & DeNisi (1996) • Black & Wiliam (1998) • Nyquist (2003) • All find consistent, substantial effects

11. Types of formative assessment • Long-cycle • Span: across units, terms • Length: four weeks to one year • Impact: Student monitoring; curriculum alignment • Medium-cycle • Span: within and between teaching units • Length: one to four weeks • Impact: Improved, student-involved, assessment; teacher cognition about learning • Short-cycle • Span: within and between lessons • Length: • day-by-day: 24 to 48 hours • minute-by-minute: 5 seconds to 2 hours • Impact: classroom practice; student engagement

12. Unpacking formative assessment • Key processes • Establishing where the learners are in their learning • Establishing where they are going • Working out how to get there • Participants • Teachers • Peers • Learners

13. Aspects of formative assessment

14. Sharing learning intentions • Explaining learning intentions at start of lesson/unit • Learning intentions • Success criteria • Intentions/criteria in students’ language • Posters of key words to talk about learning • eg describe, explain, evaluate • Planning/writing frames • Annotated examples of different standards to ‘flesh out’ assessment rubrics (e.g. lab reports) • Opportunities for students to design their own tests

15. Eliciting evidence of achievement • Key idea: questioning should • cause thinking • provide data that informs teaching • Improving teacher questioning • generating questions with colleagues • closed vs. open or low-order vs. high-order • appropriate wait-time • Getting away from I-R-E • basketball rather than serial table-tennis • ‘No hands up’ (except to ask a question) • ‘Hot Seat’ questioning • All-student response systems • ABCD cards, Mini white-boards, Exit passes

16. Feedback that moves learning on • Key idea: feedback should • cause thinking • provide guidance on how to improve • Comment-only grading • Focused grading • Explicit reference to mark-schemes and scoring guides • Suggestions on how to improve • ‘Strategy cards’ ideas for improvement • Not giving complete solutions • Re-timing assessment • (eg two-thirds-of-the-way-through-a-unit test)

17. Students as owners of their learning • Students assessing their own work • with rubrics • with exemplars • Self-assessment of understanding • Traffic lights • Red/green discs • Colored cups

18. Students as instructional resources • Students assessing their peers’ work • “pre-flight check-list” • “two stars and a wish” • Training students to pose questions/identifying group weaknesses • End-of-lesson students’ review

19. …and one big idea • Use evidence about learning to adapt teaching and learning to meet student needs

20. Keeping Learning on Track (KLT) • A pilot guides a plane or boat toward its destination by taking constant readings and making careful adjustments in response to wind, currents, weather, etc. • A KLT teacher does the same: • Plans a carefully chosen route ahead of time (in essence building the track) • Takes readings along the way • Changes course as conditions dictate

21. Putting it into practice

22. Implementing FA/AfL requires changing teacher habits • Teachers “know” most of this already • So the problem is not a lack of knowledge • It’s a lack of understanding what it means to do FA/AfL • That’s why telling teachers what to do doesn’t work • Experience alone is not enough—if it were, then the most experienced teachers would be the best teachers—we know that’s not true (Hanushek, 2005; Day, 2006) • People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999)

23. A model for teacher learning • Content, then process • Content (what we want teachers to change) • Evidence • Ideas (strategies and techniques) • Process (how to go about change) • Choice • Flexibility • Small steps • Accountability • Support

24. Strategies and techniques • Distinction between strategies and techniques • Strategies define the territory of AfL (no brainers) • Teachers are responsible for choice of techniques • Allows for customization/ caters for local context • Creates ownership • Shares responsibility • Key requirements of techniques • embodiment of deep cognitive/affective principles • relevance • feasibility • acceptability

25. Teacher learning takes time • To put new knowledge to work, to make it meaningful and accessible when you need it, requires practice. • A teacher doesn’t come at this as a blank slate. • Not only do teachers have their current habits and ways of teaching—they’ve lived inside the old culture of classrooms all their lives: every teacher started out as a student! • New knowledge doesn’t just have to get learned and practiced, it has to go up against long-established, familiar, comfortable ways of doing things that may not be as effective, but fit within everyone’s expectations of how a classroom should work. • It takes time and practice to undo old habits and become graceful at new ones. Thus… • Professional development must be sustained over time

26. That’s what teacher learning communities (TLCs) are for: • TLCs contradict teacher isolation • TLCs reprofessionalize teaching by valuing teacher expertise • TLCs deprivatize teaching so that teachers’ strengths and struggles become known • TLCs offer a steady source of support for struggling teachers • They grow expertise by providing a regular space, time, and structure for that kind of systematic reflecting on practice • They facilitate sharing of untapped expertise residing in individual teachers • They build the collective knowledge base in a school

27. How to set up a TLC • Plan that the TLC will run for two years • Identify 8 to 10 interested colleagues • Should have similar assignments (e.g. early years, math/sci) • Secure institutional support for: • Monthly meetings (75 to 120 minutes each, inside or outside school time) • Time between meetings (2 hrs per month in school time) • Collaborative planning • Peer observation • Any necessary waivers from school policies

28. A ‘signature pedagogy’ for teacher learning? • Every monthly TLC meeting should follows the same structure and sequence of activities Activity 1: Introduction & Housekeeping (5-10 minutes) Activity 2: How’s It Going (35-50 minutes) Activity 3: New Learning about AfL (20-45 minutes) Activity 4: Personal Action Planning (10 minutes) Activity 5: Summary of Learning (5 minutes)

29. The TLC leader’s role • To ensure the TLC meets regularly • To ensure all needed materials are at meetings • To ensure that each meeting is focused on AfL • To create and maintain a productive and non-judgmental tone during meetings • To ensure that every participant shares with regard to their implementation of AfL • To encourage teachers to provide their colleagues with constructive and thoughtful feedback • To encourage teachers to think about and discuss the implementation of new AfL learning and skills • To ensure that every teacher has an action plan to guide their next steps • But not to be the AfL “expert”

30. Peer observation • Run to the agenda of the observed, not the observer • Observed teacher specifies focus of observation • Observe teacher specifies what counts as evidence • e.g., teacher wants to increase wait-time • provides observer with a stop-watch to log wait-times

31. Implementations • Current pilots in: • Cleveland Municipal School District, OH • Austin Independent School District, TX • Chico Unified School District, CA • Mathematics and Science Partnership of Greater Philadelphia, PA/NJ • St. Mary’s County Public Schools, MD • State-wide pilot in 10 schools in Vermont • Further details: www.ets.org/klt

32. Summary • Raising achievement is important • Raising achievement requires improving teacher quality • Improving teacher quality requires teacher professional development • To be effective, teacher professional development must address • What teachers do in the classroom • How teachers change what they do in the classroom • AfL/FA + TLCs • A point of (uniquely?) high leverage • A “Trojan Horse” into wider issues of pedagogy, psychology, and curriculum