Interventions are Necessary but Not Sufficient Peter Tymms
Outline • A quick look backwards • The need for interventions • Two recent eye openers • Alternative perspectives • Conclusions • For teaching • For policy making
Historically • Education was an evidence free zone • Experience, opinion and good ideas ruled • Two example examples • Kenneth Baker • Estelle Morris
Interview of Lord Baker with Giles Dinot on 18 February 2011 • Q: One criticism was that the National Curriculum just “taught to test”; is there a problem with that? • A: I went to a Church of England primary school during the war in Southport. I’ve still got the little report books, and there they are, marks out of 100. I was quite clever in those days. I was being tested every term. And I took it home and showed my mother and father.
Estelle Morris: Education Secretary When in office she did not base any action on research but took considerable notice of the Daily Mail
The need for interventions • Consider • £500 million on the National Literacy Strategy with no detectable impact • Etcetcetc • World wide recognition of the issue • Campbell Collaboration • USA • What Works Clearing House • Slavin’s Best Evidence • John Hattie • Tool kit • CEM, Durham
Meta-analysis • Example: use of digital technology to increase literacy in middle schools. (2005)
Points to note • Overall: ES=0.49 • Variation of impact of interventions • Need to know more …. • But we now have a best estimate
First eye opener • Slavin et al 2014 • Cooperative Learning in mathematics has been shown to be effective in the USA • Tried a large RCT in England • And failed “most surprising” • Tried again with a revitalised approach • And failed again
Second eye opening • Lemons et al 2014 • Five RCTs of the efficacy of KG Peer-Assisted Learning on reading program. • Over 9 years involving 2,591 students. • Early results very positive and widely reported • Most recent results • Failed • Team: “shocked and depressed”
Explanations • Slavin et al • “Differences between traditional teaching practices in England and in North America” • “Teaching methods proven to be effective in one culture and system cannot be assumed to be effective in another.” • Lemons et al • “The changed context”. The bar had been raised in KG. • “the change agent - a no-nonsense Chief Instructional Officer” • The controls were different
A Further Complication • We live in a complex interactive world. • Even deterministic worlds are not predictable • http://www.math24.net/double-pendulum.html • But patterns appear (strange attractors) • https://www.youtube.com/watch?v=aAJkLh76QnM • Error propagation is the cause and it is summed up in the Dynamic linear model.
The nature of teaching • Great teaching involves • Planning • The ability to ditch Plan A and go to Plan B • Dealing with idiosyncratic • children • classes • Integrating professional judgement with scientific evidence • The bottom line • Academic progress • Positive non-cognitive progress
Some features of policy making • Making big decisions. • John Maynard Keynes • "When the facts change, I change my mind. What do you do, sir?" • But U-turns are bad news • “The lady’s not for turning” • Are “Reforms as Experiments” conceivable? • Time scales are long
Propositions • Teachers: • must have up-to-date evidence • must interact and collect feedback continuously • must not be held to account for processes • progress is the bottom line • Policy makers should: • accumulate & generate evidence • run pilot projects for proposed policy initiatives • spend large sums only when with evidence • must monitor • be held to account for consequences