1 / 23

RCTs in the Social Sciences 9 th Annual Conference Professor Carole Torgerson School of Education

What works…and who listens? Encouraging the experimental evidence base in education and the social sciences. RCTs in the Social Sciences 9 th Annual Conference Professor Carole Torgerson School of Education Durham University carole.torgerson@durham.ac.uk September 11th 2014.

trynt
Télécharger la présentation

RCTs in the Social Sciences 9 th Annual Conference Professor Carole Torgerson School of Education

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. What works…and who listens?Encouraging the experimental evidence base in education and the social sciences • RCTs in the Social Sciences • 9th Annual Conference • Professor Carole Torgerson • School of Education • Durham University • carole.torgerson@durham.ac.uk • September 11th 2014

  2. First known RCT “Five seniors, each of whom had a good scholarship record, pleasing personality, excellent health and fine social environment, were chosen to act as personnel counselors for the members of the freshman class, who at the end of the first eight weeks of school happened to be delinquent in scholarship in the School of Mechanical Engineering in I929-30. The 220 delinquent freshmen were divided into two groups by random sampling.” [italics added] • [Walters, 1931]

  3. US legislation: Institute of EducationSciences “Scientifically valid educational evaluation employs experimental designs using random assignment, when feasible, and other research methodologies that allow for the strongest possible causal inferences when random assignment is not feasible.” [page 5] [italics added]

  4. Education Endowment Foundation (EEF) • “...all EEF-funded projects are independently and rigorously evaluated...The impact of projects on attainment will be evaluated where possible, using randomised controlled trials.” [italics added]

  5. Every Child CountsThe independent evaluation • [Torgerson et al, 2013]

  6. Distribution of schools

  7. Early reading policy

  8. Two alternative designs

  9. House of Commons Education and Skills Committee, 18th July 2005 “…in conducting his review, Jim Rose will have the opportunity to draw on the findings of an independent systematic literature review of phonics use in the teaching and application of reading and spelling which we have commissioned from Professor Greg Brooks and Carole Torgerson. This delivers on the public commitment we made…in 2003 to publish an analysis of existing research on phonics teaching methodologies. The aim … is to identify what is known from existing literature about how effective different approaches to phonics teaching are in comparison with each other, including the specific area of analytic versus synthetic phonics.”

  10. Meta-analysis: Forest plot

  11. Publication bias: Funnel plot • Effect Size

  12. Publication bias: Funnel plot • Effect Size

  13. Publication bias: Funnel plot • Missing trials? • Effect Size

  14. Synthetic versus analytic phonics

  15. Policy decision • “I am clear that synthetic phonics should be the first strategy in teaching all children to read.” Ruth Kelly, Secretary of State for Education and Skills Times Mar. 21st 2006 • Sir Jim Rose • Times Mar. 21st 2006 • “The case for synthetic phonics is overwhelming.”

  16. “Synthetic phonics does look promising,” says Carole Torgerson…one of the report's authors. “We found it had a moderate effect compared with whole-language approaches, but the evidence base for this conclusion was 12 relatively small trials, only one of which was UK-based. This would be an ideal time to do a national evaluation by implementing systematic synthetic phonics in some schools and not in others and then comparing the two.” Economist Mar. 26th 2006

  17. “When the UK government recently introduced the ‘synthetic phonics’ method of teaching young children to read, they were told by Carole Torgerson…that they could easily bolster the slim evidence base by randomising which schools joined the programme first.” Financial Times Mar.18th 2010 “In 2007 the Government introduced a new reading strategy for primary schools based on synthetic phonics, which matches sounds to groups of letters. Professor Torgerson urged ministers to start a randomised trial: the introduction of phonics would have been staggered, with schools chosen at random to start it one year or the next. Every child would have received the intervention, but it would have been possible to compare outcomes and establish whether phonics really works.” Times Sept. 24th 2011

  18. Evidence of impact? • ‘KS1 results cast doubt on phonics’ • http://www.tes.co.uk/article.aspx?storycode=6021536

  19. Study characteristics of educational trials 1990-2002 • Torgerson et al, British Educational Research Journal 2006, 31:6.

  20. Annual conference: Randomised controlled trials in the social sciences

  21. Thank you for listening! Rose, J. (2006) Independent review of the teaching of early reading, London: Department for Education and Skills Torgerson, C.J.,Wiggins, A., Torgerson, D.J., Ainsworth, H., Hewitt, C. (2013) Every Child Counts: Testing policy effectiveness using a randomized controlled trial, designed, conducted and reported to CONSORT standards, Journal of Research in Mathematics Education:15(2) Torgerson DJ, Torgerson CJ The design of randomised trials in health, education and the social sciences. Palgrave Macmillan,Basingstoke, 2008 Torgerson, C.J., Brooks, G. & Hall, J. (2006) Literature review of phonics use. London: Department for Education and Skills Torgerson, C. Systematic Reviews, Continuum, London and New York, 2003 Walters, J.E. (1931) The Journal of Higher Education: 2(8), 1931

More Related