The role of teachers in the assessment of learning

1. The role of teachers in the assessment of learning Outcomes of the Assessment Systems for the Future project of the Assessment Reform Group Funded by the Nuffield Foundation

2. Assessment Systems of the Future Project • Funding: Nuffield Foundation • Focus: summative assessment of school pupils and the role that assessment by teachers can take in this • Duration: Sept 2003 to June 2006 • Method: 5 expert seminars and 2 wider sets of consultation/dissemination conferences

3. Causes for concern • ‘High stakes’ testing causes frequent testing and ‘teaching to the test’ • Frequent testing affects children’s motivation for learning • Teaching to the test restricts the curriculum and teaching methods • External testing encourages more internal testing • Reduces opportunities for formative assessment

4. Result • Too much testing • Validity is low (we do not get good information about learning, only about how good the children are at taking tests) • High cost • Negative impact on learning and teaching • No evidence that testing ‘drives up standards’

5. Costs • 165 = hours per year spent by Year 6 teachers on tests in England • 222 = hours per year on other assessment and reporting activities • 84 = hours per year spent by Year 6 children on taking tests • £240m = cost of time spent on tests and examinations (primary + sec) • £610m = total direct and indirect cost of tests and examinations (primary + sec) (2003 estimates based on surveys)

6. Properties of summative assessment • What learning outcomes are assessed (validity) • How accurately and consistently they are assessed (reliability) • Impact of the assessment (on teachers, learners, the curriculum and teaching methods) • Cost (time for teaching and learning and other resources)

7. Properties depend upon… • How the assessment is carried out • By testing • By teachers • Combination • What the results are used for • Information on individual students • Evaluation/ accountability of teaching and teachers (groups) • Monitoring the system (groups)

8. Uses of summative assessment in a national assessment system Individual results: • Internal to the school/college (records, reports, guidance) • External to the school (certification, selection, meeting statutory requirements) Group results: • Evaluation (teachers, schools and local authorities) • Monitoring (year on year comparisons of averages at regional or national level)

9. What ought to be assessed? • The full range of understanding, skills, competencies and attitudes that are the goals of a modern education • In particular • Learning with understanding, shown in ability to apply knowledge rather than recall facts • Understanding learning, being able to reflect on the learning process and aware of how to learn

10. About reliability • Tests and examinations are assumed to be reliable, but • Strong research evidence that up to 30% pupils are given incorrect grades or levels • Teachers’ own tests may be even less reliable and result in wrong decisions being made

11. Dominant impact of ‘external’ assessment Reasons: • Often ‘high stakes’ for individual students • ‘High stakes’ for teachers when results used for accountability Serious impact because of: • Influence on teachers’ own summative assessments • Influence on the use of formative assessment • Preference for using tests & examinations

12. The need for change • If current ways of assessing outcomes are not including important learning outcomes, are not as reliable as assumed and have negative impacts, then we must look for alternatives • Most users of assessment want evidence of both academic and non-academic achievements

13. Some advantages of using assessment by teachers • Potential for the full range of goals to be included when teachers collect evidence as part of normal work with students • Can relieve the pressure of terminal tests and examinations • Teachers can use information about students formatively as well as summatively • Can release time and other costs for alternative use

14. Key points: from experience in countries in & outside the UK • Where teachers have become dependent upon external tests - at least two years of trial and evaluation needed for the value of new practices to be properly judged •  Top-down approaches are not as effective as ones involving teachers in building up necessary procedures •  Effective moderation and professional development are key factors in establishing confidence

15. What about the disadvantages? • Teachers’ judgements often perceived as being unreliable • Increase in work load for teachers • Can lead to the same distortion of teaching as testing if used for high stakes accountability

16. What can be done - about reliability ? • Evidence of unreliability of teachers’ judgements comes from studies where no guidance or training was given • Research shows that, with appropriate pd and moderation procedures, teachers’ assessment can be highly reliable • The training and moderation required have benefits beyond reliability of results; they enhance the quality of teaching and learning • Access to a bank of well designed tasks

17. What can be done – about workload ? • Teachers already spend a large proportion of their time on assessment • In England, this amounts to • nearly 400 hours/year in Y6, of which almost half is on internal or external tests or test preparation; for pupils about 84 hours/year or almost four weeks • in Y7, 8 & 9 the time is about 100 hours per class for subject teachers; for pupils 20 hours per subject per year • Saving half of this would more than compensate for extra time on moderation

18. What can be done – about the ‘high stakes’ ? • Judging schools based on test results • does not reflect all that a school strives for • results in disproportional attention to ‘borderline’ pupils • encourages teaching to the test • More valid and reliable methods for school evaluation taking context into account should be used • High stakes should be transferred to how well school meets a range of goals

19. Conclusions • We want a system capable of providing reliable information about a wide range of pupil competences • Systems depending primarily on test results do not provide this • No approach to summative assessment is without problems and some negative impacts on pupils and teachers • Research evidence suggests that a system making appropriate use of assessment by teachers has far fewer negative consequences than one based on tests …contd

20. …continued • Procedures are needed to help teacher understand and use criteria consistently also benefit teaching • Assessment procedures need to be transparent to gain the confidence of users • Summative assessment should be carried out only when needed to report progress, at other time assessment should have a formative function • Procedures should enable evidence used for formative assessment to be reviewed against summative criteria …Contd

21. …continued • To reduce the ‘high stakes’ for schools that lead to distortion of the curriculum and teaching • Systems for school accountability should not depend solely on pupils achievement results • To provide more valid and useful information about national and regional standards • Achievement should be monitored by assessing a sample of pupils using a wide evidence base

22. Implications for action: policy • Recognise the short-coming of current assessment policies in relation to validity, reliability, cost and impact on the curriculum, and teaching • Consider replacing national testing, where it exists, by moderated teachers’ judgments • Divert resources from tests into quality assurance and enhancement • Review the role of teachers’ assessment in examinations for 16-19 year olds • Promote openness in assessment procedures

23. Implications for action: school management • School assessment policy should require summative assessment only when necessary for reporting progress, not more frequently • Establish, maintain and protect time for quality assurance procedures for internal summative assessment, • Ensure parents understand the formative and summative use of assessment • Resist pressure for test data and encourage positive discourse about assessment

24. Implications for action: teachers • Ensure that assessment is always used to help learning • Only when a summative assessment report is needed, ensure that best evidence is reliably judged against relevant criteria • Involve pupils in self-assessment and help them to understand the assessment criteria • Take part in moderation of judgments and other quality assurance procedures. • Use tests only when most appropriate, not as routine.

25. Implications: inspectors and advisers • Help schools to establish assessment policies that encourage formative use of assessment and moderation procedures for summative assessment • For summative assessment encourage use of a range of pupils’ achievements • Ensure that appropriate professional development in assessment is available • Help schools set targets based on self-evaluation across a range of data not only on levels achieved by pupils

26. Implications: teacher educators • Ensure that courses allow adequate time for • discussion of the different purposes of assessment and the uses made of assessment data • trainees and participants to identify, sample and evaluate different ways of gathering evidence of pupils’ performance • giving experience of generating assessment criteria linked to specific learning goals • considering evidence of bias and other sources of error in assessment and how they can be minimised

27. Some references See the ARG website for information and reports from the ASF project: www.assessment-reform-group.org Recent relevant ARG publications • The Role of Teachers in the Assessment of Learning (2006) available on the ARG website and from the CPA Office, Institute of Education, London WC1H 0AL • Gardner, J (Ed) Assessment and Learning (2006) London: Sage

28. Formative and summative assessment working together Learning activities Criteria for reporting levels Lesson goals a b c d etc Moderation