Cultures of assessment: What’s the big deal?

1. Cultures of assessment: What’s the big deal? Liz Hamp-Lyons Faculty of Education, The University of Hong Kong

2. Cultural continuum • Norm-referenced testing • Criterion/standards-referenced testing • School-based assessment

3. A norm-referenced test (NRT) • Is a solution based on a common denominator or ‘yardstick’ • Divides people according to predetermined quantities • Assumes that with large numbers, “behaviour” or “performance” falls into randomly-normal distribution: this is called the Bell curve:

4. Bell curve

5. NRT and the “standards” movement Standards-based assessment is NOT the same as norm-referenced assessment—the names are confusing! The standards movement is more closely related to criterion-referenced assessment: A criterion-referenced assessment has set criteria or standards to be achieved and therefore the pass-fail aspect of the assessment is the most important aspect. In theory all students could pass the assessment or alternatively all students could fail the assessment.

6. What is a criterion-referenced test (CRT)? • A swimming test (drown or live) • A driving test (pass or fail: mastery learning) • A diving contest (how well did she perform the classic two-and-a-half-twist?: judgemental/holistic) • A vegetable-growing contest (measured against a criterion or some criteria: How big? How healthy? How many fruits? Taste? Colour? Shape?)

7. Criteria and standards • In the previous slide, the last 2 examples depend on the use of standards and not just yes/no decisions. • This requires the fixing of the standards, documents to explain the standards, and a way of helping judges (in the case of SBA, this means teachers) to be able to understand and apply the standards.

8. Standards-referenced assessments (SRAs) • include items that are directly relevant to the learning outcomes to be measured, without regard to whether the items can be used to discriminate among students. No attempt is made to eliminate easy items or alter their difficulty. If the learning tasks are easy, then test items will be easy. • the goal is to obtain a description of the specific knowledge and skills each student can demonstrate, and the levels at which they can do so. This information is useful for planning both group and individual instruction.

9. Criterion-referenced scoring • There are multiple ways to score a criterion-referenced assessment. These include: • checklists • rating scales • grades • rubrics • percent accurate • We think rating scales work best for complex human performances like language. Rating scales are in fact standards-referenced assessment.

10. In the HKCEE 2007 English SBA • We use carefully- and specifically- developed rating scales. However, we also use rubrics in the form of sample assessment tasks. The sample tasks with their full preliminary detail (the rubric) provides a kind of ‘template’ for new task development. • Standards are important in every aspect of the SBA.

11. Why are the effects of NRTs and criterion-referenced or standards-referenced assessments different? • Because NRTs work at the large scale not the individual or small group. • The large scale needs to discriminate, to separate, to categorize and label. It needs to find the general, the common, the group identifier, the scaleable, the replicable.

12. But • CRT and standards-referenced assessment accept that each person is an individual, and should be allowed/enabled to show their best. • SRAs work well in contexts, like classrooms, where the individual, the changing, the changeable is what’s valued. • The classroom is also where many individuals will display equal but different levels of competence each in their own way.

13. School-based assessment • Is learner-focused • Encourages critical thinking • Focuses on knowledge creation • Is tailored for students • Values teachers’/students’ voices • Leads to assessment of learning

14. Traditional testing and exam culture • Is score-focused • Encourages rote learning • Focuses on knowledge production • Has a ‘one size fits all’ philosophy • Values rule-makers’ voices • Leads to ‘teaching to the test’

15. Which works better as assessment-for-learning? • [see Activity 3 Module 2: focus is on the question that heads this slide. • Trainers add materials here if desired, otherwise delete this slide before presentation]

16. Choosing an assessment culture for assessment for learning • Teachers find themselves working with a range or continuum of assessment cultures: • at one end is a learning culture, in which assessment is primarily shaped by considerations of learning and teaching; • at the other end is an exam culture, in which classroom assessment is seen as simply preparation for an externally set and assessed examination. These two cultures stem from differing ideologies that value very different things. • Teachers can find ways of using summative assessment—exams—formatively, in assessment for learning.

17. Tools for assessment for learning • The rating scales [such as in slide 18] can be used summatively to make and record judgements of students’ actual performance on an SBA task; • But they can also be used formatively to provide clear and specific feedback to individual students; and to teach them how to self-assess and to give helpful feedback to classmates.