Assessment standards: a potential role for Subject Networks

1. Assessment Standards Knowledge exchange Assessment standards:a potential role for Subject Networks Dr Chris Rust, Deputy Director ASKe CETL Directorate: Margaret Price, Jude Carroll, Berry O’Donovan, Karen Handley and Chris Rust

2. Origin • Weston Manor Group, November 07 • 40 National and International Experts in Assessment • Triggered by the Burgess Report, NSS Results • Two days of discussions Outcome • Six tenet manifesto for change to assessment practice related to standards • Sent to HEFCE, HEA, QAA, UUK, GuildHE, NUS • Lead article in THES – April 24, 08 • Submission to Parliamentary select committee

3. Focus on standards “it cannot be assumed students graduating with the same classified degree from different institutions, having studied different subjects, will have achieved similar standards; it cannot be assumed students graduating with the same classified degree from a particular institutions, having studied different subjects, will have achieved similar standards; and it cannot be assumed students graduating with the same classified degree from different institutions, having studied the same subject, will have achieved similar standards.” (QAA, 2007, p. 2) “It is unacceptable for the sector to be in receipt of departmental spending of £15 billion but be unable to answer a straightforward question about the relative standards of the degrees of students, which the taxpayer has paid for” (IUSS committee, 2009, p 153)

4. Why change is needed (1) Our current systems, focused on marks and grades, aren’t working • Belief that it is possible to distinguish the quality of work to a precision of one percentage point (Elander & Hardman, 2002) • Belief that double-marking will ensure fairness and reliability (Laming,1990) • Transactional and bestowed credits & debits (Sadler, 2009) • The combination of scores, which obscures the different types of learning outcome represented by the separate scores

5. Why change is needed (1) contd. • “…students become more interested in the mark and less interested in the subject over the course of their studies.” (Newstead 2002, p2) • Belief that consistency can be achieved through conformity, and simple numerical rules (e.g. level 1 essay 3,000 words, level 3 essay 5,000; or no more than two pieces of assessment per module) • The distortion of marks by the type of assessment (e.g. coursework c.f. examination) and the actual subject discipline/s studied (Yorke, 1997; Yorke et al, 2000, Bridges et al, 2002) • The distortion of resulting degree classifications by the application of idiosyncratic institutional rules (e.g. Armstrong et al, 1998) (Rust, 2007)

6. Why change is needed 2Assessment standards applied to high-level complex learning can only be understood through active engagement with members of a disciplinary community • “we can know more than we can tell” (Polanyi, reprinted 1998, p.136) • ‘making sense of the world’ is a social and collaborative activity (Vygotsky, 1978) • Tacit knowledge is experience-based and can only be revealed through the sharing of experience – socialisation processes involving observation, imitation and practice (Nonaka, 1991) • Meaningful understanding of standards requires both tacit and explicit knowledge….. Such criteria are socially constructed requiring the sharing of tacit knowledge over time (O’Donovan et al. 2004; Rust et al, 2005)

7. Why change is needed 3 • Changes in higher education (e.g. massification, reduced unit of resource, expectations of increased productivity in staff) threaten the ‘health’ of disciplinary communities and their ability to share and exemplify professional judgement. • There has been slow progress in the professionalisation of university teachers and limited attention paid to professionalising assessment practice • Reliance on the external examiner system to mediate standards within the system is misplaced (Newstead and Dennis,1994) • “…it cannot be assumed students graduating …. will have achieved similar standards” (QAA, 2007) If some aspects of high-level learning can only be assessed using professional judgement then we need to ensure that judgement is indeed professional

8. Tenet 6 “ Assessment is largely dependent upon professional judgement and confidence in such judgement requires the establishment of appropriate forums for the development and sharing of standards within and between disciplinary and professional communities.”

9. How? Well how about? • Staff in the same discipline, from four or five regionally close institutions get together, each bringing examples of work of different standards • Maybe just start with final year work? • The focus of the day would be looking at particular pieces of final year work and hearing the arguments behind why the tutors from the originating institution believed they warranted the particular grade awarded. Why is it considered worthy of say, a first, or only just worth a pass? Would those from the other institutions agree? And if not, why not? • This event could be organised and facilitated by the Subject Network. • Later, the Subject Network could also possibly facilitate debates about particular issues to emerge at these regional meetings and ways could be explored to enable inter-regional sharing, etc.