Download
1 / 36
360 likes | 371 Vues
e-marking in large-scale, high stakes assessments conference themes :. role of technology in assessments and teacher education use of assessments for continuous professional development assessments, teacher education, and school and systemic reform.
E N D
e-marking in large-scale, high stakes assessments conference themes: • role of technology in assessments and teacher education • use of assessments for continuous professional development • assessments, teacher education, and school and systemic reform
large-scale, high stakes assessmentsthree questions: what do we mean by technology in this context how can we best assure assessment validity how can e-marking help get the best from current large-scale high-stakes assessment practices
a. what do we mean by technology in this context?
what do we mean by technology in this context innovativeitems … enabling students to show how they arrived at an answer and allowing them to score along a range of possible point values based on how well they understand a particular process.
what do we mean by technology in this context Computer-adaptive tests …that better align with the performance levels of individual students by giving them more challenging or more manageable tasks according to how they perform in early questions.
what do we mean by technology in this context intellectually ambitious assessments … through a combination of innovative items and computer adaptive testing to measure the learning that is difficult to assess in pencil-and-paper tests.
in the context of large-scale, high-stakes assessments current issues are: hardware availability, compatibility and consistency high speed connectivity equality of access…
in the context of large-scale, high-stakes assessments cost…
how can we best assure assessment validity the greatest threat to validity for performance assessments may be construct under-representation — i.e. where the assessment is too narrow or fails to include important aspects of a construct.
how can we best assure assessment validity often, the outcome of large-scale assessments is narrow – a pass or fail, a grade; a raw or standardized number; a percentage.
how can we best assure assessment validity The greatest threat to validity for performance assessments may be construct under-representation — where the assessment is too narrow or fails to include important aspects of a construct. we need help to create good test designs we need pre-tested performance information
how can we best assure assessment validity often, the outcome of large-scale assessments is narrow – a pass or fail, a grade; a raw or standardized number; a percentage we need detailed, more useful, information about performance on different aspects of the tests
e-marking not like this!
e-marking: efficient, secure, reliable…. images of student responses captured by scanning scripts are securely stored images are automatically distributed to appropriate markers markers log on to receive and score images on line, on screen, at item level. supervisors track marking progress and quality in real-time e-marking
e-marking: security, accuracy, speed • Each year: • Pearson marks over 60 million exams in the USA and more than 7.2 million national tests and 9 million exams in the UK • Some exams taken in the England are marked in Australia Pearson designs and delivers national exam processing systems, including e-marking, for programmes as diverse as national testing in Abu Dhabi to State and national assessment programmes across Australia Results are delivered quicker and in more detail than ever before
Key marking benefits… increased efficiency – removes majority of manual handling of scripts reduced marking time – concurrent marking, image distribution, no remarking, mark as scanned reduced risk, secure scripts, secure images increased control and flexibility increased marking quality fairer outcomes for students Item-level results data capture e-marking
information to help desig good tests Automated analysis of e-marking pre-test results can inform test design and test validity: • item-level facility values • item performance • bias factors • overall test performance analysis
Item performance characteristics A well performing item: The probability of a student selecting the correct answer increases as the student’s overall test performance increases Facility 0.76
Item performance characteristics A poor performing item Success is not dependent on the students, overall test performance The item does not differentiate between a low and high ability student Facility 0.29
information to support school improvement and raise standards of student achievement • e-marking provides a level of accuracy and detail about student performance never before available • automated results analysis can directly benefit: ministry and departments of education to define policy schools and teachers to identify good practice and areas for development in teaching students and parents to take control of learning • …helping to raise standards
information contributing to professional development • helping teachers to address such questions as: how are we doing as a school how are we doing compared with other schools like ours how are we doing compared with last year how are we doing each subject how are we doing in each class how are we doing on each paper; on each question and is each student doing.
Summary role of technology in educational measurement and teacher education existing e-marking technology offers more from paper-based assessments to educate teachers about where their students are at.. use of educational measurement for continuous professional development ..leading to the sharing of professional expertise and shifting classroom expectations. educational measurement, teacher education, and school and systemic reform An emphasis on teacher education combined with quality assured assessments will profoundly affect the efficacy of classroom interactions and raise standards in education.
