Enhancing State Assessments Research on the Presentation Effect of Passage Based Computer Tests Michael RussellTechnology and Assessment Study CollaborativeBoston College
Reading Comprehension/Text Presentation Study • Research Questions • When 4th grade reading comprehension assessments are transitioned to computers, what is the impact on students’ test scores? • Are there differences in test scores when reading passages are presented on the computer using scrolling text versus whole page text? • How is student performance (modal effect) related to their computer literacy/use or fluidity?
Text Presentation Study - Participating Schools and Design • 219 Vermont public school students were assessed • Participating Students • 37 students from urban schools • 83 students from suburban schools • 99 students from rural schools • Students were randomly assigned to take the assessment on paper, on laptop using scrolling text, or on laptop using whole page text
Text Presentation Instruments • Assessment • 4 reading passages (2 NAEP, 1 PIRLS, 1 NH) • 19 multiple choice items • Computer Fluidity Exercises • Keyboarding • Click on moving object • Drag and drop • Arrow keys • Technology Survey • Computer Literacy • Eleven released VA and NC technology multiple choice items • Demo
Text Presentation Study Analyses • Group score averages show a six point difference across groups
Open-ended Survey • 82% easier on computer • Twenty-six students indicated that they had a problem while taking the test on computer. • not understanding the directions (1), problems scrolling (2), having problems using the mouse (2), not understanding how to end the test (2), and having the computer freeze (2) • 16.8% reported using the highlighting feature • 17.4% said that they did mark items for review
Reading Comprehension/Text Presentation Study Analyses • There were no significant differences in reading comprehension scores across testing modes (paper = 58.1%, scrolling = 52.2%, whole page =56.9%) • There were no statistical differences in reading comprehension scores based on computer fluidity and computer literacy, but a pattern in performance may indicate that students are disadvantaged by the scrolling text mode, particularly those lower computer skills. • The majority of students who took the reading test on a computer indicated that they would prefer to take the test on computer. • Providing highlighters and review markers is useful for some students.