1 / 24

An Exploration of Answer-Changing on a Computer-based High-Stakes Achievement Test

An Exploration of Answer-Changing on a Computer-based High-Stakes Achievement Test. Gail Tiemann Neal Kingston. Purpose. Gather baseline answer-changing patterns on a computer-based achievement test Add to the knowledge base in this area. Methods.

mikaia
Télécharger la présentation

An Exploration of Answer-Changing on a Computer-based High-Stakes Achievement Test

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Exploration of Answer-Changing on a Computer-based High-Stakes Achievement Test Gail Tiemann Neal Kingston

  2. Purpose • Gather baselineanswer-changing patterns on a computer-based achievement test • Add to the knowledge base in this area

  3. Methods • Third grade summative test in mathematics, one state testing program • One, 2012 form • 70 items, no time-limit • Students can review and change items as desired

  4. Methods • Assessment System Logs • Linear path through the test • Time spent viewing an item • Number of reviews • Number and pattern of changes • Unique benefit of CBT

  5. Sample and Test Characteristics

  6. Test Score Distribution

  7. Answer-Changing Items = 70 Students = 5328

  8. Wrong-to-Right & Right-to-Wrong • Final change in pattern • WR • RWR • RWWWR

  9. Wrong-to-Right by Items Changed Items = 70 Students = 5328

  10. Wrong-to-Right by Items Changed Items = 70 Students = 5328

  11. Wrong-to-Right by Items Changed Items = 70 Students = 5328

  12. Wrong-to-Right vs. Right-to-Wrong

  13. Wrong-to-Right vs. Right-to-Wrong

  14. Answer-Changing Patterns

  15. Answer-Changing Patterns

  16. Scatterplots

  17. Scatterplots

  18. Elapsed Time Wrong-to-Right < 60 seconds

  19. Wrong-to-Right < 10 seconds

  20. Flagging Rule Application Mean Wrong-to-Right Count (Target Form Only) + 8 Standard Deviations

  21. Discussion • Limitations • One form, one subject, one grade • Nested data • Distributions are not normal

  22. Discussion • Answer patterns and elapsed time add to answer-changing interpretations

  23. Future • Further exploration of wrong-to-right changes by elapsed screen time and position of change • Other sources of variance • Item-factors (difficulty, position) • Student-factors (demographics) • Nesting (buildings, districts, classrooms)

  24. Thank you

More Related