The Poisson-Gamma model for speed tests

1. The Poisson-Gamma model for speed tests Norman Verhelst Frans Kamphuis National Institute for Educational Measurement Arnhem, The Netherlands

2. The student monitoring system • Measurement of individual development • Common scale • Estimation of distribution (norms) • Twice per grade (M3, E3,…,M8) • Several subjects • Arithmetic • Reading comprehension • Technical reading

3. Two types of speed tests • Basic observation is the time to complete a task • AVI cards • Basic observation is the number of completed subtasks within the time limit • Tempotests (TT) • Three Minute Test (TMT)

4. Example tempotest (E4) • Op de politieschool spelen ze ook rook koor een soort toneel • Het lijkt wel wat op ‘politie en boefje spelen stelpen slepen’. • Net zoals op de basisschool. • Wat poe doe boe je bij een gevecht? • Je pistool trekken? • Nee, dat mag zomen zomaar zomer niet.

5. Easy version as fee oom uur zee oor … poot (=150) Hard version banden geluid tante beker kuiken koffer … brandweerwagen (=150) Example TMT

6. Models • Measurement model: Poisson • What is the relation between the (latent) ability and the test performance? • Structural model: Gamma • The distribution of the latent ability in one or more populations? (M3, E3, M4,…,M8)

7. Measurement model: Poisson (1)

8. Measurement model: Poisson (2)

9. Parameter estimation:incomplete design (JML)

10. Person parameters

11. Design TMT • 3 difficulty levels (1, 2, 3) • For each level: three parallell versions (a, b, c) • Each student participates twice: medio and end of same grade • At each administration: 3 cards of levels 1, 2 and 3 (in that sequence) • M3: only cards 1 and 2

13. Two step procedure • Estimate the task parameters σi • JML = CML • Estimate latent distribution while fixing the task parameters at their CML -estimate

14. Advantage

15. Structural model:distribution of reading speed (θ)

16. Marginal distribution of the sum score s

17. Negative Binomial(Gamma-Poisson)

18. Negative binomial

19. EAP

20. Reliability

21. Validation (tempo test)

22. Validation (tempo test)

23. Validation (TMT)

24. Latent class model • Population consists of two latent classes of size π and 1 - π respectively • The latent variable is gamma distributed in each class • Parameters • π • α1 en β1 • α2 en β2 • EM-algorithm

26. Validation (TMT)

27. Validation (TMT)

28. Norms (TMT)

29. Thank you

30. Example: student v

31. Problems • SE(π) large • Local maxima? • Thick right tail of observations • >2 classes? • Initial estimates • Homogeneity of test material • Local independence

34. Averages (1000 replications)

35. Standard deviations (1000 rep.)