Overview of the Rasch Measurement Model

1. Overview of the Rasch Measurement Model

2. Overview • Exam integrity, • Limitations of Classical Test Theory (CTT), • Advantages of the Rasch model, • The partial credit model, • Banking and equating, and

3. Challenges facing certification: Exam integrity • Integrity… • Reliability, • Validity, and • Security • … of exam content • Controlling for compromises to integrity… • Statistically, and • By design

4. Classical Test Theory • Basis of conventional measurement methods. • Reliability (i.e., proportion of shared variance between items) • Item difficulty (i.e., proportion answering an item correct) • Item discrimination (i.e., how well items correlate with total score) • Based on a random (and hopefully representative) sample from the population.

5. Limitations of CTT • Item parameters are sample dependent. • Typically we don’t know whether our sample is truly representative. • Test calibration based on these groups would be misleading.

6. The Rasch measurement model • A persons’ measure on any trait is a simple function of their ability and the item’s difficulty. • Person ability can be conditioned (removed) from sample estimates making item calibration sample free.

7. Rasch Measurement: Sample-free equivalent estimates • The two characteristic curves are almost indistinguishable. • Our best estimate on how any one person would perform on the test – we’ve conditioned person ability from the sample.

8. Hi/Lo Scatter A scatter plot of item parameter estimates should converge around a straight line, with tighter convergence in the middle and more scatter at the extremes.

9. Advantages of the Rasch model • Used in conjunction with CTT • Item calibration is sample free • Sample sizes… • …as small as 100 examinees (generating standard errors about .12 logits) are generally enough. • Item banks and reliability • Since items can be transformed onto a common scale, they can be replaced with equivalent ones without jeopardizing the reliability of the exam. • Linking tests • Traditional test equating becomes obsolete. A linking system can be developed that can be translated into equivalent forms of an exam.

10. The Partial Credit Model • The number of steps into which an item is divided and their relative difficulties can vary between items. • The points must be awarded sequentially, so that each increasing value represents an increase in the trait being measured (0 = totally wrong, 1 = partially correct, 2 = almost completely correct, and 3 = completely correct).

11. Item banking • Calibrations are obtained by applying a probabilistic model for what should happen when an examinee attempts an item. • Comparability is maintained because any test formed from bank items is automatically equated onto the common scale.

12. Test equating • Links of common items can be embedded in different tests. • A translation constant is computed • Passing standards can be adjusted based upon the translation constant Form A • Separate samples • Relatively few link items • Requires pilot testing through item seeding 10 Link Items Form B

13. Test equating T=-2.0