1 / 17

DIF Analysis

DIF Analysis. Galina Larina 28-31 of March, 2012 University of Ostrava. DIF analysis Definitions. Item impact “significant group difference on an item, e.g., when one group has a higher proportion of examinees answering an item correctly than another group ”

ostinmannual
Télécharger la présentation

DIF Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DIF Analysis Galina Larina 28-31 of March, 2012 University of Ostrava

  2. DIF analysis Definitions • Item impact • “significant group difference on an item, e.g., when one group has a higher proportion of examinees answering an item correctly than another group ” • Due to the true group differences in proficiency or due to item bias • Differential Item Functioning (DIF) • “It occurs when test-takers having identical levels on the latent trait that the test was designed to measure but belonging to different groups, have different probabilities of endorsing (or answering correctly) a particular item” • Examinees in different groups are matched on the proficiency If an item is found to be poor-fitting in the whole data set or within any group of test-takers, it should be remove from subsequent DIF analysis

  3. DIF analysisEffectless of fit statistics Infit and outfit mean square errors for simulated 50-item test in which item 25 has DIF

  4. DIF analysisTypes of DIF Uniform DIF Non-uniform DIF Non-uniform mixed DIF

  5. DIF analysis Statistical methods for evaluating DIF • CTT methods • Conditional p-value difference • Delta plot • Standardization • Chi-square methods • Mantel-Haenszel • etc. • IRT methods

  6. DIF analysis Mantel-Haenszel method Focal group Base group

  7. DIF analysis Mantel-Haenszel method Average factor by which the likelihood that a base group member gets the item correct exceeds the corresponding likelihood for comparable focal group members For statistically significant DIF on an item, Prob. < 0.05

  8. DIF analysis Mantel-Haenszel method • MH procedure is an extension of the chi-square test of independence • Advantages: • Easy to compute • Modest sample size requirements • Effect size • ETS DIF classification rules • ‘Large DIF’ absolute value of MH D-DIF greater than or equal to 1.5, chi-square test sig. at 0.05 level/ Category C • ‘Moderate DIF’ at least 1.0 (and less) than 1.5) and the chi-square test sig. at 0.05 level/ Category B

  9. DIF analysis Rasch approaches • Separate calibration t-test first proposed by Wright and Stone Where di1 is the difficulty of item I in calibration 1, di2 is the difficulty of item i in calibration 2 based on groups 2, s2i1is the standard error of estimate for di1, and s2i2is the standard error of estimate for di2 • Winsteps applies the above formula in DIF analysis

  10. DIF analysis IRT approaches • The between fit approach is based on a single calibration that contains at least two subpopulations of interest. where J is a number of subpopulations, N is a number of person in each populations, xni is the score for person n responding to item i, and pni is the probability of person n responding correctly to item i given the overall estimates for the ability of the person and the difficulty of the item

  11. DIF analysis Winsteps Column 20 with width 1 DIF label start in person label column 20 DIF label start in person label with a width 1

  12. DIF analysis Winsteps Press Entry Number Press OK

  13. DIF analysis Winsteps This should be at least 0.5 logits for DIF to be noticeable For statistically significant DIF on an item, Prob. < 0.05 Pairwise comparison For statistically significant DIF on an item, t > |2|

  14. DIF analysis Winsteps Item 1

  15. DIF analysis Winsteps Item 1

  16. DIF analysis Winsteps. Plots Press OK

  17. DIF analysis Winsteps. Plots. Item 1

More Related