Kyndra Middleton The University of Iowa kyndra-middleton@uiowa April 10, 2007

1. Examining Test Items for Differential Distractor Functioning Among Students with Learning Disabilities Kyndra Middleton The University of Iowa kyndra-middleton@uiowa.edu April 10, 2007

2. Purpose of the Study • To examine whether different distractor choices functioned differentially for students with learning disabilities who did not receive an accommodation, students with learning disabilities who received a read-aloud accommodation, and students with learning disabilities who received an accommodation other than a read-aloud • To help determine whether a test can be modified for students with learning disabilities by removing a distractor choice while maintaining adequate test validity and information

3. Instrument Used • 4th grade English Language Arts assessment from a criterion-referenced statewide test • Operational test data • Reading (42 MC items) • Writing (33 MC items)

4. Sample Used • 30,000 non-LD students sampled from 298,622 students • 9,056 LD students who did not receive an accommodation • 4,727 LD students who received an accommodation based on their IEP/504 plan • 1,371 LD students who received an accommodation based on their IEP/504 plus a read aloud accommodation

5. Sample Used cont’d Percentage of Students at Each Proficiency Level

6. Reference-Focal Comparisons Sample Used cont’d Note: IEP = Individualized Education Plan * = comparison did not show DIF so was not included in the DDF analyses

7. Procedure • Examine items that previously displayed DIF for DDF • DDF: when two groups that have been matched on ability have different probabilities of selecting a distractor • Standardized Distractor Analysis (SDA) • Distinguishes between distractors • Identifies uniformly and nonuniformly biased distractors • An extension of standardized p-difference

8. Procedure Used cont’d • Equation used to test for DDF: STD(i) = • : negligible DDF • : moderate DDF • : large DDF

9. Results • 70% of the items that displayed DIF also displayed DDF • 100% of DDF occurred with a comparison between the read aloud and some other group • 64% of the distractors that displayed DDF were in favor of the read aloud group

15. Results cont’d Note: +: moderate DDF in favor of the focal group ++: large DDF in favor of the focal group -: moderate DDF in favor of the reference group *R: DIF in favor of the reference group *F: DIF in favor of the focal group Shaded box: Items that did not exhibit DIF

16. Results cont’d • 17% that assessed reading standards showed DDF • 9% that assessed writing standards showed DDF • No observed pattern across content or cognitive area between groups

17. Results cont’d • Item that displayed large DDF was the most difficult item that displayed DIF • One item displayed DDF in each of the distractors (two favoring the read aloud group and one favoring the non-LD group) • Item that displayed DDF in two of its distractors was a spelling item • Both were homophones • Additional difficulty caused by read aloud

18. Conclusions/Future Research • Measurement dissimilarity between read aloud group and other groups • Exploratory study: More research needed to determine whether read aloud actually alters test’s validity • Matched on ability to provide more information at extremes