Research on Test Accommodations

1. Research on Test Accommodations Lawrence Lewandowski, Ph.D. Syracuse University Paper presented at NYSDC conference; LeMoyne College 11-1-07

2. Special Thanks Cassie Berger, M.S. Dr. Robin Codding Rebecca Gathje, M.S. Dr. Michael Gordon Dr. Ava Kleinmann Dr. Ben Lovett Dr. Rosanne Parolin

3. Dr. Miller, may I be excused my brain is full.

4. Test Accommodations • We’re all different, but some have disabilities • The law allows for “equal access” for persons with a disability • We’re obligated to make accommodations to afford equal access, not to maximize potential or guarantee outcome • Accommodations should circumvent or mitigate the specific impairment (i.e., blindness) so as to provide equal access • Accommodations should only benefit the persons with disability & should improve measurement of skills

5. The Need for Accommodations The physical/sensory disability model Qualitatively different needs Clear inability to access tests Very little overlap with nondisabled individuals

6. The Long and Winding Road • Once upon a time I was a psychologist performing diagnostic evaluations for LD, ADHD, etc. • In 1994 I was asked by the National Board of Medical Examiners to consult on ADA requests for test accommodations • I was faced with all kinds of new issues & questions • How fast do adults read? Are unimpaired people disabled? Is extended time a valid accommodation? How much time is reasonable? What about laptops? • I found little research to guide decision making, so I decided to do the research

7. Research Context • Students with LD & ADHD frequently seek/receive test accommodations • Extended time (ET) is most common accommodation • Various studies on LD show mixed findings; most show that ET helps all students to a degree, if the test involves speed and you prevent ceiling effects • ETS shows that most extended time is not used • Applicants frequently say that they are slow readers, take more time to finish, work harder than others to succeed, need an accommodation to reach their potential

8. What do most college students think about their abilities? • We conducted a survey of SU students • We listed all ADHD symptoms and common LD complaints; then asked students to rate those items that pertained to them • Students with disabilities endorsed more ADHD items (9 of 18), however typical students also endorsed an average of 4.5 items • 48.8% of nondisabled have to reread to understand text; 42.6% work harder than others for grades; 51% fidget and squirm in seat; 50.4% are easily distracted • So symptom complaints may be sensitive, but they are not specific to disability

10. Everyone claims to be a slow reader • How does one know s/he reads slower than everyone else? • Clinicians vary widely in what they think normal reading speed is (200, 300, 400 wpm) • We tested 90 college students on NDRT, PSI, CBM • Average reading speed was @189 wpm • NDRT Reading Rate (.22) did not predict Comprehension but WJ III Reading Fluency score did (.49) • WCPM reading was most reliable and predictive score

11. How important is speed?

12. Relationship of reading measures and processing speed • 125 college students given NDRT, WAIS PSI, WJ III PSC, WJ RF, SEPTAR, speed tasks • ND RR poor predictor of anything • Speed measures mildly related to ND Comp (.14-.26) • Reading Fluency correlated moderately with PSI & PSC (.53 & .51) • Best predictors of ND Comp is RF (.45) and Self Perception of Testing and Reading (.39)

13. ‘Sup with Processing Speed • There is no such thing as a Processing Speed Disorder • No research that shows PS to be sensitive and specific to a particular disability • PS as we measure it = rote visual motor speed tasks • PS is not a good predictor of outcome measures like NDRT Comp and probably any high stakes test • No research yet on PS and need for test accommodations • Clinicians might want to be careful making inferences about PS.

14. Extended Time

15. Validity of Extended Time Maximum potential Differential boost Interaction hypothesis

16. Study of Extended Time on Math Performance for Children w/wo ADHD • If we create a speeded task • Use math 3-digit addition that all can do • Provide unlimited test items to avoid ceiling effects • How would students differentially perform given a standard time and then time and one-half?

17. Participants • 27 per group • All 5th-7th grades • 10-13 yrs; M = 11yrs • Control: 15 males, 12 females • ADHD: 18 males, 9 females • 20 Combined ADHD; 7 Inattentive ADHD • 21 of ADHD participants were taking medication

18. Sample of Math Calculation Test(MCT) and Other Measures 156 772 588 + 978 + 664 + 613 • WJ III Math Fluency • WISC IV Processing Speed Index • Behavior Rating Inventory of Executive Function (BRIEF) • ADHD Symptom Rating Scale

19. ADHD Math Fluency 92.6 Processing Speed 96.8 Exec Function 69.9 DSM IV Symptoms 14.2 * All different at least p < .05 Controls 102.1 105.0 42.7 3.0 Group Comparisons

20. Math Performance at Standard and Extended (1.5x) Time

21. Math Performance: Items Correct

22. Study Conclusions • We did not find a “differential boost” for ADHD with extended time—in fact, control participants benefited more from the accommodation. • ADHD participants attempted fewer MCT items, made more errors and got fewer correct. • ADHD had poorer math fluency, processing speed and executive functioning. • ADHD performance was stable over time and was relatively accurate (90%). • Equal outcomes could be engineered on this test by giving ADHD students 50% extended time.

23. Implications • Speeded tests and time accommodations favor the nondisabled. • ET is not an accommodation specific to those with a disability. • We can set a test time or extend the time to achieve any end result we desire. • Perhaps we need to rethink our love affair with timed, high stakes tests instead of guessing who gets what amount of extra time.

24. RD Study

25. Extended Time on a Reading Comprehension Test for Adolescents w/wo Reading Disabilities • 32 RD and 32 Controls from high school • Given IQ, WJ III RF, and NDRT Comp at standard and time and one-half • Examined items attempted, correct, and % correct

26. MeasureControl RD ----------------------------------------------------------------- Raven IQ 98.76 97.8 Reading Fluency 107.1 88.88 Reading Comprehension 13 Min Correct 17.72 7.0 19.5 Min Correct 26.91 12.38 13 Min Attempt 22.88 12.0 19.5 Min Attempt 35.16 21.25 13 Min % Correct 77.48 59.2 19.5 Min % Correct 76.72 58.37 -----------------------------------------------------------------

27. Study Conclusions • Groups were similar in IQ but not Reading • RD group performed fewer items, got less correct, and had less accuracy (77% vs. 59%) • Both groups improved significantly, yet Controls improved more • RD at extended time attempted 21.3 items = to Controls at standard time (22.9 items) • RD at extended time got 12.4 correct, whereas Controls at standard time got 17.7 correct

28. Implications • Extended time is not a specific accommodation • Everyone will benefit from ET on a speeded test • ET can be used to equate work output (# of items attempted) but it won’t change skill accuracy. • The RD group appeared to be truly impaired and might need ET to get a better measure of their learning

29. Future Work • Evaluate predictive validity of extended time • Examine the effects of standard, time and one-half, and double time • Compare handwritten vs laptop essays w/wo LD • Study the utility of a private testing room • Develop a reading and test taking profiling system for 16-30 year olds

30. Handwritten vs. Laptop Essays • 140 students assigned to 4 essay writing groups • HW 10 min, HW 15, Laptop 10, Laptop 15 • Scored for length, TOWL scores, writing speed, and WJ Writing Fluency • No differences on WF, speed, or quality • Difference in # of words per story: HW 10 = 215, HW 15 = 216, Laptop 10 = 261, Laptop 15 = 356 • Laptop serves as a time saver for most, adding extended time has a multiplicative effect

31. For Additional Information Feel free to call: 315-443-1015 or write to: ljlewand@syr.edu Thanks for your attention!