1 / 36

Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD)

Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD). Jamal Abedi University of California, Davis/CRESST. Accommodations for ELLWD.

layne
Télécharger la présentation

Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validity, Effectiveness and Feasibility of Accommodations for English Language Learners With Disabilities (ELLWD) Jamal Abedi University of California, Davis/CRESST

  2. Accommodations for ELLWD The process of provision of accommodations to ELLWD is more complex than the process used for either SD or ELL students. Care must be taken to choose accommodations that are appropriate for this particular subgroup of students. ELLWDs need accommodation addressing both their language needs and their disabilities.

  3. Why Should English Language Learners Be Accommodated? • Their possible English-language deficiency may interfere with their content knowledge performance. • Assessment tools may be culturally and linguistically biased for these students. • Linguistic complexity of the assessment tools may be a source of measurement error. • Language factors may be a source of construct irrelevant variance.

  4. Why Should Students With Disabilities be Accommodated? • Their disabilities put them at disadvantage. • Accommodations must be provided to level the playing field.

  5. Site 2 Grade 7 SAT 9 Subsection Scores

  6. Site 4 Grade 8 Descriptive Statistics for the SAT 9 Test Scores by Strands

  7. Normal Curve Equivalent Means & Standard Deviations for Students in Grades 10 and 11, Site 3 School District Reading Science Math MSDMSDMSDGrade 10 SWD only 16.4 12.7 25.5 13.3 22.511.7 LEP only 24.0 16.4 32.9 15.3 36.8 16.0 LEP & SWD 16.3 11.2 24.8 9.3 23.6 9.8 Non-LEP/SWD 38.0 16.0 42.6 17.2 39.6 16.9 All students 36.0 16.9 41.3 17.5 38.5 17.0 Grade 11 SWD Only 14.9 13.2 21.5 12.3 24.3 13.2 LEP Only 22.5 16.1 28.4 14.4 45.5 18.2 LEP & SWD 15.5 12.7 26.1 20.1 25.1 13.0 Non-LEP/SWD 38.4 18.3 39.6 18.8 45.2 21.1 All Students 36.2 19.0 38.2 18.9 44.0 21.2

  8. Accommodations for ELLs and SWDs • Can the same accommodations used for students with disabilities be used for ELLs? • Can the same accommodations used for ELLs be used for students with disabilities?

  9. How Are We Doing in Practice Nationally? • Are the states and districts across the nation cognizant of this important principle of using accommodations that are appropriate for a particular subgroup? • Are there any objective national criteria to help states to select appropriate accommodations for ELLWD students? • Or, is the assignment of accommodations to these students based on temporary and subjective decisions?

  10. Accommodation in Stanford 9 Testing Used by a District

  11. SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies There are 73 accommodations listed: N: Not Related R: Remotely Related M: Moderately Related H: Highly Related From: Rivera (2003) State assessment policies for English language learners. Presented at the 2003 Large-Scale Assessment Conference

  12. SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies I. Timing/Scheduling (N = 5) N 1. Test time increased N 2. Breaks provided N 3. Test schedule extended N 4. Subtests flexibly scheduled N 5. Test administered at time of day most beneficial to test taker N = not related; R = remotely related; M = moderately related; H = highly related

  13. SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies II. Setting (N = 17) N 1. Test individually administered N 2. Test administered in small group N 3. Test administered in location with minimal distraction N 4. Test administered in familiar room N 5. Test taker in separate location (or carrel) N 6. Test administered in ESL/Bilingual classroom N 7. Individual administration provided outside school (home, hospital, institution, etc.) N 8. Test taker provided preferential seating N 9. Increased or decreased opportunity for movement provided N 10. Teacher faces test taker N 11. Special/appropriate lighting provided N 12. Adaptive or special furniture provided N 13. Adaptive pencils provided N 14. Adaptive keyboards provided N 15. Person familiar with test taker administers test N 16. ESL/bilingual teacher administers test N 17. Additional one-to-one support provided during test administration in general education classroom (e.g. instructional assistant, special test administrator, LEP staff, etc.) N = not related; R = remotely related; M = moderately related; H = highly related

  14. SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies III. Presentation (N = 32) R 1. Directions repeated in English R 2. Directions read aloud R 3. Audio-taped directions provided in English N 4. Key words or phrases highlighted M 5. Directions simplified M 6. Audio-taped directions provided in native language M 7. Directions translated into native language N 8. Cues provided to help test taker remain on task M 9. Directions explained/clarified in English M 10. Directions explained/clarified in native language M 11. Both oral and written directions in English provided M 12. Both oral and written directions in native language provided M 13. Test items read aloud in English H 14. Test items read aloud in simplified/sheltered English N 15. Audio-taped test items provided in English H 16. Test items read aloud in native language H 17. Audio-taped test items provided in native language N = not related; R = remotely related; M = moderately related; H = highly related

  15. SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies III. Presentation (N = 32) N 18. Assistive listening devices, amplifications, noise buffers, appropriate acoustics provided N 19. Key words and phrases in test highlighted H 20. Words on test clarified (e.g. words defined, explained) H 21. Bilingual word lists, customized dictionaries (word-to-word translations) provided N 22. Enlarged print, magnifying equipment, Braille provided N 23. Memory aids, fact charts, list of formulas and/or research sheets provided N 24. Templates, masks or markers provided N 25. Cues (e.g. arrows and stop signs) provided on answer form N 26. Acetate shield for page provided N 27. Colored stickers or highlighters for visual cues provided R 28. Augmentive communication systems or strategies provided (e.g. letter boards, picture communication devices, voice output systems, electronic devices) H 29. Simplified/sheltered English version of test provided H 30. Side-by-side bilingual versions of test provided H 31. Translated version of the test provided N 32. Test interpreted for the deaf or hearing impaired/use of sign language provided N = not related; R = remotely related; M = moderately related; H = highly related

  16. SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies IV. Response (N = 17) N 1. Test taker marks answers in test booklet N 2. Test administrator transfers test-taker’s answers N 3. Test taker’s transferred responses checked for accurate marking N 4. Copying assistance provided between drafts N 5. Test taker types or uses a machine to respond (e.g. typewriter/word processor/computer) N 6. Test taker indicates answers by pointing or other method N 7. Papers secured to work area with tape/magnets N 8. Mounting systems, slant boards, easels provided to change position of paper, alter test taker’s position N. 9. Physical assistance provided N 10. Enlarged answer sheets provided R 11. Alternative writing systems provided (including portable writing devices, computers and voice-activated technology) R 12. Test taker verifies understanding of directions R 13. Test taker dictates or uses a scribe to respond in English N 14. Test taker responds on audio tape in English H 15. Oral response in native language translated into English H 16. Written response in native language translated into English H 17. Spelling assistance, spelling dictionaries, spell/grammar checker provided N = not related; R = remotely related; M = moderately related; H = highly related

  17. SY 2000-2001 Accommodations Designated for ELLs Cited in States’ Policies V. Other (N = 2) N 1. Out-of-level testing provided N 2. Special test preparation provided Preliminary Findings: State Assessment Policies for English Language Learners, SY 2000-2001 GW/CEEE, Large-Scale Assessment Conference 2003 Crivera@ceee.gwu.edu N = not related; R = remotely related; M = moderately related; H = highly related

  18. There are 73 Accommodations Listed 47 or 64% are not related 7 or 10% are remotely related 8 or 11% are moderately related 11 or 15% are highly related

  19. The most important issue is the concern over the validity of accommodation strategies: • Research findings suggest that providing accommodations may increase performance of ELLs/SDs, while also benefiting non-ELLs/SDs. • There is not enough research support for many of the accommodations currently being used in the national and state assessments. • The only way to make judgments about the efficiency and validity of accommodations used by states is to use them in experimentally controlled studies with both ELL/SWD and non-ELL/SWD students.

  20. Concern over the validity of accommodation strategies: A research example Some forms of accommodation strategies, such as the use of a glossary with extra time, raised the performance of both ELL and non-ELL students (Abedi, Hofstetter, Lord, and Baker, 1998, 2000) • ELL students’ performance increased by 13% when they were tested under glossary with extra time accommodation. • While this looks promising, it does not present the entire picture. • Non-ELL students also benefited from this accommodation, with an increase of 16%. • English and bilingual dictionary recipients may be advantaged over those without access to dictionaries. This may jeopardize the validity of assessment.

  21. Concerns over the validity of accommodation strategies: A research finding There are, however, some accommodations that help ELL students with their English language needs without compromising the validity of assessment. • Linguistic modification of test items is among these accommodations. • This accommodation also helped students with learning disabilities. • Thus, an accommodation may have the potential to be effective and valid for both LEP and SD, consequently relevant for LEPWD.

  22. Concern over the validity of accommodation strategies Validity: The goal of accommodations is to level the playing field for ELL/SWD, not to alter the construct under measurement. Consequently, if an accommodation affects the performance of non-ELL/SWD, the validity of the accommodation could be questionable. Feasibility: For an accommodation strategy to be useful, its implementation must be possible in large-scale assessments. Strategies that are expensive, impractical, or logistically complicated are unlikely to be widely accepted.

  23. How validity of accommodations can be examined? Only through experimentally controlled research where: • ELL/SWD and non-ELL/SWD students are randomly assigned to experimental and control groups • Both ELL/SWD and non-ELL/SWD students are observed under accommodated and non-accommodated assessments

  24. Can existing data (from national and state assessments) be used for examining the effectiveness and validity of accommodations? • Effectiveness of accommodations can be examined only if ELL/SWD students are randomly assigned to the accommodated and non-accommodated conditions • Validity of accommodations can be examined only if non-ELL/SWD students are randomly assigned to the accommodated and non-accommodated conditions

  25. How the validity of accommodations can be examined? • Using existing data? • Through experimentally controlled field study?

  26. LEP Students Assessed With and Without Accommodations, 1998 NAEP Writing Assessment: National Sample, Public and Nonpublic Schools Combined

  27. Research findings • Pitoniak, M., Lutkus, A., Cahalan-Laitusis, C., Cook, L. & Abedi, J. (2005). Are Inclusion Policies and Practices for State Assessment Systems and NAEP State Assessments Aligned? • Sireci, S. G., Li, S. & Scarpati, S. (2003). The effects of test accommodation on test performance: A review of the literature (Center for Educational Assessment Research Report No. 485). Amherst: University of Massachusetts. • Thompson, S., Blount, A., Thurlow, M. (2002). A summary of research on the effects of test accommodations: 1999 through 2001 (Technical Report 34). Minneapolis: University of Minnesota, National Center on Educational Outcomes. • Thurlow, M. L., McGrew, S., Tindal, G., Thompson, S. J., Ysseldyke, J. E., & Elliott, J. L. (2000). Assessment accommodations research: Considerations for design and analysis (NCEO Technical Report 26). Minneapolis: University of Minnesota, National Center on Educational Outcomes. • Tindal, G., & Fuchs, L. (2000). A summary of research on test changes: An empirical basis for defining accommodations. Lexington, KY: University of Kentucky, Mid-South Regional Resource Center. • Abedi, J., Hofstetter, C., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1-28

  28. How the validity of accommodations can be tested in an experimentally controlled condition?

  29. Conclusions and Recommendation Accommodations: Must be relevant to the subgroups of students Must be effective in reducing the performance gap between accommodated and non-accommodated students Must be valid, that is, accommodations should not alter the construct being measured The results could be combined with the assessments under standard conditions Must be feasible in the national and state assessments

  30. Conclusion There is not enough research support for many of the accommodations that are currently used in national and state assessments. The only way to make judgments about the efficiency and validity of these accommodations is to use them in experimentally controlled situations with both ELL/SWD and non-ELL /SWD students and examine their validity and effectiveness under a solid experimental condition. The results of CRESST studies along with other studies nationwide have provided support for some of the accommodations used for ELL students.

  31. Conclusion cont. Examples of research-supported accommodations: • Providing a customized dictionary is a viable alternative to providing traditional dictionaries. • The linguistic modification of test items that reduce unnecessary linguistic burdens on students is among the accommodations that help ELL students without affecting the validity of assessments. • Computer testing with added extra time and glossary was shown to be a very effective, yet valid accommodation (Abedi, Courtney, Leon and Goldberg, 2003)

  32. Conclusion cont. It is thus imperative to examine different forms of accommodations before using them in state and/or national assessments. Without information on important aspects of accommodations such as validity, it would be extremely difficult to make an informed decision on what accommodation to use and how to report the accommodated and non-accommodated results.

  33. For more information, please contact Jamal Abedi at: (530) 754-9150 or jabedi@ucdavis.edu

More Related