Georgia Modification Research Study Spring 2006

1. Georgia Modification Research Study Spring 2006 Sharron Hunt Melissa Fincher

2. Georgia’s Student Assessment Program • Includes an array of criterion-referenced tests (including an alternate) and a norm-referenced test • Built from the ground up to be inclusive • GA includes teachers of special populations in all phases of test development • GA includes all students in pilots and field tests

3. Georgia’s Student Assessment Program • GA provides a menu of traditional accommodation options • Presentation • Response • Setting • Scheduling • Accommodations are provided during pilots and field tests

4. Georgia’s Student Assessment Program • Although Georgia is inclusive, some students still struggle to access the tests • Schools sought additional avenues for students who were caught between assessments – neither the general or the alternate were completely appropriate for these students

5. Georgia’s Student Assessment Program • Georgia looked for avenues to investigate how to make assessments more accessible • Questions to answer • Who are the students struggling to access the general assessments? • What additional accommodations/modifications might assist in making the test more accessible? • About the same time…. • Enhanced Assessment Instruments Grant announced (ultimately pulled back) • Announcement of the 2% flexibility

6. Georgia’s Student Assessment Program • Georgia moved forward with an empirical research study designed to answer those questions • The Testing Division worked with the • Division for Exceptional Students • Curriculum and Instructional Services Division

7. Study Purpose • To investigate the effects of certain modifications on the reliability and validity of the state testing programs • Criterion-Referenced Competency Tests (CRCT) • Georgia High School Graduation Tests (GHSGT) • Focus on reading and math

8. Study Purpose • This study will help Georgia determine • What works • Who benefits • The goal of the study is to help Georgia operationalize additional options for the 2006-2007 school year

9. What is a modification? • an alteration in the administration of an assessment that results in a possible change in the content or construct being assessed • typically either through the addition or removal of content

10. What is an accommodation? • an alteration in the administration of an assessment that allows students to participate • how the test is administered or presented • how the student responds • Construct (what the test measures) remains the same

11. Georgia Modification Study • Focused on selected grades and content areas • Students in grades 4 and 7 took the Reading and Math CRCT (March 2006) • Students in grade 12 took the ELA and Math GHSGT (April 2006)

12. Georgia Modification Study • For security reasons, off-level testing was used • Grade 4 students took the 3rd grade Spring 2005 CRCT • Grade 7 students took the 6th grade Spring 2005 CRCT • Grade 12 students took the Spring 2005 GHSGT

13. Georgia Modification Study • Study included both special education students and regular education students • The number of special education students who took the test last year at each school was identified • Tested the number of students identified, if possible • Schools randomly selected an equal number of general education students to participate

14. Georgia Modification Study • Included all special education students, with the following exceptions: • GAA students • Students who require Braille or large print • Set a broad net to help determine the characteristics of students assisted by modifications

15. Georgia Modification Study • Each school assigned a condition for this study • All students within the school took the test under the same condition • Both special and regular education students

16. Georgia Modification Study CRCT (grades 4 and 7) • Three conditions • Condition 1: Resource Guides • Condition 2: Read Aloud/Calculator • Condition 3: Traditional Administration

17. Georgia Modification Study GHSGT (grade 12) • Two conditions • Condition 1: Resource Guide • Condition 2: Traditional Administration

18. Condition 1 – CRCT & GHSGT • Resource Guide • A single sheet (one for each content area) that provides key definitions, examples, multiplication tables etc. • A short practice test was provided to help introduce the resource guides to students; each item related to something on the resource guide • Recommended introducing sample tests and guides a couple of days before the administration

19. Condition 1 – CRCT & GHSGT • Resource Guide Development • Committee comprised of Testing, Special Education, and Curriculum specialists reviewed the tests • identified concepts students might struggle with and discussed appropriate information for inclusion on the guide

20. Condition 2 - CRCT • Read Aloud (Reading CRCT) • Teacher read the reading passages and questions to students • Teacher reads test verbatim • Calculator (Math CRCT) • Basic function calculators provided to schools assigned this condition

21. Condition 3 - CRCT • Traditional Administration • Students took tests as they have always done • Special education students received the standard accommodations they normally receive • Regular education students did not receive accommodations

22. Condition 2 - GHSGT • Traditional Administration • Students took tests as they have always done • Special education students received the standard accommodations they normally receive • Regular education students did not receive accommodations

23. Georgia Modification Study • A survey was be provided for each special education student • The person most knowledgeable about the student completed the survey • This information will be critical in determining who benefits from the modifications

24. Georgia Modification Study • Survey collected important information about each special education student • Student’s disability/ies • Severity of disability/ies • IEP objectives for reading and math • Time in mainstream/general ed classroom • Area/s served • Instructional services • Type of accommodation, if any

25. Georgia Modification Study • Administration Procedures • Follow all administration protocols as outlined in the Examiner’s Manual • Complete all student demographics on answer documents, including State-Required Codes

26. Georgia Modification Study • Administration Procedures • Followed all administration protocols as outlined in the Examiner’s Manual • Completed all student demographics on answer documents, including State-Required Codes • Complete survey for each special education student

27. Georgia Modification Study • Key Research Questions • Are there performance differences for students with disabilities on the CRCT and GHSGT when these assessments are administered under modified conditions? • Are there performance differences for regular education students on the CRCT and GHSGT when these assessments are administered under modified conditions?

28. Georgia Modification Study • Key Research Questions • How, if at all, do performance differences on the traditional versus modified version of the CRCT or GHSGT differ for students with disabilities compared to regular education students? • Are the CRCT and GHSGT reliable and valid assessments when administered under modified conditions?

29. Georgia Modification Study • Currently analyzing data to answer the research questions • Analysis at the item level • Are certain items more likely to function differently? • Analysis at the form level • How does the test form function under each condition (modified vs traditional) for each group of students?

30. Georgia Modification Study Sharron Hunt, Director of Testing shunt@doe.k12.ga.us / 404.657.0314 Melissa Fincher, Assistant Director of Testing mfincher@doe.k12.ga.us / 404.651.9405