John F. Olson, Ph.D. Principal Investigator & Project Coordinator, CTEAG Project

1. Establishing the Validity of Test Accommodations for Students with Disabilities: A Collaborative of State-based Research CTEAG Project Summary of Accomplishments and Dissemination of Final Products John F. Olson, Ph.D. Principal Investigator & Project Coordinator, CTEAG Project CCSSO NCSA and Joint ASES-TILSA Meeting Detroit, MI June 23, 2010

2. CTEAG Project - Overview Goals • Conduct research that focuses on validity of test accommodations and interpretations of scores from accommodated state assessments • Use a common research design to conduct the studies and to analyze data for comparisons across states and accommodations • Build a shared body of research-based validity evidence on test accommodations across multiple states • Share findings from project that provide states with additional evidence (e.g., to submit to the USED for peer reviews) in support of their decisions on the use of various accommodations

3. CT EAG Project Structure • Lead State – Connecticut • Other Participating States – Kentucky, Michigan, and Nevada • Additional State Input and Involvement – from ASES and TILSA members • Project Management Team – group of SCASS advisors, experts, and consultants that oversaw and managed all validity studies and reporting activities • CCSSO -- provided logistical support, staffing, and consultants to do tasks for the project

4. CTEAG Research Questions • Does the accommodation affect item and test form statistics? • Does the accommodation change the content structure of the test, i.e., is there an impact on the test structure under accommodated and standard administrations? • Does the accommodation increase performance of students? Does it increase performance more for SWD than it does for SWOD?

5. CT EAG – Validity of Accommodations ProjectState Participation – Details of Studies

6. Summary of Results • In most cases, accommodations did not alter item stats (difficulty, discrimination) • Accommodations also did not change test form stats/characteristics (mean item difficulty, reliability) • Accommodations did not have an effect on content structure of test forms • Accommodations did not change dimensionality of tests John Olson Establishing the Validity of Test Accommodations June 2009

7. Summary of Results (continued) • Overall, results from ANOVA were mixed • Accommodations increased performance of students in KY study; enhanced directions increased performance in MI • Differential boost effect found in some cases, i.e., SWD scores increased more than SWOD in CT#2, KY, and MI studies • Key areas of focus – • Differential boost effects • Increasing the accessibility of the assessment John Olson Establishing the Validity of Test Accommodations June 2009

8. CTEAG Project - Accomplishments • Assisted and guided states in conducting 5 different validity studies on test accommodations • Analyzed all data from the studies and reviewed results with the PMT and states (and sometimes with states’ vendors) • Summarized results in a series of detailed state reports • Shared updates and information on progress of project at 8 ASES and TILSA meetings from 2007 to 2010 • Completed and disseminated 3 reports and a Project Database • Presented information on the project at 4 national conferences and meetings (2007-2010) • Conducted 3 evaluations of the work, surveying states, and summarizing the info in an Evaluation Report C

9. CTEAG Final Reports/Products • Guidebook for Test Accommodations Validity Studies --provides overall summary of results/findings and guidance to states on validity studies in a nontechnical manner [report was distributed to ASES and TILSA in Feb. 2010] • CTEAG Technical Report – providestechnical information on the project and details of each state’s study, e.g., designs, methods, analyses, statistics, results, etc.

10. CTEAG Final Reports/Products • Lessons Learned from State Validity Studies – provides information on the background and context in which the validity studies were conducted, issues that states may need to consider, and lessons learned by the four participating states • CTEAG Project Database -- provides additional info from the five studies, such as, samples, variables, data files, and statistical data for use in further research

11. Dissemination Activities • Presentation Session at 2010 NCSA on June 22 • State participants present results and perspectives from conducting validity studies • Representative from USED discusses importance of findings from federal perspective • Dissemination Workshop at Joint ASES-TILSA meeting on June 23 • Share all remaining products and summary of accomplishments • State participants discuss lessons learned and how data and findings can be used to benefit states • Expert advisors and USED representative discuss implications at national level

12. Final Dissemination Activities • Distribute the reports/products widely to states, the USED, and others • Place PDFs of all final products on websites (CSDE , CCSSO/EAG) • Issue formal announcement (press release?) and Chiefline newsletter article • Coordinate with project partners (NCEO and NASDSE) to share information with others • Submit final report on project to USED

13. Questions & Comments For more information, contact: John F. Olson President/Founder Olson Educational Measurement & Assessment Services 617-965-1490 (home/office) 210-859-5593 (cell) jmclkolson@yahoo.com