Presentation Transcript

1. DAC Academy Assessment 101 August 9, 2011 Colorado Department of Education Unit of Student Assessment
2. DAC Academy 2011 DAC Roles & Responsibilities In regards to the administration of the state assessments, DACs ensure: Quality Accuracy Security Documentation Communication Training
3. What is a DAC? District Assessment Coordinator Appointed by the district superintendent Responsibilities and powers may vary Need to have a specific and specialized skill set/level of expertise Educational or psychometric background DAC Academy 2011 DAC Roles & Responsibilities
4. Training/Communication Train SACs and Test Proctors (documentation) Consistent and reliable distribution of critical information to superintendents, teachers, and core assessment team members Manage Materials Maintain security of all materials Ensure accurate score reports Pre-coded label use and student data grid DAC Academy 2011 DAC Roles & Responsibilities
5. Oversee Administration Scheduling Ethical practices Misadministrations Distribution, collection, packing and shipping Accommodations are provided and documented Document the chain of custody Keep the communication lines open DAC Academy 2011 DAC Roles & Responsibilities
6. DAC Academy 2011 DAC Roles & Responsibilities Failure to implement the responsibilities of the DAC role will probably result in some catastrophic disaster in your district. Whenever you are gone, you need to have a backup.
7. You Have a Vital Leadership Role! Official point of contact - One DAC (One official district email address) ONLY district staff member officially designated to communicate with the school district, USA/CDE, and the assessment vendor CTB The first question we ask those who call us: “Are you the DAC?” DAC Academy 2011 DAC Roles & Responsibilities
8. DAC Academy 2011 DAC EMAILS The DAC must forward the important information to the appropriate people in the district. When responding to a DAC email DO NOT click Reply Send an email directly to the contact person Forward the message to the contact person OR click on the contact person’s email address
9. DAC Academy 2011 Current Assessment System Unit of Student Assessment Achievement(CO Model Content Standards/ CO Academic Standards) College Entrance National and State Trends EnglishLanguage Acquisition(Colorado ELD Standards) Transitional Colorado Assessment Program/ Colorado Alternate Colorado ACT National Assessment of Educational Progress Colorado English Language Acquisition Program COACT NAEP CELApro TCAP CELAplace LC&E CoAlt
10. DAC Academy 2011 Transitional Colorado Assessment Program TCAP PURPOSE: To measure student achievement relative to the Colorado Model Content Standards. 31 tests Reading(Grades 3-10) Writing (Grades 3-10) Mathematics (Grades 3-10) Science (Grades 5, 8, & 10) Lectura/Escritura (Grades 3 & 4)
11. DAC Academy 2011 Transitional Colorado Assessment Program TCAP
12. DAC Academy 2011 Colorado Alternate CoAlt PURPOSE: Standards based achievement test for students with a significant cognitive disability. Eligibility Determined through a student’s IEP team. Less than 1 percent of all students. 27 tests (the same grades and content areas as TCAP) Mandatory Training prior to testing
13. DAC Academy 2011 Colorado English Language Acquisition Assessment CELApro PURPOSE: English Language Acquisition NOT to be confused with an achievement test Measures four domains of language development Speaking, Listening, Reading, Writing Academic Language & Social, Basic Interpersonal Communication Given to all NEP and LEP K-12 students As defined per LCE Unit definitions Mandatory Training prior to testing Early Testing Window for Speaking may begin Dec 12
14. DAC Academy 2011 Colorado ACT College Entrance Exam Accepted by U.S. Colleges, Universities, Military Academies and NCAA Colorado ACT (COACT) State test date: April 24, 2012 Make-up test date: May 8, 2012 2012 Test Date on TUESDAY Accommodations Testing Dates: April 24 – May 8, 2012 All 11th grade students by law 11th Grade Alternate for students eligible to take CoAlt Managed by Exceptional Student Leadership Unit 2012 dates TBD Contact Tanni Anthony, ESLU 303-866-6681 anthony_t@cde.state.co.us
15. DAC Academy 2011 Colorado ACTCOACT Test Supervisor (TS) DACs may need to develop a communication process with Test Supervisors for Accountability*. CONTRACT DAC COMMUNICATION *Test Supervisors must develop a communication plan with Back-up Test Supervisors and Test Accommodations Coordinators.
16. DAC Academy 2011 CO ACT UPDATES ONLINE Schools New changes online schools: Two national tests date options to online students ONLY: February, 11, 2012 & April 14, 2012 Student can choose either OPTION Student can pay $20 to take April 14, 2012 if they registered and miss Feb. 11, 2012 test option
17. DAC Academy 2011 National Assessment of Educational Progress NAEP 2011-2012 National sample of grades 4 and 12 and ages 9, 13 and 17 yr old students (About 2,040 schools and 72,150 students nationwide) NAEP 2011-2012 Assessment Windows *Each selected student is tested in one subject, only.
18. DAC Academy 2011 National Assessment of Educational Progress NAEP Participating schools selected by National NAEP Statisticians Most schools identified last May-Small sample Most participating NAEP districts have received initial notification from NAEP State Coordinator Will receive national results for 9, 13 and 17 year olds in reading and math for Long-term Trend tests. Other tests are field and pilot tests. Do not receive disaggregated results for districts or individual schools- not designed for this.
19. DAC Academy 2011 National Assessment of Educational Progress NAEP Roles National NAEP Office NAEP State Coordinator NSC School & Community Contracted Assessment Team: Westat
20. DAC Academy 2011 National Assessment of Educational ProgressNAEP- Relationship between NAEP & School Community NAEP State Coordinator NAEP School Coordinator Confirms the assessment date Provides schools with info for parental notification Responds to questions Works with district/school personnel to ensure a smooth process District Contact Assists NSC in School communications Works with NSC and the Supervisor to Oversee the process School & Community
21. DAC Academy 20101 NAEP-The Nation’s Report Card® public web site: http://nces.ed.gov/nationsreportcard/
22. DAC Academy 2011 NAEP-The Questions Tool http://nces.ed.gov/nationsreportcard/itmrls/
23. DAC Academy 2011 DATA OPERATIONSYou Live and Die by Your Data!
24. DAC Academy 2011 Data OperationsEssentials for DACs Online Enrollments Purpose: To get a count of needed materials ADE Collections Purpose: To collect accurate data for labels or correct student biographical data after testing Logistics Purpose: To ensure smooth handling of testing materials before, during, and after testing Final Assessment Results Purpose: To obtain and validate final data results
25. DAC Academy 2011 Data OperationsEssentials for DACs (continued) Online Enrollments – on CTB Navigator this fall CELApro – 10/3/2011 through 10/14/2011 Co Alt – 10/17/2011 through 10/28/2011 TCAP special populations – 10/31/11 through 11/23/2011 Online Enrollments are used to determine what and how many materials are shipped to your district. Navigator website:
26. DAC Academy 2011 Data OperationsEssentials for DACs (continued) Automated Data Exchange (ADE) Collections: – Pre-Coded Labels (PCL) – December 2011- January 2012 Student Biographical Data (SBD) Review CELApro – mid March 2012 TCAP/Co Alt – late May-early June 2012 COACT - late June-early July 2012 The respondent for each collection is authorized separately. SBD training will be online in late February 2012 ADE website has links to documentation, respondent forms, and the system to submit files. https://cdeapps.cde.state.co.us/
27. DAC Academy 2011 Data OperationsEssentials for DACs (continued) Logistics – receiving, processing, and shipping test materials Training in November online Bubbling test invalidations and accommodations at time of testing New: More than one accommodation can be marked! The importance of the School Group List (SGL) Keeping track of the number of tests returned by content area, grade, and school Separating tests for Home Schooled students N Count on Navigator
28. DAC Academy 2011 Data OperationsEssentials for DACs (continued) Final Assessment Results:CELApro, TCAP, Co Alt The final data files are known as the GRF files (General Research Files) They are downloaded from Navigator Paper reports for parents in August Tentative schedules for file delivery: CELApro: early April TCAP and Co Alt: mid July Final COACT Results: sent to high schools in August All results are embargoed from public distribution until a specific date
29. DAC Academy 2011 Data OperationsWhen do I use CDE’s ADE system? Secured Site - for authorized respondents - To submit a file of students that has been updated since the Student October count to create the barcode labels for TCAP, Co Alt, and COACT. To download the student biographical portions of the CELApro, TCAP, Co Alt, COACT, and 11th grade Alternate for updates during those collections. To check the post SBD files for applied changes.
30. DAC Academy 2011 Data OperationsWhen do I use CTB’s Navigator? Secured Site – for DACs Online training Online enrollments and special populations Access oral scripts, translated oral scripts, teacher read directions for CSAP Non-standard Accommodations Forms Short/Add Forms Archived Emails Data: NCounts, GRF files, official data retrieval
31. DAC Academy 2011 Core Assessment Team District Assessment Coordinator (DAC) Oversees and manages all aspects of state assessment administration Ensures standardized ethical test administrations Training Manage data collection processes
32. DAC Academy 2011 Core Assessment Team Who is it? District Assessment Coordinator ELA Director/Coordinator SPED Director/Coordinator CBLA Contact ADE Respondent(s) Title I and Title III (AYP, AMAO) Others…
33. DAC Academy 2011 Core Assessment Team District Trainers Proctors, staff, faculty and administrators must be trained through an interactive format on a yearly basis! This is true for EVERYONE… How do you as the DAC know they have been trained? Training Schedule Attendance List Copy of Materials (November training will be discussed in future slides.)
34. DAC Academy 2011 CDE/USA – Support and Service Scoop Weekly newsletter sent out to districts Subscribe by contacting the CDE Communications Unit at: CDE_Communications_Office@cde.state.co.us Includes important deadlines, announcements and reminders Compiles multi-unit info from CDE for easy access to: Superintendents, DACs, etc. (anyone who subscribes)
35. DAC Academy 2011 Assessment Resources 2011-2012 Procedures Manual 2011-2012 Accommodations Manuals 2012 SAC/DAC Manuals Test Proctor’s/Test Examiner’s Manuals SBD Manual Technical Reports Data Interpretation Guidelines Guides to Test Interpretation Parent Brochures
36. DAC Academy Essential Information for 2011-2012 August 9, 2011 Colorado Department of Education Unit of Student Assessment
37. Adopted Assessment System Components: Formative Interim Summative Nationally-recognized college admissions assessment (11th grade) English Language Proficiency School Readiness Individual Career and Academic Plan (ICAP)
38. Phases of Implementation for the Colorado Academic Standards
39. Summative Assessment Transition Timeline Spring 2011 CSAP as usual Spring 2012 Transitional Colorado Assessment Program (TCAP) administered Spring 2013 TCAP administered New assessment pilot Spring 2014 New Colorado assessment administered
40. Colorado Transitional Assessment Program (TCAP) How is the TCAP similar to CSAP? Math, reading, and writing will be assessed in 3rd-10th grades Science will be assessed in 5th, 8th, and 10th grades No grade level shifts in content knowledge Use of same test blueprint Application of Colorado Growth Model and accountability ratings continues
41. TCAP How is the TCAP different from CSAP? Assessment of new Colorado Academic Standards which are common to the old Colorado Model Content Standards, where possible CDE will release a description of content eligible for inclusion on TCAP
42. DAC Academy 2011 What’s New for 2011-2012 2011-2012 Procedure Manual Posted in September Updates with Accommodations Multiple accommodations may be marked on the student data grid These will be explained later Test Security Test books must not be altered after students complete test sessions Proctor/Examiner Trainings CDE will provide information that must be included in district trainings
43. All students will have the same test content, resources, directions, testing conditions, and scoring procedures. One score obtained by one student in one part of Colorado, will mean the same as the same score obtained by another student in a different part of Colorado. DAC Academy 2011 Standardized Assessment
44. DAC Investigations DACs must investigate all concerns that arise whether from a student, parent, staff member, CDE, or any other source Notify the Unit of Student Assessment of the results of the investigation
45. Test Security Test books and test items are secure Do not go through test books to erase underlining/stray marks and do not alter the answer bubbles. DAC/SAC Manuals will be updated to reflect this.
46. DAC Academy 2011 Assessment Procedure Reminders Chain of Custody Documented before, during, and after testing Large print, Braille, Teacher Read Directions, Oral Scripts, Translated Oral Scripts Materials should not be delivered to schools more than 1 week in advance Distribute only the content area being assessed Returned to a designated secure location Not stored in classrooms
47. DAC Academy 2011 Questions about Items CSAP items are developed and then reviewed by CTB Content Editors and Content Leads Items are reviewed by the CDE CSAP Consultant Colorado Teachers and Educators review the items at CVAR (CSAP Item Review) CTB and CDE have a Post-CVAR review CTB does their internal review of the items CDE does a final review of the items in the test book Colorado Teachers and Educators review how constructed response items will be scored at Anchor Review All test items are the property of the state of Colorado
48. DAC Academy 2011 Questions about Items Proctors should not be studying items Items only accessed during test administration If a test proctor or examiner has a question about an item… Contact the SAC, who will contact the DAC with the grade, content area, and item number for CDE to review The DAC contacts the appropriate person at CDE in the USA by phone Include in proctor training instructions: Do Not copy the item Do Not discuss the item (with other teachers, etc.) Do Not email the content of the item
49. DAC Academy 2011 Questions about Scores If school/district personnel or parents have a question about a score… First, the DAC should review: GRT file Past results District assessment results Irresolvable questions are forwarded to CDE by the DAC USA works with CTB to examine the data May possibly call you in to review material with us Parents cannot view assessments
50. DAC Academy 2011 Media The media are not allowed to have access to the tests, or take pictures or video of the testing materials (including the front or back covers of test booklets and packaged boxes of testing materials for shipping), during any part of the assessment process. Not only is media presence in the school disruptive, it creates a non-standard assessment environment and can lead to misadministrations.
51. DAC Academy 2011 Assessment Procedure Reminders Training “Inadvertent” misadministrations do not occur Comprehensive, interactive, ongoing trainings must include: Test Security Standardized Environment Test Proctor Requirements Teacher vs. Test Proctor Ethics Required documentation Individual documentation (DACs keep) One form submitted to Unit of Student Assessment School administrations must be involved
52. DAC Academy 2011 Important – Procedures Manual Please Note: Due to the complex nature of a standardized assessment process, any practice not specifically permitted should be presumed inappropriate until and unless specifically authorized by the program sponsor (in this case, CDE).
53. DAC Academy 2011 Assessment Procedure Reminders TCAP Testing Administration Schedules The DAC must ensure that all testing follow the requirements stated in the Procedures Manual and Proctor’s Manual. Allow time for the proctors to read the directions from the Proctor’s Manual and distribute materials before and to collect and secure materials after the administration of each assessment. The sessions for each content area must be administered in chronological order.
54. DAC Academy 2011 Assessment Procedure Reminders Multiple grades and multiple content areas must not be tested in the same room. CDE strongly recommends that the SAC keep on file the schedules of exactly when, where and to whom each assessment is being administered. CDE strongly recommends that each proctor be given a clear schedule of when to administer each test by content area and session. Make sure that accommodation considerations are included in the development of the schedule.
55. DAC Academy 2011 Assessment Timeline 2011-2012 TCAP, Co Alt, CELApro and CELAplace Training September: ELA, Accommodations, and Procedures Manuals November: Required training for TCAP, Co Alt, CELApro, and logistics (i.e. SAC/DAC manuals) Ordering and Delivery of Materials October: Online Enrollments for CELApro October: Online Enrollments for Co Alt November: Online Enrollments for TCAP special populations November: Finalize shipping information for CTB May: Online enrollments for CELAplace 2012-2013 Test Administration October-December: Submit Nonstandard Accommodation forms for approval November: Early testing window request completed January: CELApro Administration window February-April: TCAP and Co Alt Administration window Data September: Student October collection opens (CELApro Pre-coded labels) December: Pre-coded Labels collection opens (TCAP, Co Alt, and COACT March: Grade 3 TCAP Reading Ncount and CELApro Ncount and SBD April: Final CELApro 2012 Data Release May: TCAP/Co Alt Ncount and SBD July: Final TCAP/Co Alt 2012 Data Release
56. DAC Academy 2011 Assessment Timeline 2011-2012 COACT Test Supervisor Checklist of Dates: Late September 2011: ACT mails initial letter from CDE and School renew form to principals with Checklist of Dates and Standard Testing Requirements. October 2011: Receipt Deadline for principals to return School Renewal Forms. Receipt Deadline for TS and TACs to return renewal forms to ACT November 2011: TACs receive mailing which includes procedures for requesting accommodations, quantities of individual request forms, and an order form for practice test. Online schools receive a special mailing which includes National Test Date Voucher Eligibility Request and Test arrangements. Mid November 2011: Schools are reminded to submit off-site testing requests, if appropriate. Late November 2011: Announcement of training workshop locations and dates mailed to TS, BTS, TACs, and DACs. December 2011: Receipt deadline for schools to submit completed proposals for off-site testing arrangements to ACT
57. Accommodations Colorado Department of Education
58. Title I Reasonable adaptation and accommodations for students with disabilities The inclusion of limited English proficient students…and provide reasonable accommodations on the assessment
59. What’s New for Assessment Accommodations 2011-2012 Multiple accommodations may be marked on the student data grid  “Specific” added to wording related to accommodation documentation Clarification on Word Prediction w/audio w/o audio Clarification on Assistive Technology Clarification on using Sign Language and Oral Scripts Updated NSA form New code on charts “R”
60. Two Accommodations Resources
61. What are Accommodations? Definition: Accommodations are practices and procedures in the areas of presentation, response, setting, and timing/scheduling that provide equitable instructional and assessment access for students with documented needs. Accommodations are designed to provide access and equity but not to provide students with an advantage.
62. The use of accommodations is linked through each of these areas: Accommodations
63. Types of Accommodations on Statewide Assessments
64. Based upon need Documented in an Educational Plan Team includes the parent Used regularly
65. Table Activity In alphabetical order by last name go around the table and list the requirements of using a standard accommodation on state assessments.
66. Accommodations on State Assessments Accommodations must be based upon an individual student’s needs. Accommodations may not be based on a category of disability, EL classification, or be specific to a program Accommodations must be determined as the result of a formal evaluation conducted by the educational team which monitors and creates the student’s formal education plan. The evaluation process and plan formulation must include the parent and the student.
67. Accommodations on State Assessments Accommodations must be used regularly during instruction, and on classroom and district assessments. They must be in place at least three months prior to the statewide assessment. Accommodations must not be introduced for the first time during state test administration. ** In the case of translation accommodations for students identified as EL, students do not need to be consistently receiving translation in the classroom.
68. Accommodations on State Assessments Accommodations must be clearly documented in a formal education plan such as an IEP, 504 plan, advanced learning plan, health care plan, intervention (RtI) plan, Individual Learning Plan (ILP), English Language Acquisition (ELA) plan or any other school or district formal education plan which is the result of a formal evaluation conducted by an educational team. Educational teams should only select accommodations for state assessments that do not invalidate the student’s score. Documentation should be specific on what accommodations are needed to provide access to instruction and assessments.
69. Accommodations on State Assessments Accommodations that do not require documentation on a state assessment data grid must still be documented as a need on the student’s formal education plan or another school or district document, which is the result of a formal meeting, and kept with the formal plan. A student may be allowed more than one accommodation. For example, a student who is administered the Oral Script of TCAP may also require extra time. If more than one accommodation is used on the TCAP, all accommodation bubbles must be filled in on the student data grid.
70. If the student does not meet the criteria the accommodation must be requested through the nonstandard accommodation process New codes on accommodation grids: “R” = Restricted “S” = Standard
71. Forms now only on Navigator If it does not appear on the accommodations grids, it is nonstandard… when in doubt, Call Mira! DEADLINE: December 15
72. Nonstandard Accommodations Policies pertaining to accommodations for statewide assessments are set by the Colorado Department of Education (CDE) based on current research and assessment practices.  Per Code of Federal Regulations The State's (or, in the case of a district-wide assessment, the LEA's) guidelines must--(i) Identify only those accommodations for each assessment that do not invalidate the score; and(ii) Instruct IEP Teams to select, for each assessment, only those accommodations that do not invalidate the score (34 CFR d 300.160(b)(2).  Therefore, accommodations listed for use on statewide assessments on a student’s IEP should follow the policies included in this manual. Use of accommodations that do not follow these policies will result in a misadministration.
73. Nonstandard Accommodations Nonstandard accommodations are changes to the standardized testing procedures that have not been predetermined to maintain the integrity, validity, reliability and comparability of the assessment, and must be approved by CDE Office of Assessments, Research and Evaluation prior to use on statewide assessments.
74. Table Activity The person with the earliest birth date at your table (Jan, Feb…), select a nonstandard accommodation that may be requested by your district, Share with the table why it is a nonstandard accommodation, and discuss as a table what needs to be included when submitting to CDE.
75. Nonstandard Accommodations This example shows what a nonstandard accommodation request contains. Duplicating this sample does not guarantee approval of a nonstandard accommodation request and it is meant only as a guide.Nonstandard accommodation requests must be individualized for each student and based on need documented in an IEP or 504.
76. Step 1 Look at the Basics
77. Step 2IEP or 504 Minimum that must be included: The front page (showing student name and disability) Accommodations pages from the IEP or 504 plan – it must show: Instructional use District Assessment use State Assessment use You may include the entire IEP.
78. Step 3What is being requested?
79. Step 4Use of Accommodation Instructional District Assessments State Assessments
80. Step 5Justification
81. Step 6Work Samples
82. Step 7Letter back to DAC - Approved Approved NSA bubble is marked on student data grid Student may use the accommodation Student receives a score Student is counted in participation
83. Step 7Letter back to DAC – Denied Denied, and the student Does not use the accommodation Denied, and the student Uses the accommodation Non-approved Nonstandard Accommodation Bubble is marked on student data grid Student receives a “no score” Student does not count towards participation Student may use standard accommodations Student receives a score Student is counted for participation
84. Direct Linguistic Adjustments of text of assessment to reduced linguistic load needed to access test Indirect Linguistic Adjustments to the conditions under which the assessment is given which allow for ELs to use linguistic resources Not valid on state assessments
85. Linguistic Accommodations Word to word dictionaries Teacher read direction in native language Translated Oral scripts Scribe Response in native language Oral Written
86. Linguistic Accommodations Grades 3 & 4Lectura & Escritura Decision Making Flowchart NEP & LEP FEP M1 & M2 FEP Exited
87. Accommodations Time Line Monitor Accommodations
88. Forms and databases used to monitor the implementation of accommodations during instruction and for classroom, district and state assessment Plan on attending Statewide Accommodations Training. September/October September/October
89. DACs order “extra special” tests this includes 200% Black and White (high contrast) Student Last Name SASID Grade Content Area Early November November
90. December 15 December
91. Determine scheduling considerations related to accommodated sessions. Be certain test administrators and proctors know the accommodations that must be provided to individual students Know how to administer accommodations Determine where accommodated sessions will occur. Prior to Assessment
92. Standardization: The adherence of uniform administration procedures and conditions during an assessment The objective of providing assessment accommodations is to make fair comparisons with other students taking the test Strict adherence to guidelines for instructions and procedures for the administration of accommodations is necessary to ensure that test results reflect actual student learning Ensuring Standardization Standardization
93. Are there procedures in place to ensure test administrators adhere to directions for the administration of accommodations? How many students are receiving accommodations? What types of accommodations are provided and are some used more that others? Questions to guide accommodation evaluation at the school/district level. Evaluate Evaluate
94. Are students receiving accommodations as documented in their formal educational plan? What are the results for students when accommodations are used? If students are not demonstrating expected level of performance, did students have access to instruction in assessed content? Was the accommodation effective? Questions to guide accommodation evaluation at the school/district level. Evaluate Evaluate
95. Selected Districts will have accommodations monitoring as part of their scheduled Title I visit. Document review Interviews with District Personnel, School Assessment Coordinators, and Proctors of accommodated sessions State Accommodations Monitoring
96. For More Information Colorado Accommodations Manual http://www.cde.state.co.us/cdeassess/publications.html Mira Monroe 303.866.6709 Colorado Accommodations Guide for English Learners http://www.cde.state.co.us/cdeassess/publications.html Glen Sirakavit 303.866.3266
97. DAC Academy 2011 Unit of Student Assessment Martin Petrov – Business Process (303)866-6929 Joyce Zurkowski – Assessment Director Margaret Lake – Data Operations Christina Wirth-Hawkins – Assessment Research Glen Sirakavit – Assessment Policy Jason Clymer – TCAP Mira Monroe – CoAlt Vacant – CELApro Christine Deines – COACT Pam Sandoval – NAEP