District Scoring and Reporting Training

1. District Scoring and Reporting Training April 18, 2018

2. Overview • Scoring and Reporting Resources • Score Inquiry System • Discrepancy Resolution System (DRS) & PearsonAccess Next • Caveon Appeals • Understanding FSA/NGSSS Reports • Q&A Discussion

3. Scoring and Reporting Resources • Location: FDOE ShareFile – K-12 Scoring and Reporting -- Resources • Contents include: • District Student Results (DSR) and District/State Aggregated Results (DAR/SAR) file layouts • Guides and Instructions • Caveon Results

4. Score Flag/Not Reported Codes • 1 / 9 = Score Reported (initial vs. late reporting) • 0 = Not Tested (NT) • 2 = Did Not Meet Attemptedness • 3 = Do Not Score/Invalidated during testing • 4 = Insufficient Match to TIDE (FSA PBT Only)/ Last Name = NONAME and the FLEID is FL000000000000 (NGSSS PBT Only) • 5 = Below Grade Tester • 6 = Duplicate Record • 7 = FDOE Hold • 8 = Caveon Invalidation

5. Not Reported Codes Not Reported Codes Where To Take Action Score Inquiry System Discrepancy Resolution System/PearsonAccess Next* Caveon Appeal (ShareFile) • NT • NR2 • NR3 • NR7 • NR4 • NR5 • NR6 • NR8

6. Score Inquiry System • After the initial release of the test results, the Department opens a website called the Score Inquiry System. • The Score Inquiry System allows District Assessment Coordinators (DACs) to inquire about missing reports or to ask for a student’s score to be reviewed. • A deadline for submitting a score inquiry is established for each test administration. We encourage you to submit score inquiries before the deadline and to periodically check the system for inquiry updates.

7. Score Inquiry System For example: • NT: Request to search for missing documents if you believe your student submitted a completed test. • NR3: Request to UNDO DNS so the score can be released during the late reporting. • Please refer to the Score Inquiry System User Guide (posted to the FDOE ShareFile) for more details.

8. Discrepant Records • Each test administration, a Not-Reported code of 4, 5, or 6 may be applied to a test record due to incorrect or missing information on the student. These records are referred to as discrepant records in the FSA systems. • After the initial test results have been released, FDOE assists the districts in resolving these discrepancies to ensure that student results are released in the most timely manner.

9. Discrepancy Resolution System (DRS) • DRS was developed to simplify and streamline this process by providing a user-friendly interface to more easily and efficiently resolve these discrepancies. • Once the initial results have been released, DRS will provide district-level users direct access to the discrepant records in their district. It will also allow them to take the appropriate action needed to resolve them. • Just like the Score Inquiry System, a deadline for resolving these discrepancies via the DRS will be established for each test administration.

10. DRS – Where to start • DRS is found on the Test Information Distribution Engine (TIDE) homepage, in the After Testing column, under the Data Cleanup task menu.

11. DRS – Search Discrepancy Resolution Page Make your selections in the District and School drop-downs for the discrepancies you wish to view and resolve. Click Search, the list of discrepancies will be displayed as shown on the next slide.

12. Resolve Discrepancy Page: Discrepancy List To filter the retrieved discrepancies by discrepancy type or other criteria, enter a term in the field above the search results. The DRS will display only those records containing the entered value. Status: Column is blank until action is taken Type: Student Not Found Non Eligible Test Duplicates Click the Resolve Icon for the discrepancy you want to resolve. The appropriate window will appear, depending on the type of discrepancy you wish to view. Click the PDF file in the View column to view the student demographics and to find the scanned answer document information, such as the document’s secure barcode number.

13. DRS – Reviewing and resolving discrepancies • Using the student data already provided in TIDE, the DRS will detect and identify the following types of test discrepancies: • Student Not Found (Score Flag 4) • Non-Eligible Test (Below Grade Tester Score Flag 5) • Duplicate (Score Flag 6) • Again, after these discrepancies are detected, an authorized user is able to review and resolve them by selecting different courses of action available in the system.

14. DRS – Score Flag 4 Resolve a Student Not Found Discrepancy (Score Flag 4) Common Cause: A student’s paper answer document is returned without a PreID label. Action: Click the Associate icon ( ) in the Resolve Discrepancy: Student Not Found panel, then search for the student in TIDE by using the Search for Student to Associate panel.

15. Resolution – Score Flag 4 Scenario A: Student’s record is available in TIDE in the list of retrieved students. Scenario B: Student’s record is not found in TIDE or the PreID information/gridded FLEID was not matched with a student in TIDE. Click ( ) to Add/Edit the student. Click ( ) to match the student’s score record with their TIDE record. Then confirm. or

16. DRS – Score Flag 5 Resolve a Non-Eligible Test Discrepancy (Below Grade Tester – Score Flag 5) Cause: A student’s enrollment grade in TIDE is higher than the test grade in the data file. Action: Click the Resolve Override icon ( ) if you believe the Enrolled Grade matched the Test Grade at the time of testing.

17. Resolution – Score Flag 5 • After overriding the discrepancy, the test record will be forwarded for scoring and reporting. • If the student truly took a below-grade test, then leave the record as is by taking no action. The test record will remain as a score flag 5 record in the file. • Please be aware that while the invalidation column and icon ( ) appear, only PBTs can be invalidated in the DRS. CBT tests can be invalidated to resolve a discrepancy, but this must be done via the Score Inquiry System.

18. DRS – Score Flag 6 Resolve a Duplicate Discrepancy (Score Flag 6) Cause: More than one test record in the same subject are found for a single student in TIDE. Action: Click the Reassign icon ( ) if you believe one of the records belongs to another student in TIDE.

19. Resolution – Score Flag 6 • After reassigning a record to a student in TIDE, the record will be forwarded for scoring and reporting. • If a student took the same subject test more than once, one of the tests should be invalidated. • Again, only PBTs can be invalidated in the DRS. Online test invalidations MUST be submitted through the Score Inquiry System.

20. DRS – User Guide • The DRS User Guide provides step-by-step instructions for resolving discrepancies. It will be emailed to you and posted to the FDOE ShareFile. • After consulting the user guide, any additional questions or issues can be directed to the FSA Help Desk (866-815-7246).

21. Reminders • It is not necessary to resolve every case in DRS (ex. Student with SF5 record tested below grade). • If a district makes a mistake while seeking to resolve a discrepancy, please contact FDOE. • Only discrepancies corresponding to NR Codes 4, 5, and 6 can be resolved in the DRS. All other not reported records must be resolved via the Score Inquiry System or Caveon appeal.

22. Rejected Student Tests in PearsonAccess Next (PANext) During testing you will receive rejected student test alerts.

23. PearsonAccess Next • Update Missing Demographic Data (Enrollment status, First Name/Last Name, Date of Birth, FLEID). • Correct a duplicate student test by setting Do Not Score. • The user guide for resolving the NR4 and NR6 cases via PearsonAccess Next will be emailed to you and posted to the FDOE ShareFile.

24. Caveon - Overview • Each administration, Caveon looks at answer patterns for a given test and flags any groups of students (2 or more) that have extreme similarity. • These records will show as ScoreFlag of 8 on your District Student Results (DSR) files and will not have any score information populated. • Districts and schools have the opportunity to appeal and release any flagged scores during the Score Inquiry window.

25. Caveon – Where to Start • Shortly after results are released for a given administration, FDOE Scoring and Reporting will post a spreadsheet on ShareFile with information on students flagged by Caveon for similarity. • You will receive an email with a guide and instructions if you had any students flagged for that administration.

26. Caveon – Where to Find Invalidations • Location:https://fldoe.sharefile.comK-12 Scoring and Reporting > Districts > ## - District Name > YYYY Season > Student Invalidations • Ex. K-12 Scoring and Reporting > Districts > 67 – Washington > 2018 Spring > Student Invalidations • This information will be included in the email you receive should you have student invalidations.

27. Caveon – Using the Guide • The Caveon Appeals Guide will be attached to the email notifying you of student invalidations. • You can also find the guide on ShareFile atK-12 Scoring and Reporting > Resources > Guides and Instructions > Caveon > Appeals Guide. • The guide explains the appeal process, includes examples, and lists all required documentation for an appeal.

28. Caveon – Submitting Appeals • It is up to district and school staff to decide when it is appropriate to appeal a Caveon-flagged student score. • Some schools will feel pressure from parents to appeal even though it may not result in a score release. • If you have any questions about what to say or include – Call or Email Gabriel or Jim. • You will post any appeals on ShareFile in the same folder the spreadsheets were posted.

29. Caveon – Post-Appeal • Appeals are reviewed in the order they are submitted. • We will post decision letters on ShareFile informing you whether scores will be released. This will happen after the appeals deadline but before Late Reporting results are released. • For any released scores, you will see these students on the late reporting DSR. You will also receive an Individual Score Report (ISR) with your other late reporting results.

30. Caveon – Changes/Notes • We are no longer accepting appeals via fax or mail. • These methods proved to not be as reliable as posting via ShareFile. • The Superintendent Signature requirement has been removed. • This happened about a year ago, but it was a big change that caused some confusion. We are only requiring the DAC signature on appeals now.

31. Understanding Florida Assessments Reports • These publications describe the various report types used to distribute test results to students, schools, and districts. They also include information about the test content areas and a glossary of technical terms. • These documents will be posted to FDOE’s website by the end of April.

32. Contact Information K-12 Scoring and Reporting Team Qian Liu 850-245-0867 Qian.Liu@fldoe.org Gabriel Colorado 850-245-0779 Gabriel.Colorado@fldoe.org Jessica Colvin 850-245-5048 Jessica.Colvin@fldoe.org Shukun Fang 850-245-0904 Shukun.Fang@fldoe.org Seung-Yeon Ha 850-245-0783 Seung-Yeon.Ha@fldoe.org Patrick Herrington 850-245-0523 Patrick.Herrington@fldoe.org Esra Kocyigit 850-245-0769 Esra.Kocyigit@fldoe.org Yingru Liu 850-245-5158 Yingru.Liu@fldoe.org James Low 850-245-0085 James.Low@fldoe.org

33. Questions?