ScorePoint Training

1. ScorePoint Training March 2019

2. Introductions

3. Today’s Presentation The ScorePoint presentation slides are available on CBT Support from the CBT Resources section: https://cbtsupport.nysed.gov/hc/en-us/articles/360024911412

4. Agenda • General Information • Roles & Responsibilities • Security & System Recommendations • Logging into ScorePoint • Readers • Team Leaders • Scoring Directors • Reports • Finishing the Project • Contact Information

5. Scoring CD Delivered to Scoring Location • The Scoring CD will be delivered to scoring location by: • ELA 4/2/19 • Math 5/1/19

6. Training Expectations • Training approach for today’s session • We value your questions! • ScorePoint system questions • Scoring policy questions • Vendor questions • Handouts/Logins

7. How do ScorePoint and Nextera work together? NY CBT Testing for Spring 2019: High-Level Interactions between Nextera & ScorePoint

8. The ScorePoint Application What is ScorePoint? • A secure, online, web-based scoring platform • Accessed through Google Chrome • No installation required • Role-based access • Reader • Team Leader • Scoring Director

9. What Is ScorePoint? All of the same paper-based scoring activities can be accomplished in ScorePoint when scoring 2, 3 and 4 point questions such as: • Scoring responses • Marking non-score codes and sending alerts to be escalated • Identifying non-score items where the student left an answer blank • Performing read behinds • Managing scoring periods

10. CBT Operational Scoring Dates ** A teacher may not begin to train for scoring or rate student responses until the primary administration of a specific grade level test that the teacher will be rating has been administered in that teacher's school.

11. Roles and Responsibilities

12. Roles and Responsibilities Reader Responsibilities • Score student responses • Identify and mark alerts while scoring responses for the items assigned to you • Mark responses identified as unable to score • Identify and use the non score code when appropriate

13. Roles and Responsibilities Team Leader Responsibilities • Distribute Login Tickets to Readers • Monitor progress of scoring for Readers and items assigned to your scoring group • Release Reserved Responses • Perform read-behinds for Readers and items assigned to your scoring group • Handle alerts for Readers and items assigned to your scoring group

14. Roles and Responsibilities Scoring Director • Train Team Leaders and Readers • Implement the policies and procedures for scoring provided in the Scoring Leader Handbook, 2019 edition. • Request login tickets • Setup days and times for scoring • Monitor progress of scoring for all Readers and all items assigned to your consortium • Release Reserved Responses • Monitor read-behinds for all Readers and all items assigned to your consortium • Handle alerts for all Readers and all items assigned to your consortium • Generate reports

15. Security & System Recommendations • CBT Scoring Security Recommendations • Consortium to work with local IT staff ahead of scoring to ensure a secure scoring environment • Whitelist the ScorePoint URL (https://ny-scorepoint.questarai.com/scorepoint)and block access to all other websites. • Devices with photographic capabilities or that could be used to transport electronic data should not be allowed • Printing from devices (roles that include printing in ScorePoint) • Scoring room actively monitored at all times • Devices should always be locked with passwords when scoring personnel step away from their workstations and should not be left unlocked and unattended.

16. Security & System Recommendations • General System Requirements: • System Memory/Hard Disk Space • Minimum 512MB Free RAM • Recommended 1GB Free RAM • Minimum 1GB Free Storage Space • General System Requirements: • LAN Network • Recommended available LAN bandwidth at each workstation 2Mbps • Internet Speed • Minimum per device: 150Kbps • Recommended: 300Kbps • Browser • Google Chrome 55+

17. Logging inTeam Leaders & Readers

18. Identifying Yourself Log In and Update Password • Team Leaders will receive the printed login tickets from the Scoring Director • Team Leaders will distribute login tickets to Readers

19. Identifying Yourself Log In and Update Password • Team Leaders distribute login tickets to Readers • Verify login information on ScorePoint • If a Reader or Team Leader has forgotten their password, they will receive a new login ticket from the Team Leaders. Password Requirements • 8-16 characters • one capital letter • one number

20. Identifying Yourself Hands-On Activity: Logging In Open Chrome and clear history • Choose 3 dots in right top corner • Choose More Tools • Choose Clear Browsing Data • Verify all 3 boxes are checked; • Choose Clear Data

21. Identifying Yourself Hands-On Activity: Logging In URL: https://ny-preview-scorepoint.questarai.com/scorepoint For each login ticket you were given: • Log in to ScorePoint using your provided generic credentials • Validate your User Identification and Login Information on your physical login ticket matches ScorePoint • Enter your name and create your new password when prompted • Write your name and new password on your physical login ticket • Log Out Repeat these steps for all login tickets provided to you today.

22. Questions What questions do you have?

23. Readers

24. Scoring Responses • Click Score Responses • Select the item to be scored • Click Score Selected Items

25. Scoring Responses • Display Item checkbox • Response ID & Scores • Note the Response ID if you want to review this response in the Reserved queue.

26. Scoring Responses • Display Item checkbox

27. Scoring Responses • Submitting a score

28. Scoring Responses Hands-On Activity: Scoring a Response • Log in to ScorePoint as a Reader for ELA • Score three responses • Log in to ScorePoint as a Reader for Math • Score three responses Note: All sample ELA and Math responses have the same text for these activities. Verify a new response by the “Response ID”.

29. Questions What questions do you have?

30. Scoring Responses Marking Non Score Codes

31. Scoring Responses Unable to Score

32. Scoring Responses Unable to Score Process • Reader marks a response as Unable to Score • Team Leader locates the response in Reserved Responses queue • Team Leader finishes scoring this response

33. Scoring Responses Reserved Responses • To make sure all student responses are scored, it is critical that you score ALL responses in the queue

34. Scoring Responses How do responses end up in my Reserved Responses queue? • When a response is displayed for you on the Score Responses or Read Behind screen, it is reserved for you.  It remains reserved for you until you actively click on a button to release it.  For example: Submit & Get Next, Cancel & Exit, Unable to Score, etc.. • If you leave a response for any other reason, the system will retain your reservation.  For example: click on another ScorePoint menu option, close your browser tab, or shutdown your workstation, etc..

35. Scoring Responses Reserved Responses Important Notes • After scoring a reserved response, if you click Submit and Next, it will not bring you to the next reserved response, but to a new response in the queue. • To continue scoring your remaining reserved responses, return to the main Score Response screen and select another reserved response to score. • You cannot close out of a project if you have reserved responses in your queue.

36. Scoring Responses Completion of scoring an item Message appears: “All available responses have been scored. Check your reserve bin for remaining responses.” Perform the following steps: 1. Attempt to select the item one more time 2. Check that your Reserved Responses queue is empty

37. Questions What questions do you have?

38. Alerts Marking Alerts • Select Alert Reason

39. Alerts Marking Alerts • Comments: Direct to the location of the content in the response • Do not include details or emotional responses • Example alert text: “See paragraph 2, line 4” • Selecting “Harm to Self or Others” will generate an immediate email to the Scoring Director prompting them to address the alert as soon as possible.

40. Team Leaders

41. Alerts Managing Alerts • Team Leaders and Scoring Directors have access to the Alerts menu

42. Alerts Managing Alerts • Update Alert Status and Comments after reviewing the response

43. Alerts Alert Report • “Alert Problem Sheet With Images/Text” • A Summary of all alerts including student Personal Identifiable Information (PII) • Alerts stay on Alert Report even after Team Leader adds comments.

44. Alerts Hands-On Activity: Managing Alerts • Log in to ScorePoint as a Reader for ELA • Score one item marking it as an alert with appropriate comments • Log in to ScorePoint as a Team Leader for ELA • Using the Alerts menu option, update and add new comments to the alert I will now login as a Scoring Director • Run “Alert Problem Sheet With Images/Text” report to view details

45. Questions What questions do you have?

46. Performing Read Behinds • Number of Read Behinds is a local decision • Options to search for responses to Read Behind. • Required Fields: Content Area, Grade, Item, and Staff number/or First and Last Name 2 3 4 1

47. Performing Read Behinds When you agree with the Reader’s score • Click Submit and Exit Important Note: Read Behinds must occur the same day as the initial scoring for that response.

48. Performing Read Behinds If you agree with the Reader’s score after your review, click Submit and Exit.

49. Performing Read Behinds When your score does NOT match the Reader’s Score • Consult with the reader in person at your workstation

50. Performing Read Behinds When your score does NOT match the Reader’s Score • Consult with the reader in person at your workstation