2019 TELPAS Holistic Training

1. 2019 TELPAS Holistic Training Brownsville ISD – Assessment, Research & Evaluation Department

2. Agenda • Test Security & Confidentiality Requirements • Campus Training Requirements • Selecting Raters & Writing Collection Verifiers • Raters & Verifiers Responsibilities • Writing Collection Overview • What’s Next?

3. Disclaimer This training does NOT take the place of reading the appropriate manuals.

4. TELPAS Test Security and Confidentiality Requirements

5. Test Security • Test security of the TELPAS program involves protecting the contents of all assessments and student performance documentation. This requires compliance with, but is not limited to, the following guidelines • Personnel must meet the requirements, have been trained and have signed an oath to have access to the assessment materials and information • Campuses must implement the controls necessary to ensure proper storage of secure materials throughout the test administration • Campuses must place confidential documentation (i.e. – rating rosters) in a limited-access locked storage

6. Confidentiality Requirements • Testing personnel undergo training and sign the oath affirming they understand their obligations concerning the security and confidentiality of the TELPAS program • Testing personnel must administer the tests in strict accordance with the instructions • Testing personnel do not view, reveal or discuss the contents of an assessments before, during or after a test administration

7. Confidentiality Requirements • No person may duplicate, print, record, write notes about, or capture any portion of an assessment (electronically or by any other means) • Only students may respond to test questions • No person may receive or provide answers to student profiles during TELPAS calibration activities

8. Placement of restrictions on renewal or holding of educator certificate • Issuance of an inscribed or non-inscribed reprimand • Suspension of a Texas educator certificate for a set term • Revocation or cancelation of an educator certificate without opportunity for reapplication for a set term or permanently Penalties for Prohibited Conduct Any person who violates, assists in the violation of, or solicits another to violate or assist in the violation of test security or confidentiality, as well as any person who fails to report such violation is subject to the following penalties

9. Release or disclosure of confidential test content could result in criminal prosecution under TEC §39.0303, Section 52.352 of the Texas Government Code, and Section 37.10 of the Texas Penal Code. Further, 19 TAC §249.15 stipulates that the State Board for Educator Certification may take any of the above actions based on satisfactory evidence that an educator has failed to cooperate with TEA in an investigation. • Additionally, irregularities resulting in a breach of test security or confidentiality may result in the invalidation of students’ assessments. Penalties for Prohibited Conduct

10. Security Oaths and Confidentiality Statements Oaths must be kept on file for at least five years All personnel participating in state mandated testing must • Meet the eligibility requirements for their role • Be trained • Sign the test security oath Each oath must be read and completed AFTER the training and BEFORE handling or viewing any secure materials or confidential information

11. Testing Irregularities Serious Examples These constitute severe violations of test security and/or confidentiality and can result in the individual(s) responsible being referred to the TEA Educator Standards and Certification Legal Division for consideration of disciplinary action. • Using answer sheets or notes that contain answers to the TELPAS calibration activities • Receiving or providing answers to student profiles during TELPAS calibration activities • Falsifying TELPAS ratings or writing samples • Fraudulently exempting a student from the administration of TELPAS • Assisting students directly or indirectly with responses to test questions

12. Testing Irregularities NOTE!!!! It is a violation of state assessment procedures and a serious testing irregularityto record, discuss, or share answers to the rating practice and calibration activities. After completion of calibration activities, raters must turn-in all notes taken during the calibration to the proctor.

13. Testing Irregularities Procedural Examples Eligibility error IEP implementation issues Improper accounting of secure materials Monitoring error Other procedural error Less severe and more common irregularities that are typically the result of minor deviations in testing procedures.

14. Testing Irregularities – Procedural Examples Eligibility Error IEP Implementation Error SPED student was/was not provided an accommodation • Student not rated in one or more domains • Student not administered the reading or listening & speaking test • Ineligible student was assessed

15. Procedural error TA who wasn’t trained rated students A writing collection was not submitted according to the assembly criteria TA failed to provide student the correct test ticket for online testing Testing Irregularities – Procedural Examples Improper Account of Secure Materials error • Teacher misplaced a writing collection • CTC did not collect writing collection from Teacher Monitoring error • TA left room unattended when online tests were open • Student was not prevented from using cell phone

16. Reporting testing irregularities • Each person is responsible for reporting ANYviolation or suspected violation • Campus staff should notify their CTC or DTC if they witness an irregularity or suspect one has occurred • DTC will contact TEA to report ANY incidents as soon as DTC is made aware of the situation • Contact DTC if unsure about whether an irregularity has occurred

17. Campus Training Requirements

18. Responsibilities of telpas coordinators You are expected to: Train staff on holistic rating training requirements Train staff on writing collection procedures and timelines Ensure raters understand the holistic rating process Ensure raters are informed about the date, time, location of monitored sessions for calibration Understand the consequences of cheating during calibration activities

19. Campus Training & Documentation Requirements TELPAS Coordinators are expected to: • Schedule monitored calibration sessions • Establish procedures to ensure validity and reliability • Maintain documentation of procedures followed

20. Prepare For Training Coordinators ensure that: • TELPAS Raters & Verifiers are identified and properly trained • Procedures are established to collect & verify the years in the U.S. • Ensure that testing infrastructure is in place • Ensure that assessment information is entered and verified • Materials are collected and filed as appropriate • Ensure security oaths are signed • All ELL’s are identified and assessed

21. Planning the Online Training Components Online BASIC TRAINING Course • Required for all NEW ratersin Brownsville ISD • Submit certificate to TELPAS Coordinator • May be completed at home • May be used as a review • Gives raters practice rating students in each language domains. (approximately 4-5 hours)

22. Planning the Online Training Components Online CALIBRATION Course • Required for ALL raters. • Each calibration set contains 10-12 students to rate and all holistically assessed language domains are represented in each set. (approximately 1- 2 hours) • Raters taking the grades Kinder-1st course – will rate listening, speaking, reading and writing • Raters taking the Grades 2nd – 12th course – will rate writing ONLY

23. When/Where to Complete Online Activities • All raters must complete calibration in a monitored session that has been established by the campus. • Monitored calibration sessions may take place during the school day, after school, or as part of professional development. It is a violation of state assessment procedures to record, discuss, or share answers to the rating practice and calibration activities. Prior to creating a TELPAS Online Training Center account, raters are required to read an online statement and affirm that they will complete the rating activities independently.

24. Monitored Calibrations Requirements • All raters must receive administration procedures training prior to attending a calibration session • All raters must have a signed oath on file. • A roster of participants in each calibration session must be maintained. • Raters will be given a passcode by the session proctor in order to access calibration. • Raters may be provided a copy of the PLDs. The copy must be turned-in to the proctor at the end of the calibration session. • Raters may be provided scratch paper. The scratch paper must be turned-in to the proctor at the end of the calibration session. • Raters will need to complete calibration activities individually within the monitored group session.

25. Monitored Calibration Sessions • Raters must begin with calibration set 1. If successful, a certificate will need to be printed and turned in to the proctor. • If rater is unsuccessful on set 1, they will be required to • Complete a supplemental support training session. • After completion of supplemental training, raters will be notified by TELPAS Coordinator when set 2 will be held. • After completing a calibration set, raters see results and annotations explaining the correct ratings. • Raters will need 1-2 hours to complete a calibration set. • Raters must finish a calibration set once it is started.

26. Raters who fail to calibrate • If a rater fails to calibrate on set 1, they will have to complete a supplemental support training session provided by the Bilingual Department • Option 1- Face to face training: Two sessions have been scheduled for raters who fail to calibrate • Session 1: Monday, February 18 from 5:00 to 6:30 pm., CAB 220 • Session 2: Wednesday, February 20 from 5:00 to 6:30 pm, CAB 220 • Option 2 - Online Training: An online training module will be available beginning February 13 for raters who prefer to complete their training on their own.

27. If a rater fails both attempts to calibrate: • The district may choose not to assign the individual to be a TELPAS rater. • If the individual is needed to serve as a rater, the district & campus will implement rater support procedures to ensure that the rater’s students are evaluated consistent with the rating rubrics. What To Do If Someone Is Unsuccessful after Calibration Individuals are not authorized to serve as raters unless they complete all required training components.

28. Recap of Calibration Process • Calibration is conducted in monitored sessions. • A passcode is required to access calibration. • There are 2 calibration sets. • Raters who calibrate on set 1 are done. • Raters who don’t calibrate on set 1 must complete set 2 after support from Bilingual Department has been provided. • Certificates must be printed and turned in to the proctor.

29. Document Retention • TELPAS Coordinators must be aware of all the documentation that must be kept in the event of an audit. • All documentation necessary will be posted in a document titled SCANNING LAYOUT in the A.R.E. website. Documents needed include (not limited to): • Training Documentation: Workshop Attendance and Agendas for both Holistic and Online Testing, 2019 Training Certificates, Oaths… • Testing: Procedures, Seating Charts, Restroom Logs, Materials/Ticket Control Form…

30. Selecting Raters & Writing Collection Verifiers

31. Rater Credentials Each teacher selected to rate an ELL must • have the student in class • be knowledgeable about the student’s ability to use English in instructional and informal settings • hold valid Texas education credentials such as a teacher certificate or permit • be appropriately trained, as required by TEA • rate the student in all eligible language domains

32. TELPAS Rater Responsibilities • A student’s TELPAS rater is the teacher designated by the district as the official rater of the student’s English language proficiency. • A Kindergarten or 1st grade student’s rater must rate the student in all domains for which the student is eligible. It is not permitted for a student to have one rater for some domains and another rater for other domains.

33. Training

34. Raters must know their assigned grade cluster to select the appropriate online training. • Raters should consult with their campus coordinator if they are unsure of their assigned cluster. Rater Calibration Clusters Grade Clusters Grades K–1 Grade 2 Grades 3–5 Grades 6–8 Grades 9–12

35. Raters with students in multiple grades • Raters are required to train and calibrate in the cluster in which they have the most ELLs. • As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s). • Example:A grade 2–5 ESL teacher has most of her ELLs in grade 3. She must complete training course and calibration in grades 3–5. As a best practice, she should review the online practice activities for grade 2 to check her readiness to apply the rubrics appropriately.

36. Raters with students in multiple grades (K-2) • Because of differences in the instructional content and rating rubrics, these raters must complete training and calibration for K–1 and at least one other cluster in 2–12. • Raters with more than one additional cluster should train in the cluster in which they have the most ELLs. • As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s). Example: A grade 1–3 ESL teacher has most of his ELLs in grade 2. He must complete training course and calibration in grades K–1 and 2. As a best practice, he should review the online practice activities for grades 3–5 to check his readiness to apply the rubrics appropriately for his 3rd grade students.

37. Writing Collection Verifiers • Selected by Campus Principal and TELPAS Coordinator • Any certified personnel can serve as a verifier • Writing collection verifiers MUST be trained • Verifiers CANNOT be raters

38. Raters & VerifiersRESPONSIBILITIES This training does not exempt you from reading the required manuals!

39. TELPAS Rater Responsibilities

40. TELPAS Online Training Center Reminders • The TELPAS Online Training Center does not save training histories or certificates. Accounts and certificates are purged from the training site at the end of each training window. • Pearson and TEA do not have access to previous year certificates; staffshould maintain a record of completed rater training. • District coordinators will receive an email containing the keywordusers need to create an account for this year.

41. TELPAS Online Training Center - Updates • Each training course will now have a practice activity outside of the modules. • After completing the modules and practice activities, certificates of completion will be available to print in the scoring summary tab. Testing coordinators will no longer need to print certificates. • The Grades 2–12 Online Basic Training Course and calibration activities will now only include the domain of writing. • A separate holistically-rated listening and speaking rater training will be available for the rare circumstances in which a student cannot access the online listening and speaking assessment.

42. TELPAS Online Training Center Log In/Sign Up TexasAssessment.com/TELPASTrainingCenter

43. Create Account • Coordinators will receive a keyword for users to create an account. • All fields are required. Check box for Rater Agreement and click the Submit button.

44. Educator Homepage

45. Choose Your Training • Raters must know their assigned grade cluster to select the appropriate online training. • Raters should consult with their campus coordinator if they are unsure of their training requirements. • Writing Collection Verifiers should select that course.

46. Holistically Rated Listening & Speaking Training • T.E.A. approval is required to holistically rate a student (in grades 2-12) in listening and speaking • Passcode will be provided with T.E.A. approval

47. Writing Training Course

48. Listening and Speaking Course

49. Practice Sample

50. Sufficient Calibration • For K–1, each set contains ten students to be rated. Raters must rate at least 70 percent of the students correctly. • For 2–12 grade clusters, each set contains ten writing collections to be rated. Raters must rate at least 70 percent of the collections correctly. • For 2-12 listening and speaking, each set contains six students to be rated. Raters must rate at least 67 percent of the collections correctly