Administering the 2010 HSTW Assessment for TCTW Sites

1. Administering the 2010 HSTW Assessment for TCTW Sites

2. About the HSTW Assessment • HSTW and TCTW are research-based school improvement models • HSTW Assessment is used to set baselines, acknowledge progress, identify areas for improvement, and set priorities • Only initiatives combining student achievement data with students’ perceptions of school and classroom practices • Required of HSTW and TCTW sites in even-numbered years • Administered to seniors to collect data on entire high school experience

3. Examples of How Schools and Centers Use the Results • Lobby for state actions to support school improvement • Identify a need for improvement • Gain support for school improvement • Engage the entire faculty in school improvement • Set priorities for school improvement • Improve classroom practices

4. Examples of How Schools and Centers Use the Results • In 2008, one TCTW center said “the teacher survey was a real eye-opener. Through the data, we very quickly realized that we had teachers on our campus with very different views regarding the purpose of career/technical education, particularly in the area of academic reinforcement. The survey data will allow us to identify the highest priority staff development needs and to begin creating a common understanding of our center’s role in a students’ technical and academic success.”

5. Five Components • Reading test • Mathematics test • Science test • Student survey • Course-taking patterns • School/center and classroom experiences • Teacher survey

6. Reading Test • 90 minutes • 80% informational texts; 20% literary non-fiction texts • Includes meaning vocabulary • Includes paired texts (to assess critical analysis) • Does not include literary fiction or poetry

7. Mathematics Test • 70 minutes • Content • 10% Number Properties and Operations • 30% Measurement/Geometry • 25% Data Analysis, Statistics and Probability • 35% Algebra

8. Science Test • 70 minutes • Content • 40% Life Sciences • 40% Physical Sciences • 20% Earth and Space Science

9. Student Survey • Section 1 • Course-taking patterns • Sections 2 and 3 • School and classroom experiences • Based on HSTW Key Practices

10. Teacher Survey • Center and classroom experiences • Center mission • High expectations and extra help • Guiding and supporting students • Curriculum content and engaging students in learning • Transitions • Leadership and professional growth

11. Results • HSTW Award of Educational Achievement • Student reports • School report

12. HSTW Award of Educational Achievement • Mailed to centers April 30, 2010 • Awarded to students who: • Meet all three readiness goals • Complete at least two parts of the HSTW-Recommended Curriculum • English/language arts, mathematics, science • Complete a concentration • Career/technical, mathematics/science, humanities

14. Student reports • Mailed to centers April 30, 2010 • Performance on reading, mathematics and science test • Performance level • Completion of recommended curriculum and concentration • Award status

16. HSTW Assessment Report • Mailed to centers June 30, 2010 • Two months earlier than previous years! • Executive Summary • One page summary of performance and implementation • Overview • Slightly longer summary • Full report • Tables breaking out achievement by experiences • **No teacher survey results • Teacher survey results will be provided in a separate report.

21. Administering the Assessment 1. Place order • Order window • September 1 – October 30, 2009 • Orders must be placed online on SREB Web site • All communication and materials will be sent to contact person listed on order form

22. Information needed to place order • If your center has previously participated in an assessment • Name, e-mail address and phone number of contact person (will receive all materials and communication via mail and e-mail through summer 2010) • Complete shipping address (no P.O. boxes) • Director name, e-mail address and phone number • Sampling method and number of students to be assessed • Payment information and billing address • Dates the center plans to assess within window • Number of teachers

23. Payment Information • If your center or district is paying for the assessment, a purchase order is due to SREB by November 16, 2009 to confirm your order • Final payment is due to SREB by April 30, 2010 • SREB will accept payments beginning immediately (call Fran Mathis at 404-879-5611 to request an early invoice)

24. Administering the Assessment 2. Review administration materials • Available on SREB Web site beginning September 1, 2009 • A link also will be included in the order confirmation e-mail sent upon placing an order • Review materials immediately • Set timeline for preparation and administration

25. Administration Materials • Test Administration Guide • Sampling Instructions • Student Roster Form • Course Description Guide • Course Matching Chart • Checklist • Test Description and Sample Question Booklet • Information for Sites • Tasks and Timelines

26. Sample Timeline

27. Administering the Assessment 3. Identify students • Use sampling plan selected on order • All seniors • Random sample of 60+ seniors • All CT seniors • Random sample of 60+ CT seniors • Include all currently enrolled students in sampling • Only exclude students if IEP exempts them from state assessments • Use sampling instructions provided to select sample

28. Identifying Students to Participate • Only include seniors • If selecting a sample, select a true random sample using the instructions provided • Only exclude students if their IEP exempts them from state assessments • Accommodations provided for state assessments should be provided for this assessment • Contact ETS by October 30, 2009 to request large-print or CD materials • Accommodations exist for English language learners

29. Identifying Students When Home Schools Are Also Participating • Center should select sample. • Home school should select sample. • Center and home school should compare lists to see if any students were double-selected. • If no students were double-selected, proceed as usual. • If students were double-selected, students should assess at home school. • Call ETS (Lisa Rion – 609-734-5646) to obtain a special code. • Enter that special code on all double-selected students’ answer documents. Their results will go into both reports while they only participate once.

30. Administering the Assessment 4. Scheduling • Identify assistants • One assistant per 15 students is ideal • Schedule dates, times and locations • Include make-up sessions • SREB recommends administering assessment over 2-3 days • Schedule date, time and location for teacher survey • Web-based teacher survey

31. Administering the Assessment 5. Notification • Notify students, parents and teachers • Provide informational materials and/or schedule an informational meeting

32. Administering the Assessment 6. Prepare student roster form • Include names of all participating students • Will need to include each student’s Assessment ID number from the answer document he or she is assigned • Keep fully completed roster on file with testing records for five years • Recommend including students’ e-mail addresses to assist with Follow-Up Survey of 2010 Graduates • Needed to identify students for student reports and awards if they failed to include their name on their answer document

33. Administering the Assessment 7. Course-taking preparation • Work with home schools to complete course-matching chart, using course description guide • Used to link local course titles to generic SREB course titles • Include all courses current seniors could have taken during high school • Collect transcripts and course schedules for participating students, including spring courses

34. Administering the Assessment 8. Receive assessment materials • Materials schedule to arrive by January 20, 2010 • TCTW Teacher Survey instructions will be sent separately • Verify contents according to instructions immediately • This gives ETS time to send new materials should an error have occurred • Keep secure (locked) until testing begins • These are secure testing materials and all materials (used and unused) will need to be returned • Keep the boxes – you will need them to return materials

35. Administering the Assessment 9. Prepare assessment materials • Assign each student an answer document • Record each student’s Assessment ID number from the answer document on the student roster form • Complete section 1 (transcript analysis) for students • Strongly recommended, but not required • Reduces testing time • Increases accuracy

36. Administering the Assessment 10. Review testing arrangements • Confirm dates, times and locations • Remind participants • Review administration plan with assistants

37. Administering the Assessment 11. Administer assessment • Student and Teacher Window: January 29 – February 19, 2010 • Administration guide includes scripts and complete instructions for administering the student assessment • Students must complete all four components • All materials must be postmarked for return using the enclosed instructions and labels no later than February 19, 2010 • TCTW Teacher Survey will be Web-based and must be completed by February 19, 2010.

38. Administering the Assessment 12. Return materials • Pack student materials using enclosed instructions • Completed answer documents will be packed in a different envelope and use a different return label than the test books and other materials • If the necessary envelopes and labels are not included, you must call ETS (Lisa Rion at 609-734-5646) for replacement materials • Teacher survey will be Web-based • All materials must be postmarked for return no later than February 19, 2010 • Keep the receipt and tracking number!!!

39. Important Dates to Remember • Order window • September 1 – October 30, 2009 • Administration materials available online • September 1, 2009 • Test materials arrive at sites • January 20, 2010 • Student and teacher administration window • January 29 – February 19, 2010 • Final return deadline • February 19, 2010 • Awards and student reports mailed to sites • April 30, 2010 • Payment due to SREB • April 30, 2010 • School reports mailed to sites • June 30, 2010

40. Making the 2010 Assessment a Success • Communication • Letters/informational sessions for students and parents • Announcements to faculty and staff • Stress importance, intended use, and value of assessment

41. Making the 2010 Assessment a Success • Cooperation • Importance of the assessment to the center’s improvement goals • Importance of students, administration and faculty taking it seriously • Handling make-up work • Cooperation from teachers whose classes might be interrupted by the assessment

42. Making the 2010 Assessment a Success • Encourage all students to do their best • Promote the HSTW Award of Educational Achievement • Recognize those who receive the HSTW Award of Educational Achievement • Explain the new student readiness reports and how it can help with their resumes • Build enthusiasm • Be honest – explain how this helps the center and future students • Make students feel special for participating in the assessment and thank them for their participation

43. For More Information… Go to SREB’s Web pages on assessment for more ideas and information! http://www.sreb.org/programs/hstw/Assessment/assessindex.asp **Please note SREB will be launching a new Web site this fall. The assessment direct link will be different but the new Web site will be easier to navigate. Allison Timberlake, Director of Assessment and Evaluation allison.timberlake@sreb.org (404) 879-5575 Fran Mathis, Research Assistant fran.mathis@sreb.org (404) 879-5611 Ann Benson, Director of Technology Centers That Work gben@brightok.net