North Carolina

1. North Carolina National Assessment of Education(NAEP) 2009 LEA Training --Session B December 2008 Iris L. Garner, Ph.D. igarner@dpi.state.nc.us NAEP State Coordinator North Carolina Department of Public Instruction 1

2. Session B--Training Objectives: 1) To illustrate the NAEP sampling and data collection design 2) To describe the corresponding NAEP activities that take place before, during, and after the assessment

3. NAEP Sampling & Data Collection Design Training Objective #1

4. NAEP 2009Assessment Assessment Window: 1/26/09 – 3/6/09 Subjects to be Assessed: Reading, Math, Science Some schools will have a Science Interactive Computer Task (SCIT) or Hands-on Task (HOT) for the science portion of the Field Test.

5. NAEP 2007 NC Performance Math grade 8: NC average score was (284) in 2007 was higher than that of the nation’s public schools (280). Reading grade 8: NC average score (259) in 2007 was not significantly different from that of the nation’s public schools (261).

6. NAEP 2009Assessment Results will be reported at the state and national level The assessment will take approximately 90 minutes of student time (two 25 min. subject matter blocks and survey questions). Students take one subject

7. NAEP Design: Sampling and Data Collection • Multi-Stage Stratified Random Sampling • selection of schools • students within schools are selected randomly • List of schools are obtained from the National Center of Education Statistics (NCES) Common Core of Data (CCD)

8. NAEP Design: Sampling and Data Collection The Common Core of Data (CCD) • U.S. Department of Education's National Center for Education Statistics Program • Annually collects fiscal and non-fiscal data about all public schools, public school districts and state education agencies in the United States • Data are supplied by state education agency officials

9. NAEP Activities Training Objective #2

10. NAEP Team Structure NAEP Field Manager(Gene Biggerstaff) NAEP Coordinator (Iris Garner) Field Supervisor Field Supervisor Assessment Coord. (ACs) Assessment Admin.(AAs) Schools

11. Before Assessment

12. NAEP Activities Preliminary Activity • The North Carolina Department of Public Instruction notified the districts of the school that were selected to participate in NAEP. • NAEP Data Collection Forms have been completed and submitted on-line via the MyNAEP Web site at www.mynaep.com. • NAEP field staff started contacting the school test coordinators on 12/15/08 to set up the Pre-Assessment Visit (PAV).

13. NAEP Activities Preassessment Visit (PAV) • PAV will be scheduled between January 6 and January 23. • PAV should take approximately two hours. • The visit could take longer if the school test coordinator has not prepared for the PAV. • Prior to the PAV you will receive a packet of materials. Please refer to the INSTRUCTIONS FOR THE SCHOOL COORDINATOR.

14. NAEP Activities Before Assessment Before the Preassessment Visit 1) Distribute questionnaires to staff most knowledgeable about the student • Refer to the Guide for Completing SD/ELL Questionnaires • Select the method for completing the questionnaire: online or hardcopy

15. Questionnaire GuidanceStudent with Disabilities & English Language Learners Students with Disabilities (SD) and English Language Learners (ELL) Questionnaires Questionnaires for each student in the sample that was identified as SD or ELL. The questionnaires need to be filled out and returned to the coordinator prior to the PAV. If you have more students in the sample that are identified as SD/504 or ELL, let the NAEP field staff if additional questionnaires are needed.

16. Student with Disabilities and English Language Learners If the student participates in the regular EOGs or EOCs with or without accommodation, they should participate in NAEP. If the student can access a “paper/pencil” assessment, they should participate in NAEP with accommodations. In 2002, NC had the highest exclusion rate in the nation for grade 4 reading. The exclusion rate has reduced in 2003, 2005 and 2007.

17. Student with Disabilities and English Language Learners The exclusion rates will be recorded after the pre-assessment visit and will be monitored for high rates. NAEP field staff will administer all accommodated sessions. These sessions may have to occur after the standard administration. If a special needs student feels more comfortable with a known test administrator, then the teacher can administer the assessment with the NAEP field staff in the room.

18. Questionnaire GuidanceStudent with Disabilities & English Language Learners • Questionnaire Guidance Includes: Folder-A Guide For Completing The Student With Disabilities and English Language Learner Questionnaires • Frequently Asked Questions • Step-by-step directions • Accommodations Chart • State-specific policies

19. Questionnaire GuidanceStudent with Disabilities & English Language Learners • Students assessed using the NCEXTEND1 most likely will not be able to be assessed on NAEP. • Students assessed using the NCEXTEND2 most likely will be assessed on NAEP since they can access a “paper-pencil” assessment. • Supply of Blank Paper: The NAEP assessment does not allow a supply of blank paper. All notes, explanations, etc. should be written directly in the test booklet.

20. Questionnaire GuidanceStudent with Disabilities & English Language Learners • All students that take the NAEP assessment are allowed to write in the test booklet. NAEP does have separate answer sheets. Writes in test booklet is not considered an accommodation for NAEP.

21. Questionnaire GuidanceStudent with Disabilities & English Language Learners • A Spanish version of the NAEP assessments in Mathematics and Science is available. North Carolina at this time does not have translated versions of the state tests. This accommodation is not allowed. • Except for the items listed above the North Carolina State Testing Program accommodations can be used with the NAEP assessments. If an accommodation is part of a student’s Individualized Education Program (IEP) or 504 plan and is used regularly in the classroom, and on the state assessments, it should be able to be used on the NAEP 2009.

22. NAEP Activities Before Assessment Before the Preassessment Visit 2) Distribute the parent letters

23. Parent Letter Parent letters will need to be sent home to students selected to participate in the NAEP assessment prior to the PAV. Use school letterhead Include date

24. Parent Letter Share the Sample Questions Booklet with the parents.

25. NAEP Activities Before Assessment Before the Preassessment Visit 2) Review and update Student Information Report – List of all participating students and demographic information. Note: The Student Information Report for the National Indian Education Study (NIES) are included in the Important NIES Material envelope.

26. NAEP Activities Student Information Review • Collect any missing demographic information. • Review student demographic data for accuracy. • Determine if there have been any changes in students identified as SD and/or ELL. • Identify which students, if any, have withdrawn or graduated, and who are neither enrolled nor attend the school in the grade to be assessed. • Make all corrections directly on the report.

27. NAEP Activities Before Assessment—Before the Preassessment Visit • 3) Print out a current list of the students in the selected grade • 4) Complete logistics questionnaire.

28. NAEP Activities Before Assessment—During the Preassessment Visit • At the beginning of the PAV the NAEP field staff will meet with you to collect the information you completed prior to the PAV: • Printed list of current students at the selected grade • Corrected Student Information Report • SD and ELL Questionnaires • Copy of the parent letter • Logistics Questionnaire

29. After the Pre-Assessment Visit • Notify the students of the assessment. • Notify the teachers of the assessment. • Place the NAEP Storage Envelope in a safe place.

30. Supporting NAEP in Your School • NAEP has produced a short movie that you can share with your teachers and students to help them understand the importance of NAEP. • You can access Introducing NAEP to Teachers at the NCDPI Web site: http://www.ncpublicschools.org/accountability/policies/naep/5 • You can access Introducing NAEP to Students at the NCDPI Web site: http://www.ncpublicschools.org/accountability/policies/naep/5

31. Before Assessment

32. NAEP Assessment Day Assessment Day • The NAEP assessment team will arrive at the school an hour and half prior to the scheduled assessment time. • The NAEP assessment team will need a staging area throughout the assessment day. • The NAEP team will administer all assessments, including the accommodated sessions.

33. NAEP Assessment Day Assessment Day • NAEP does not need proctors for the assessments. • Classroom teachers and/or other school personnel are welcome and encouraged to stay in the room during the NAEP assessment. • School personnel will have to sign a confidentiality agreement if they observe the NAEP assessment.

34. NAEP Activity Assessment Day • Your biggest role on assessment day is to make sure all students in the sample arrive at the designated assessment locations. • You may need to work with you attendance staff to make sure missing students are really absent that day. • If 90% of the students in the sample are not there that day, then a make-up session will need to be scheduled. • The 90% does not include students who have withdrawn, students who’s parents have refused participation, or students who were excluded.

35. After Assessment

36. NAEP Activity After Assessment NAEP representative conducts a short interview with the school coordinator • A make up assessment day is scheduled if needed. • The school coordinator will securely destroy the contents of the storage envelope on the date listed on front of the envelope. • The school coordinator will mail the postcard to NAEP stating that the materials have been securely destroyed.

37. Thanks You We appreciate all you do; you make the NAEP administrations possible.