Florida

1. Florida National Assessment of Educational Progress (NAEP) 2009 Assessments DISTRICT PRESENTATION Michele Sonnenfeld NAEP State Coordinator Florida Department of Education

2. National Assessment of Educational Progress (NAEP) • The National Assessment of Educational Progress (NAEP), also known as the Nation's Report Card, is the only ongoing nationally representative assessment of what U.S. students know and can do in various subject areas. • NAEP is authorized by Congress and administered by the National Center for Education Statistics (NCES) in the U.S. Department of Education.

3. National Assessment of Educational Progress (NAEP) • Since 1969, NAEP has been conducted periodically at the national level in reading, mathematics, science, writing, U.S. history, civics, geography, and the arts. • Starting in 1990, NAEP has also been conducted at the state level to provide participating states with results in reading, mathematics, science, and writing. • Beginning with the 2003 NAEP administration, all selected schools must participate in the assessment as required by the No Child Left Behind Act of 2001. • Florida has participated in NAEP at the state level since 1990.

4. National Assessment of Educational Progress (NAEP) • Because state-level assessments vary from state to state, the results cannot be used to compare student performance across states. • NRT results cannot be used because it is not administered in every state. • The SAT cannot be used because it is administered to a self-selected group. • With NAEP, a sample of students in all 50 states, Washington D.C., the Department of Defense schools, Puerto Rico, and certain urban districts each take a portion of NAEP. • The sampled students represent the geographic, racial and ethnic, and socio-economic diversity of this country’s student population.

5. No Child Left Behind Act of 2001 • Section 111(c)(2) of NCLB requires states/districts that receive Title I funding to participate in biennial state-level NAEP administrations of Grades 4 and 8 reading and mathematics, beginning with the 2003–2004 school year. • Names of schools and students selected to participate in NAEP are kept confidential. • NAEP does not report data for individual students, schools, or districts (except in certain Trial Urban Districts). • National results are produced for Grades 4, 8, and 12. • State-level results are produced for Grades 4 and 8.

6. NAEP in Florida Florida State Statute 1008.22 (2) “The Commissioner of Education shall direct Florida school districts to participate in the administration of NAEP . . both for the national sample and for any state-by-state comparison programs.”

7. Current OrganizationalStructure of NAEP

8. National Assessment Governing Board (NAGB) • NAGB was established in 1998 by Congress to oversee and set policy for NAEP. The Board is responsible for: • Selecting subject areas to be assessed and setting the assessment schedule; • Developing subject frameworks that serve as the basis for the assessments; • Developing guidelines for the reporting and disseminating of NAEP results; and • Planning and executing the initial public release of NAEP reports.

9. NAEP Test Items • NAGB develops the test specifications. • Test items are developed by teachers, subject area specialists, and assessment experts and reviewed for quality, bias, and sensitivity by content area experts. • Both multiple-choice and constructed-response questions are included in the assessment. • No one student takes the entire NAEP assessment. • Each student receives one booklet of approximately 16 to 20 questions.

10. NAEP Frameworks • http://nces.ed.gov/nationsreportcard/frameworks.asp

11. Sampling • Every school and student has some chance of being selected for the sample. • The sample is representative of the entire student population. • Lists of schools are obtained by Westat from the National Center for Education Statistics Common Core of Data (CCD). • NSC reviews the list of schools and flags those that have been closed and supplies list of those that will be opening by the following fall.

12. Sampling • Schools are grouped into strata by two primary characteristics: • Type of location (central city, urban fringe, rural) or • Minority enrollment. • Within strata, schools are ordered by a measure related to achievement. • A proportional sample is selected systematically from the ordered list.

13. NAEP Inclusions and Accommodations • Since 1998, NAEP has included students who require accommodations. • Schools may exclude students according to carefully designated criteria. • NAEP’s most frequent accommodations include: • Extra testing time • Individual or small group administrations • Large-print booklets • NAEP accommodations do not include reading passagesor questions aloud on the NAEP reading assessment. • Florida emphasizes inclusion.

14. Analysis and Reporting-Average Scale Scores NAEP reports average scale scores for groups of students: • Subgroup comparisons (e.g., SD, ELL) • State and regional comparisons Average scale score ranges: • Reading and mathematics scale scores range from 0 to 500. • Science scale scores range from 0 to 300.

15. NAEP Grade 4 Reading Scale Scores 1992–2007 In 1992*, 1994*, and 1998, Florida scored significantly below the nation. In 2002 and 2003, there was no significant difference between the average scale scores of Florida and the nation. In 2005 and 2007, Florida scored significantly higher than the nation. *Accommodations were not offered in 1992 and 1994 nAccommodations were not offered n

16. NAEP Grade 8 Reading Average Scale Scores • Florida was one of only four states to significantly improve its Grade 8 reading average scale scores between 1998 and 2007: Wyoming, Massachusetts, and Delaware.

17. NAEP Grade 8 Reading Average Scale Scores of African American Students • Florida was one of only four states whose African American students significantly improved their Grade 8 reading average scale scores between 1998 and 2007: Georgia, Maryland, and Delaware.

18. Analysis and Reporting-Achievement Levels NAEP achievement levels indicate how well students within the total population, or in a particular subgroup, meet or exceed expectations of what they should know and be able to do. • Advanced - superior performance • Proficient - solid academic performance on challenging subject matter • Basic - partial mastery of prerequisite knowledge and skills that are fundamental for proficient work • below Basic - scale scores that fall below the cut point for Basic

19. Comparing NAEP and FCAT Achievement Levels Grade 4 Reading Comparison of FCAT and Florida NAEP Proficiency Results 1998–2007

20. FCAT AND NAEP • FCAT–measures student performance on selected benchmarks, as defined by Florida’s Sunshine State Standards. • NAEP–reports on the performance of groups of students at a given time and across time, without specifying how a subject should be taught.

21. State-Level NAEP Administrations in Florida 1998–2009 • In 2009, Florida will also be participating in: • Internet Computer Tasks (ICT) • Hands-on-tasks in Science (HOTs) • High School Transcript Study (HSTS) in Mathematics and Science • Grade 12 Pilot Study in Mathematics and Reading • Trial Urban District Assessment (TUDA)

22. Interactive Computer Tasks (ICT) • Westat will provide 15 laptop computers and flash drives. • Students use the computers to perform experiments by manipulating conditions on the screen and then answering questions about the experiments.

23. Hands-on-Tasks in Science (HOTs) • Students perform hands-on experiments using kits provided by Westat and then answering related questions.

24. 2009 High School Transcript Study (HSTS)

25. 2009 High School Transcript Study (HSTS) • HSTS allows researchers to examine the relationship between course-taking patterns and educational achievement by linking HSTS data to NAEP assessment data in mathematics and science. • Florida is the only state that will participate in the HSTS electronically, minimizing the burden to schools. • Schools will need to provide the NAEP field staff with a list of the textbooks used in Grade 12 mathematics and science.

26. How Are Florida’s Grade 12 Students Doing? We don’t know!! NAEP 2009 is the first time that Florida state-level results will be reported for Grade 12.

27. Grade 12 Pilot Assessment • In 2009, NAEP will administer a Grade 12 Pilot Assessment in 11 states, including Florida. This will be the first time state results will be available for Grade 12 in mathematics and reading. • Florida will be able to compare its Grade 12 mathematics and reading results with those of Idaho, South Dakota, Iowa, Illinois, Arkansas, West Virginia, New Jersey, Massachusetts, Connecticut, and New Hampshire.

28. NAEP 2005 Reading Average Scale Scores, Grade 8, All Students, Grade 12 Study States We looked at how Florida seniors did four years ago as Grade 8 students on the NAEP 2005 reading assessment compared to the performance of their peers in the other NAEP 2009 Grade 12 Study states. Florida scored • lower than 8 states and the nation, • not significantly different from 2 states, • higher than none of the other 10 states participating in the study.

29. NAEP 2005 Mathematics Average Scale Scores, Grade 8, All Students, Grade 12 Study States We looked at how Florida seniors did four years ago as Grade 8 students on the NAEP 2005 mathematics assessment compared to the performance of their peers in the other NAEP 2009 Grade 12 Study states. Florida scored • lower than 8 states and the nation, • not significantly different from 1 state, • higher than 1 state.

30. Trial Urban District Assessment (TUDA) • TUDA is designed to report at the district level on the performance of Grades 4 and 8 public school students in reading and mathematics. • 2002 – Atlanta, Chicago, Houston, Los Angeles, New York City • 2003 – added Boston, Charlotte, Cleveland, San Diego • 2005 – added Austin • 2009 – added Baltimore, Detroit, Fresno, Louisville, MIAMI-DADE, Milwaukee, and Philadelphia • Washington, D.C. is also included in TUDA

31. TUDA Districts

32. NAEP 2009Assessment Schedule and Components • Assessment Window: January 26 to March 4, 2009 • Subjects to be Assessed: • Grades 4, 8, and 12 – Reading, Mathematics, or Science • Grades 4 and 8 – Science Interactive Computer Tasks (ICT) or Hands-on-Tasks (HOTs) • Grade 12 – Science Interactive Computer Tasks (ICT) and Hands-on-Tasks (HOTs) • Grades 4, 8, and 12 – U.S. History, Civics, or Geography

33. Tasks Completed by Schools June–November 2008 • Receive and post the assessment date on your school calendar. • Use the MyNAEP Web site to register as a new MySchool user and complete the School Data Collection Form online.

34. MySchool Web Site–Step 1

35. 2009 School Data Collection Form

36. My School Web Site–Step 2

37. Parent/Guardian Notification Letter • Download the Parent/Guardian Notification Letter and customize bracketed information at http://www.fldoe.org/asp/naep/naep2009.asp or https://www.mynaep.com/default.asp, Step 2. • Send a copy of the customized letter to your District Assessment Coordinator.

38. Tasks Completed by the Schools June–November 2008 • Review the Preparation Checklist for School NAEP Coordinators.

39. Tasks Completed by Schools June–November 2008 • Review Florida’s Instructions for the School NAEP Coordinator.

40. Tasks Completed by Schools June–November 2008

41. Tasks Completed by Schools June–November 2008 • Work with teachers to make sure they understand the importance of NAEP. • Begin motivating students to participate in and do their best on NAEP. • Make plans to hold a meeting to explain NAEP and how important it is. • Identify school incentives to use to encourage student participation.

42. School Incentives • Free tickets to school events • Coupons to local eateries • A raffle for participating students • Pizza party • Complimentary breakfast • Community service credits

43. Best Practices for Improving Twelfth-Grade NAEP Participation Guide includes: • Strategies effective in engaging Grade 12 students in NAEP • Talking points that principals can use with teachers and parents • DVD and PowerPoint to share with teachers and students.

44. Yes, obviously NAEP is important and I want do my best on the NAEP 2009 assessment!

45. Tasks Completed by SchoolsDecember 2008–January 2009 • When a NAEP field staff representative calls in December, make an appointment for a Pre-Assessment Visit (PAV) to take place between January 5 and 19, 2009. • Prior to the visit, review the package of pre-assessment materials that will arrive at your school immediately after winter break.

46. Pre-Assessment Packet • NAEP Student Information Report: • List of students and their demographic data. • Use a No. 2 pencil to make any necessary corrections.

47. Pre-Assessment Packet • Department of Agriculture Letter – Free/Reduced-Price Lunch • NAEP Storage Envelope–used to store student lists. Student names cannot leave the building.

48. SD and ELL Questionnaires • SD and ELL students are identified on the Student Information Report • SD Questionnaires are used to determine if an SD student can participate and what accommodations he/she will need. • ELL Questionnaires are used to determine if an ELL student can participate and what accommodations he/she will need. • For clarification on NAEP allowed accommodations, refer to: • The 2009 National Assessment of Educational Progress in Florida: Participation of Students with Disabilities and English Language Learners • Comparing NAEP and FCAT Allowable Accommodations for SD and ELL Students

49. SD and ELL Questionnaires • The school NAEP coordinator will distribute the SD and ELL questionnaires to the teacher most knowledgeable about each student, setting a due date for their return. • On the Roster of Questionnaires, record the name of the teacher responsible for each student’s questionnaire and complete the “Returned” column. • Teachers are to return the questionnaires (either the completed hardcopy or the blank form if done online) to the school NAEP coordinator by the established deadline. Questionnaires can be found online at www.naepq.com after January 2, 2009.