Using the 2009 NECAP Reports February 1-5, 2010

1. Using the 2009 NECAP Reports February 1-5, 2010

2. Welcome and Introductions Susan Smith MEA/NECAP Coordinator Maine Department of Education Tim Crockett Senior Vice President Measured Progress

3. Welcome and Introductions Department of Education Staff

4. Welcome and Introductions NECAP Service Center 1-877-632-7774 4

5. Purpose of the Workshop • Review the different types of NECAP reports • Review Maine NECAP baseline data • Conduct a demonstration of the Analysis & Reporting System

6. Involvement of Local Educators • Test Item Review Committees • Bias and Sensitivity Review Committees

7. NECAP Grade Level Expectations • NECAP test measures the NECAP Grade Level Expectations (GLEs) • GLEs were adopted by the legislature as Maine’s Federal, State, and Local Accountability Standards • GLEs became part of the Maine Learning Results: Parameters for Essential Instruction • GLEs are located at http://www.maine.gov/education/necap/standards.html • GLEs should be fully incorporated into each school’s curriculum to ensure that students have an opportunity to learn the content assessed on the NECAP 7

8. 2009-10 NECAP Scores are Baseline Data • Maine adopted NECAP content standards (GLEs), cut scores and achievement standards. • This means that 2009-10 NECAP results are Maine’s baseline data that begin a new trend line. • Although both the MEA and NECAP programs use an 80 point scale to report scores, they are different tests that measure different content standards and have different cut scores between achievement levels so the scores can NOT be compared. 8

9. Why MEA and NECAP Scores Can’t be Compared Mathematics 9

10. Why MEA and NECAP Scores Can’t be Compared Reading 10

11. Basics of NECAP Test Design • Fall test – previous year’s GLEs • Reading and Mathematics – grades 3-8 • Writing – grades 5 and 8 (2009 was a Pilot test) • Variety of Item Types • Reading: multiple choice and constructed response • Mathematics: multiple choice, short answer (one point), short answer (two points), and constructed response • Writing: multiple choice, constructed response, and extended response • Common and Matrix Items 11

12. Basics of NECAP Test Design • Reading – Grades 3-8 • 52 Common points • Mathematics – Grades 3 & 4 • 65 Common points • Mathematics – Grades 5-8 • 66 Common points • Writing – Grades 5 & 8 • 34 Common points • No results this year because of Pilot testing 12

13. Workshop Materials 13

14. Types of NECAP Reports Public Reports Results Report School and District level Summary Report School/District/State level Password Restricted Reports Released Items Summary DataSchool and District level Achievement Level SummarySchool and District level 14

15. Types of NECAP Reports Confidential Reports Student ReportInformation for Parents/Guardians Item Analysis ReportSchool level by student Longitudinal Data Student level, across test administrations Student Level Data FilesExcel/csv files by grade on district and school confidential site

16. The Family Educational Rights and Privacy Act (FERPA) • Access to individual student results is restricted to: • the student • the student’s parents/guardians • authorized school personnel • Superintendents and principals are responsible for maintaining the privacy and security of all student records. • Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties. • FERPA website: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html 16

17. Student Report 17

18. Looking at the Student Report This part of the report provides the individual student’s achievement level and scaled score 18

19. Looking at the Student Report 19

20. Looking at the Student Report This part of the report provides a comparison between the performance of this student and his/her school, district, and state 20

21. Looking at the Student Report 21

22. Looking at the Student Report This part of the report gives specific information about the student’s performance in content area subcategories 22

23. Looking at the Student Report 23

24. Teaching Year vs. Testing Year • Rationale • Inclusion of students in “Teaching Year” reports 24

25. Looking at a School-Level Results Report Schools can view reports for Testing Year (2009-10) 25

26. Looking at a School-Level Result Report Or for Teaching Year (2008-09) 26

27. District and School Results Report 27

28. School-Level Results Report – Grade Level Summary 28

29. School-Level Results Report – Grade Level Summary 29

30. School-Level Results Report – Grade Level Summary 30

31. School-Level Results Report – Grade Level Summary 31

32. School-Level Results Report – Content Area Results 32

33. Looking at the Results Report – Content Area Results Three years of data are typically shown on this report. As Maine schools participate in NECAP in coming years, more data will be added. After three years, a Cumulative Total will be shown. 33

34. Looking at the Results Report – Content Area Results 34

35. Looking at the Results Report – Content Area Results Mathematics Total Possible Points includes both common and matrix items (not field-test). Total Possible Points also represents the test’s balance of representation. 35

36. Looking at the Results Report – Content Area Results Please note: The Total Possible Points column is organized differently on the Reading Results Report 106 possible points are represented here – they are sorted by “Type of Text” The same 106 possible points are represented here – they are sorted by “Level of Comprehension” 36

37. Looking at the Results Report – Content Area Results Reading Total Possible Points includes both common and matrix items (not field-test).

38. Looking at the Results Report – Disaggregated Results 38

39. Looking at the Results Report – Disaggregated Results Important Note: Disaggregated results are not reported for sub-groups of less than 10 39

40. Looking at the Results Report – Disaggregated Results Because this is a small school, and so many of the sub-groups are smaller than 10, this part of the report is not as useful. Does this data match what we know about the district’s program? But we can still look at district and state disaggregated results. 40

41. Summary Report 41

42. NECAP District and School Student-Level Data Files • Contain: • All demographic information for each student that was registered through MEDMS/Infinite Campus • The scaled score, achievement level, and subcategory scores earned by each student in all content areas tested • NECAP files also contain: • Performance on released items • Student questionnaire responses 42

43. Item Analysis Report 43

44. Item Analysis Report 44

45. Looking at the Item Analysis Report 45

46. Looking at the Item Analysis Report This part of the report gives specific information about the released items and student performance on individual items 46

47. Looking at the Item Analysis Report 47

48. Looking at the Item Analysis Report This part of the report represents all of the items used to compute student scores - points are displayed by subcategory. Each student’s Scaled Score and Achievement Level are shown. 48

49. Supporting Materials and Resources www.maine.gov/education/necap • Guides to Using the 2009 NECAP Reports • NECAP Analysis and Reporting System User and Training Manual • Companion PowerPoint presentation • Grade Level Expectations • NECAP Accommodations Guide • Released Items documents • Preparing Students for NECAP: Tips for Teachers to Share with Students 49

50. How Do I Find Public Assessment Reports? Go to… www.maine.gov/education/ necap/results 50