Testing and Accountability in North Carolina Charter Schools C urtis Sonneman Accountability Services, NCDPI December

1. Testing and Accountability in North Carolina Charter Schools Curtis Sonneman Accountability Services, NCDPI December 16, 2013

2. Overview • North Carolina Testing Program • Test Program Overview • Test Coordinator Responsibilities • Uses of Testing Program Data • READY Accountability Model • Federal Annual Measurable Objectives (AMO) and State Targets • School Performance Grades

3. What is North Carolina’s Testing Program?

4. Classroom Assessments: Formative and Diagnostic Information Balanced Assessment System Summative Assessment (Classroom, Statewide) Benchmark Assessment (Classroom, School, District) Formative Assessment (Classroom)

5. School Accountability AssessmentsState  LEA  School Measures of Student Learningto be included in Student Growth component in teacher evaluation Teacher  School Diagnostic and Formative AssessmentsStudent  Teacher 12 State & Local Instructional Improvement ToolsFormative & Benchmark Assessments and Data Analysis for Instruction ACT WorkKeys High School NC Final Exams in core academic non-tested subjects The ACT The ACT 11 3 End of Course (EOC)Biology, Math I and Eng II ACT Plan 10 9 8 ACT Explore End of Grade (Math, English Language Arts, Science) Elementary and Middle School NC Final Exams in currently non-tested subjects 7 EOG (Math, ELA) 6 EOG (Math, ELA) 5 EOG (Math, ELA, Science) 4 EOG (Math, ELA) K-5 Diagnostics 3 EOG (Math, ELA) Lower Elementary Measures of Student Learning K - 2

6. Current Testing ProgramAssessments Administered in North Carolina • End of Grade (Grades 3-8) • ELA/Reading • Mathematics • Science (Grades 5 & 8) • Alternate Assessments • End of Course • Math I • English II • Biology • Alternate Assessments • Field Tests • LEP/ESL Assessments • W-APT • ACCESS for ELLs • ACT Suite of Assessments • Dependent Upon State Funding • ACT Explore - Grade 8 • ACT Plan - Grade 10 • The ACT - Grade 11 • ACT WorkKeys - Grade 12 (CTE Concentrators) • NAEP Testing

7. State Assessments 2013-14 • Aligned to the new standards in 2012-13 • Inclusion of item types other than Multiple Choice • Math/Math I—Gridded Response (All) • Science/Biology and English II—Technology Enhanced (Online Only) • English II—Constructed Response (All) • Moving towards full online administration in 2014-15

8. How do we deliver the assessments?

9. Assessment Delivery Format • Online Administration • All End of Course Assessments (Math I, English II, & Biology) • End of Grade 5 & 8 Science • NCEXTEND2 Alternate Assessments (Final Year 2013-14) • NCEXTEND2 ELA/Reading and Math 3-8 • NCEXTEND2 Science 5 & 8 • NCEXTEND2 Math I, English II & Biology • Paper/Pencil Administration • ELA/Reading and Math 3-8 • All online assessments will be available in this format • NCDPI recommends online administration, but paper/pencil will be available

10. Online Assessment Delivery Plan • 2013-14 Online Assessment System • NCTest will continue to be the online assessment delivery system • Requires a locked down browser or app • 2014-15 Online Assessment System (Transitional Year) • Home Base will be new system for delivering summative assessments online (this will include delivery of Smarter Balanced Assessments) • Test Nav • NCTest may continue to provide online assessment delivery as transition to Test Nav moves forward

11. Some Additional Tests • CTE Post-Assessments • Used for CTE certifications • Used for Educator Effectiveness • NC Final Exams (aka Common Exams 12-13) • Used for Educator Effectiveness

12. Testing Window-GA Legislation • 5-day window for semester-long courses • Includes EOC, NC Final Exams, CTE post assessments, teacher made finals. • 10-day window for year-long courses • Includes EOG, EOC, NC Final Exams, CTE post assessments, teacher made finals.

13. What about NC Final Exams?

14. NC Final Exams Educator EffectivenessTests to measure student growth as a part of educator evaluation • When: • End of Year or Semester; Same window as EOG/EOCUsed For: • Providing an EVAAS score • Only for those teachers whose subjects are currently non-tested • For professional growth, and as a required component of employment decisions* • Logistics: • Once a year • 1 or 2 class periods • Replaces teacher- made final exam Library of NC Final Exams * Note: Not part of School Accountability Model (“A-F”)

15. NC Final Exams • Charter Requirement • Must participate in the NC Final Exams administration if-- • Received Race To the Top funds • Have beginning teachers who are seeking continuing licensure (Year 1) • Must have 3 years of growth data • Charters may elect to participate in NC Final Exams

16. Test Coordinator Responsibilities

17. Testing and Accountability Responsibilities • Principals/Directors/Board Members are ultimately responsible for Testing and Accountability Duties • North Carolina Testing Code of Ethics • Responsible for Test Security • Materials must be stored in a secure locked facility (plan your facilities accordingly) • Accessible by 1-2 staff (Test Coordinator and Principal) • Room, closet, or locked cabinet that cannot be removed

18. School Test Coordinators • Each charter school must have an assigned Testing Coordinator • Must be employee of charter school • Recommend a certified staff , but not regular classroom teacher if at all possible (not enough time to do both) • Must be available to attend all trainings (off-site) • Have no other responsibilities during test administration days • Available for training and updates during summer months (accountability reporting ends June 30)

19. Test Coordinator Responsibilities • Ensure test security with leadership • Test materials • Testing violations • Distribution of test materials • Ensure test materials are on copied or reproduced in any way • Order/Request test materials • Attend and conduct training • NC Train the Trainer Model • NCDPI Regional Staff Train School Test Coordinators • School Test Coordinators Train School Staff

20. Test Coordinator Responsibilities • Create school testing plan • Schedule test administrations • Develop local policies and procedures to ensure proper test administration • Submit materials for scanning/scoring • With assistance from RAC determine testing irregularities • Check online systems for updates

21. Test Coordinator Responsibilities • Maintain confidentiality of student test scores and teacher information • Will have access to all test results and accountability for the school • Submit data requested by RAC or NCDPI • Ensure data accuracy

22. School Test Coordinator Skills • Strong Computer Skills • Microsoft Excel • Microsoft Word • Internet savvy • Able to learn new programs easily • Secure File Transfer System • NCEducation online system • HomeBase • Able to send/receive information via email/phone and respond in a timely manner • Strong organizational skills

23. School Test Coordinator Skills • Available to attend regularly scheduled meetings (generally 1 time each month) • Strong interpersonal relationship skills • Training staff • Work closely with school leadership and NCDPI staff • Work closely with other staff to obtain data • Meet/talk with parents/staff to answer questions • Good listening skills • Not afraid to ask questions or address issues

24. Testing Questions

25. How are schools held accountable?

26. Accountability • Accountability of Test Scores • All public schools must participate in the North Carolina Testing Program • Schools are expected to test all eligible students • 95% Tested Rule • Failure to meet results in non-compliance notification • Additional requirements for not testing 95% of students for federal reporting

27. Accountability • Accountability of Test Scores (cont.) • Data Collections • Economically Disadvantaged Students (EDS) • Whether or not you have a school lunch program • Power School • Limited English Proficient—Consolidated Federal Data Collection • CECAS • Test Administration Data • Data Validation

28. Accountability • Accountability Model • READY Accountability Model 2012-2013 • Yearly Performance—Status • Performance over time—Progress (targets) • Student Growth—EVAAS Growth Model

29. Elementary and Middle School Accountability ProgressIndicators“trend over time for groups of students” Growth Indicators“combined individual student’s growth” StatusIndicators “this year” End of Grade 3-8 Math EVAAS Growth School-wide Δ End of Grade 3-8 Math End of Grade 3-8 ELA Δ End of Grade 3-8 ELA End of Grade 5 & 8 Science ΔEnd of Grade 5 & 8 Science

30. High School Accountability ProgressIndicators“trend over time for groups of students” Growth Indicators“combined individual student’s growth” StatusIndicators “this year” Δ End of Course End of Course Δ ACT ACT EVAAS Growth School-wide Δ Graduation Rates Graduation Rates Δ Math Course Rigor Math Course Rigor Δ ACT WorkKeys ACT WorkKeys Graduation Project

31. READY Accountability Model Reporting and School Performance Grades Indicators

32. 2012-2013 Ready Accountability Model Reporting • Accountability Indicators • Indicator will display data if school has 5 or more assessments/students • Federal Targets • English language arts/reading, math, graduation rate, and attendance • State Targets • Science, The ACT, and ACT WorkKeys • EVAAS Growth (School-wide Accountability Growth) • Reported as Exceeds, Meets, or Does Not Meet on website • Linked to EVAAS public reporting website • Graduation Project • Reported as Yes, No or N/A • Ability to drill down into data for analysis

33. How will the School Performance Grades be calculated?

34. School Performance Grades • First passed by the General Assembly in 2012 • Assigns a letter grade A-F for each public school in North Carolina • Includes consequences for “F” schools

35. School Performance Grades Legislation

36. School Performance Grades

37. School Performance Grades Break Out

38. School Achievement Scores • Components of the School Achievement Scores (80%) • Test Scores: Percent of students who score at or above Level 3 • End-of-Grade Tests • End-of-Course Tests • Graduation Rate: Percent of students who graduate in four years • Math Course Rigor: Percent of graduates who successfully complete Math III (Algebra II, Integrated Math III, Math III) • The ACT: Percent of 11th grade students who score 17 or above (the UNC System’s minimum composite score requirement) • ACT WorkKeys: Percent of CTE concentrator graduates who achieve a Silver Certificate or above

39. READY Accountability Model Reporting and School Performance Grades Indicators

40. School Growth • EVAAS School-wide Accountability Growth (20%) • Includes • End-of-Grade Tests • End-of-Course Tests • Reported for each school as • Exceeds Expected Growth • Meets Expected Growth • Does Not Meet Expected Growth • Not included in School Performance Grades for schools that have a School Achievement Score at or above 80 % and Meets Expected Growth

41. School Growth

42. Growth Conversion • Index range constrained at -10 to +10 • Index value converted to 50-100 point scale Did Not Meet Growth Met Growth Exceeded Growth -2 -8 -10 -6 -4 0 +2 +4 +6 +8 +10 70 50 60 80 85 90 100

43. Elementary Achievement Calculations + + + 3-8 Reading % Proficient 3-8 Math % Proficient 5&8 Science % Proficient Math I % Proficient Biology % Proficient English II % Proficient + + = Points # of Available Indicators + + + 73.1% (294/402) 83.6% (336/402) 60.1% (98/163) 93.8% (30/32) 0% (0/0) 0% (0/0) + + = 77.7 4 73.1 + 83.6 + 60.1 + 93.8 = 310.6 and 310.6 ÷ 4 = 77.65 ≈ 77.7

44. Elementary Grade Calculation Achievement Score under 80 OR Achievement Score at least 80 AND “Did Not Meet Growth” or “Exceeded Growth 90-100pts=A 80-89pts=B 70-79pts=C 60-69pts=D Under 60pts=F School Performance Grade Points ( ) ) ( Achievement Score = Growth Points + 0.8 0.2 X X ( ) ( ) + = 77.7 0.8 80.5 91.3 0.2 X X (62.2)+(18.3)= 80.5 80.5pts=letter grade of “B”

45. High School Achievement Calculation + + + + + + Grade Rate % 4yr Math I % Proficient English II % Proficient Biology % Proficient MCR % Math III The ACT % Met 17 ACT WorkKeys % Silver = Points # of Available Indicators + + + + 85.7% (90/105) 89.7% (78/87) 82.8% (77/93) 63.4% (52/82) 69.1% (56/81) 85.7% (42/49) 98.8% (80/81) + + = 82.2 7 85.7 + 89.7 + 82.8 + 63.4 + 69.1 + 85.7 + 98.8 = 575.2 and 575.2 ÷ 7 = 82.17 ≈ 82.2

46. High School Grade Calculation Achievement Score 80 or above AND “Met Growth” 90-100pts=A 80-89pts=B 70-79pts=C 60-69pts=D Under 60pts=F School Performance Grade Points Achievement Score = = 82.2 82.2 82.2pts=letter grade of “B”

47. High School Grade Calculation Achievement Score under 80 OR Achievement Score at least 80 AND “Did Not Meet Growth” or “Exceeded Growth 90-100pts=A 80-89pts=B 70-79pts=C 60-69pts=D Under 60pts=F School Performance Grade Points ( ) ) ( Achievement Score = Growth Points + 0.8 0.2 X X ( ) ( ) + = 82.2 0.8 78.9 65.6 0.2 X X (65.8)+(13.1)= 78.9 78.9pts=letter grade of “C”

48. Additional Reporting • Growth will be reported for each school: exceeded, met, or did not meet expected growth • A separate achievement score for math and reading will be reported for schools serving grades K-8 • The report card for schools serving 3rd graders will contain information on the number and percentage of third graders who are retained or promoted based on reading performance

49. What is new in testing in 2014-15 and beyond?

50. Testing • Read to Achieve (Beginning 2013-14) • End-of-grade test form administered at beginning of 3rd grade (BOG3) • End-of-grade retesting for end of 3rd grade ELA/Reading administrations or alternative assessment • ACT • HB 587: requires an alternative for ACT in Spring 2014 and for ACT Plan in Fall 2014