Quarterly Update

1. Quarterly Update Blue Ribbon Commission’s Recommendations of the 2009 Criterion Referenced Competency Test (CRCT) Erasure Investigation Dr. Beverly L. HallSuperintendentAtlanta Public Schools November 1, 2010

2. 2 Quarterly Update Agenda • Background • Overview • Update on BRC Recommendations • Next Steps

3. 3 Background • Blue Ribbon Commission’s report included 29 recommendations: • Security (including Caveon’s recommendations) • Process • Leadership • Comparison • Recommendation Owners • Each recommendation was assigned to a division and individual owner • Each owner is a member of the senior leadership team with functional responsibility that aligns with the recommendation

4. 4 Overview • Twenty-nine recommendations outlined in the BRC report • Each recommendation to date has a status of one of the following: • Complete • In Progress • On Hold

5. 5 Security Recommendations • Security recommendations encompass: • Developing a comprehensive test security plan that includes best-in-class test administration procedures • Continuing to research and understand the high number of erasures • Reinforcing and communicating existing testing policies and any new policies or changes

6. 6 Security Recommendations • BRC Security Recommendation 1 • Implement a well-defined program for follow-up and continued monitoring of CRCT Test Security. A commitment by APS to a thorough follow-up should be formal and public. These efforts should be implemented by the next testing period if possible and evaluated subsequently for effectiveness • Status: Complete • Administration’s Update • An updated program has been disseminated and will be enhanced annually with appropriate guidelines • Work is ongoing to develop a plan to conduct regular internal audits for each CRCT • The APS Test Security Manual will be posted on the district website

7. 7 Security Recommendations • BRC Security Recommendation 2 • Research and identify causes of the unusually high number of erasures within the district and identify and implement best practices in test preparation across the district • Status: On Hold • Administration’s Update • The Governor’s Office of Student Achievement has only partially fulfilled the district’s requests for 2009 CRCT erasure data • Limited CRCT test data for 2009 has been analyzed to the extent possible • The district will continue to implement best practices in test preparation and administration

8. 8 Security Recommendations • BRC Security Recommendation 3 • Reinforce APS' testing policies and procedures, and ensure that all appropriate training takes place for employees involved in the administration of the CRCT • Status: Complete • Administration’s Update • Standardized school-level pre-administration protocols and forms have been updated • A year-long training schedule has been enhanced for all main test administrations • Security measures and annual enhancements are posted to the district website annually • All APS staff involved with testing have more clearly defined security duties and responsibilities

9. 9 Security Recommendations • BRC Security Recommendation 4 • Communicate a policy that emphasizes ethics and accountability for adherence to the policies and training requirements -- as well as the ramifications for violating such policies and requirements • Status: Complete • Administration’s Update • Communication regarding policy training, inclusive of ethics and accountability, has been been provided to employees district-wide • Testing workshops reinforce and review ethics policies • Employees are required to annually acknowledge receipt and understanding of district ethics policies

10. 10 Security Recommendations • BRC Security Recommendation 5 • Formally document any policy or schedule changes related to the CRCT test materials (i.e., changes in return time or dates of the delivery of the CRCTs to/from the Brewer Test Center) • Status: Complete • Administration’s Update • A plan has been implemented to formally document changes in test distribution and collection schedules

11. 11 Security Recommendations • BRC Security Recommendation 6 • Establish a test security plan and ongoing budget for test security • Status: Complete • Administration’s Update • The APS Test Security Manual (comprehensive test security plan) is completed • Test security budget is established and will be re-evaluated annually • Completed a facilities assessment to determine opportunities for improved security

12. 12 Security Recommendations • BRC Security Recommendation 7 • Implement annual compliance and ethics certifications and approvals (for all employees involved in testing – including central office personnel, teachers, test coordinators, assistant principals, principals, and School Reform Team executive directors) • Status: Complete • Administration’s Update • Ongoing testing workshops reinforce and review ethics policies • Employees are required to acknowledge receipt of training before they are involved in any testing

13. 13 Security Recommendations • BRC Security Recommendation 8 • Establish ongoing external monitoring and reporting for standardized testing at all APS schools • Status: Complete • Administration’s Update • The district trains all APS testing monitors • External monitors will be requested from the Georgia Department of Education annually

14. 14 Process Recommendations • The process recommendations involve: • Following up to ensure immediate assistance is given to students impacted by testing irregularities • Developing a plan to track implementation of all of the recommendations • Following up on the results of the investigation • Enhancing existing protocols for future CRCT administrations and new audit routines

15. 15 Process Recommendations • BRC Process Recommendation 1 • Commit publicly to aggressively follow-up on the results of the investigation by establishing a well-defined plan to consider the findings and recommendations in this report, conduct additional investigations where necessary, and take whatever administrative, legal, and professional action that APS considers appropriate • Status: In Progress • Administration’s Update • Presented a response and plan of action to the ABE and public on August 16, 2010 • Presenting first quarterly report today, November 1, 2010 • Full cooperation with investigations conducted by the Governor’s Office

16. 16 Process Recommendations • BRC Process Recommendation 2 • Develop a plan that includes a project leader, clear objectives, work plan, schedule and deliverables • Status: Complete • Administration’s Update • A project leader has been appointed • APS Project Management Methodology has been implemented

17. 17 Process Recommendations • BRC Process Recommendation 3 • Conduct further investigation of the referred individuals within the Identified Schools and take whatever administrative, legal or professional action APS considers appropriate • Status: On Hold • Administration’s Update • The PSC approved 27 of 108 referred individuals • APS is providing full cooperation with the special investigation

18. 18 Process Recommendations • BRC Process Recommendation 4 • Conduct an analysis of individual student performance of those students APS determines may have been impacted by testing irregularities to determine if additional academic assistance and support are needed. If support is warranted, immediate action should be taken to mobilize appropriate assistance for the students for as long as necessary • Status: Complete • Administration’s Update • All identified students received services in the 12-week Accelerated Academic Recovery Program • Regular communications with parents

19. 19 Process Recommendations • BRC Process Recommendation 5 • Develop protocols to pro-actively conduct erasure analysis after future CRCT administrations • Status: Complete • Administration’s Update • District has protocols to request CRCT data files and to conduct erasure analysis • APS will continue to research and implement best-in-class security and testing practices

20. 20 Process Recommendations • BRC Process Recommendation 6 • Develop regular internal audit routines for each administration of the CRCT focused on: test security protocols; test administration; and detailed APS erasure analysis • Status: In Progress • Administration’s Update • APS Internal Auditor is developing a plan to conduct regular audits for each CRCT test administration

21. 21 Process Recommendations • BRC Process Recommendation 7 • Document any testing irregularities identified and specific plans for follow-up and resolution, including referrals to the Georgia Professional Standards Commission and/or Fulton County District Attorney's Office, as appropriate • Status: In Progress • Administration’s Update • Cross-divisional meetings are taking place to document process flow for reporting irregularities

22. Leadership Recommendations • Leadership recommendations involve • Driving culture change • Re-assessing current incentive, performance and evaluation systems • Utilizing new forms of review and feedback • Publishing and addressing any additional findings related to the 2009 testing period • Reinforcing and communicating existing, supporting policies 22

23. 23 Leadership Recommendations • BRC Leadership Recommendation 1 • Drive culture change throughout the district to create an environment that effectively balances attention to measurable student outcomes with attention to positive, ethical behavior • Status: In Progress • Administration’s Update • Modified the APS Balanced Scorecard objective of building a culture of alignment and accountability to include ethics • Aligning with the Effective Teacher in Every Classroom (ETEC) and Race to the Top initiatives

24. 24 Leadership Recommendations • BRC Leadership Recommendation 2 • Proactively re-assess incentive systems and performance evaluation processes for all district employees with a focus on achieving this appropriate balance • Status: In Progress • Administration’s Update • Planning for this recommendation is in progress as part of the Effective Teacher in Every Classroom (ETEC) and Race to the Top initiatives

25. 25 Leadership Recommendations • BRC Leadership Recommendation 3 • Establish annual goals and performance plans that include a significant emphasis on leadership, integrity and ethics • Status: In Progress • Administration’s Update • Align annual performance goals and objectives to the APS Balanced Scorecard • Leadership emphasis on ensuring implementation of the test security plan with integrity and providing site-based ethics training

26. 26 Leadership Recommendations • BRC Leadership Recommendation 4 • Incorporate self-assessment into the annual plan evaluation processes for leadership (i.e. principals, assistant principals, curriculum leaders, School Reform Team executive directors and central office personnel) • Status: In Progress • Administration’s Update • The district is incorporating the GA DOE Leader Keys Self-Assessment tool into the annual plan evaluation process for leadership (school, SRT and CLL)

27. 27 Leadership Recommendations • BRC Leadership Recommendation 5 • Incorporate surveys of teachers and support staff into annual performance planning for principals to provide qualitative evidence on principal performance • Status: In Progress • Administration’s Update • The school climate survey instruments are currently being reviewed

28. 28 Leadership Recommendations • BRC Leadership Recommendation 6 • Provide observations of principals, which may include shadowing or other efforts, for assessments of leadership ability compared to the critical skills determined for the position. Incorporate data on observed behaviors into the annual assessment process for leaders to assess their competency relative to those skills deemed to be critical for each position • Status: In Progress • Administration’s Update • The 2010-2011 instructional site visit schedule has been finalized and visits are in progress • Category I Schools are the focus of these audits

29. 29 Leadership Recommendations • BRC Leadership Recommendation 7 • Provide opportunities for regular, anonymous peer and upward feedback (i.e. teacher-to-teacher, teacher-to-assistant/principal, teacher-to-test coordinator, principal-to-executive director) as well as processes for following through and managing this information • Status: In Progress • Administration’s Update • Standardizing school climate surveys • Administering systemic climate surveys in district schools • Implementing recommendations for improvement and leadership professional development

30. 30 Leadership Recommendations • BRC Leadership Recommendation 8 • Conduct principal peer reviews and incorporate this information into annual evaluation processes • Status: In Progress • Administration’s Update • Principals will visit each other’s schools and make observations (peer review process)

31. 31 Leadership Recommendations • BRC Leadership Recommendation 9 • Provide the ABE additional recommendations on plans to monitor the principals and other leaders of the schools being referred by the BRC • Status: In Progress • Administration’s Update • Reassigned 12 principals • Provided additional monitoring and instructional support to those schools

32. 32 Leadership Recommendations • BRC Leadership Recommendation 10 • Reinforce and publicize externally and internally APS' policies that protect APS employees who report misconduct or violations from reprisals, retaliation, and other hostile workplace actions • Status: Complete • Administration’s Update • Launched the new external hotline and updated information on district’s website • Policies specific to work place ethics have been shared through appropriate notice to all employees and enhanced with appropriate training

33. 33 Leadership Recommendations • BRC Leadership Recommendation 11 • Publish additional findings made with respect to events that may have occurred during the 2009 testing period • Status: In Progress • Administration’s Update • A communication plan has been developed and is being implemented • The district website is being utilized for regular updates

34. 34 Leadership Recommendations • BRC Leadership Recommendation 12 • Given that the findings relate to events that may have occurred during the 2009 CRCT and that personnel changes may have occurred since that time period, APS must provide appropriate support for the implementation of recommendations by the current leadership of schools • Status: In Progress • Administration’s Update • Leadership and staff are being provided support and training • Resources have been identified to implement the recommendations • An educational specialist is supporting and monitoring implementation of recommendations

35. 35 CRCT Comparison Recommendations • BRC CRCT Comparison recommendations encompass: • Additional recommendations focused on the findings related to the comparison of the 2009 and 2010 CRCT results

36. 36 CRCT Comparison Recommendations • BRC CRCT Comparison Recommendation 1 • Given the high number of declines in Math, the BRC recommends that APS conduct a broad assessment of Math outcomes to further determine the root causes of the declines. This assessment should include, but not be limited to, a review of the curriculum and schools/teachers where the declines were most dramatic, as well as an assessment of the effectiveness of current teaching methods • Status: In Progress • Administration’s Update • Analyses to determine probable root causes are in progress • Updated APS alignment of math curriculum to mirror GA DOE Scope and Sequence • Training of mathematics coaches in new alignment and areas of content weakness is in progress • 12 week Accelerated Academic Recovery Program (Mathematics Navigator) supports this recommendation

37. 37 CRCT Comparison Recommendations • BRC CRCT Comparison Recommendation 2 • There are 33 schools that demonstrated a significant decline in the 2010 CRCT test results. The BRC computed this number by looking at two factors - - the number of declines in grade/subject combinations, as well as 10% or greater increase in failure rates. The BRC recommends that the District conduct a further in-depth assessment in these schools • Status: In Progress • Administration’s Update • A list of variables required to begin research has been developed • Once the assessment is complete, the district will provide support, progress monitoring and targeted intervention strategies for the selected schools to improve instructional quality

38. 38 Next Steps • Remain committed to implementing 100% of recommendations • Continue collaborative effort across divisions and departments as appropriate • Ensure full implementation, continue to utilize: • APS Project Management Methodology • Balanced Scorecard (BSC) for reporting progress in the various departments/divisions • Weekly reports to senior staff • Quarterly reports to ABE

39. 39 Closing • Our highest responsibility is to provide the best • possible education for our students in every • classroom and in every school. We will continue • the implementation of the recommendations • from the BRC investigative report while staying • focused on our main purpose – greater student • achievement