Considerations While Auditing Virtual Education

Considerations WhileAuditing Virtual Education Nancy Reeder, Audit Manager NSAA IT Conference – September 2012

Why Audit Virtual Education? • Virtual education (per Wikipedia) • Online education using the Internet • Often referred to as cyber school • Courses can be real time (i.e., synchronous) or self‑paced (i.e., asynchronous) • Courses can be accessed via a Web-based delivery system (i.e., online) or a combination of online and “brick and mortar” (i.e., blended) • Virtual education is rapidly growing across the country Auditor General, State of Florida

The State of the States • All 50 States have implemented some level of virtual education(per Keeping Pace with K-12 Online Learning – 2011) • 13 States have full-time virtual education available to all K-12 students • Several States now require a virtual learning credit or online course completion for high school graduation • Funding methods and levels vary • Average daily attendance • Count days • Size-based • Successful completion Auditor General, State of Florida

Educational Structure in Florida • State Board of Education • Commissioner of Education • Department of Education (DOE) • 67 School Districts • District School Boards • District School Superintendents • School Principals Auditor General, State of Florida

Educational Choice • Florida Constitution guarantees the opportunity for a high quality education to all K-12 public school students • Traditional options: • Public schools (includes charter schools) • Private schools • Home education • Florida enters the digital age Auditor General, State of Florida

Virtual Education at the State Level • Florida Virtual School (FLVS) - Section 1002.387, FS • Began in 1997 as a grant-based pilot project • Legislature removed pilot status and established it as Florida On-Line High School in 2000 • Effective in 2003, it became known as the Florida Virtual School and was funded based on the concept of successful completion • Full-time and part-time programs • Largest State virtual school in the nation • Offers virtual education to other states and countries Auditor General, State of Florida

Initial Considerations • Virtual education funding at the State level * figure not yet finalized Auditor General, State of Florida

Initial Considerations (Cont.) • Funding method - successful completion • Does an incentive exist to pass students in order to receive funding? • What documentation is available to support: • Eligibility of students? • Existence of students? • Successful completion of courses? • Are teachers properly certified? Auditor General, State of Florida

Audit Approach – State Level • Florida Virtual School - Report No. 2012-020 • Attestation examination • For fiscal year ended June 30, 2010 • First audit (examination) of FLVS by the State • Looked for compliance with State requirements governing the determination and reporting of full‑time equivalent (FTE) students for funding Auditor General, State of Florida

What We Found • Missing documentation to support successful completion of courses • Certain required transcript information not recorded on transcripts • Compulsory attendance not verified according to required procedures • Teachers not properly certified and not approved to teach out of field Auditor General, State of Florida

What We Found (Cont.) • Florida residency (Cont.) • The State was other than “FL” for 375 students • Addresses for 159 students were valid but contained incomplete or inaccurate data  • Two students were children of military personnel  • Documentation provided for 183 students did not confirm Florida residency at the time of successful completion  • No documentation provided for 31 students  Auditor General, State of Florida

What We Found (Cont.) • Florida residency (Cont.) • In response to our preliminary and tentative findings, the Florida Virtual School provided additional documentation • Documentation provided for 36 of the previous 183 students supported eligibility for funding • Documentation provided for 1 of the 31 students for whom no documentation had previously been provided supported eligibility for funding • We revised our proposed adjustments accordingly Auditor General, State of Florida

What We Found (Cont.) • Policies and procedures • Although written policies and procedures had been established for various functions, verification of Florida residency could be improved by: • Implementing memoranda of understanding or similar documents with Florida school districts • Defining documentation to be retained to adequately support Florida residency of students • Confirming Florida residency of students on a periodic basis Auditor General, State of Florida

FLVS - Summary • Learning process for FLVS • FLVS’ response to the findings indicated subsequent implementation of procedures and controls • Estimated gross dollar effect of our proposed adjustments was a negative $190,484 Auditor General, State of Florida

The Next Chapter • Language in the Florida Statutes continues to be tweaked regarding virtual education • Chapter 2011-52, Laws of Florida, revised the audit responsibilities of the Auditor General to include virtual education providers Auditor General, State of Florida

Virtual Education at the District Level • Virtual Instruction Programs (VIPs) • Section 1002.45, FS, includes: • Definition of and requirements for VIPs • Provisions related to • provider qualifications • contract requirements • student eligibility • student participation requirements • assessment and accountability • exceptions • marketing • rules Auditor General, State of Florida

Virtual Education at the District Level (Cont.) • History • Beginning with the 2009-10 school year, each school district had to provide a VIP option to eligible students • VIP language in the Florida Statutes has been revised in each subsequent session (e.g., student eligibility requirements, increasing the required number of VIP options, etc.) • As of the 2012-13 school year, each school district is required to provide options for both full-time and part‑time virtual education for K-12 students Auditor General, State of Florida

Virtual Education at the District Level (Cont.) • District Options • Contract with FLVS • Establish a franchise of FLVS • Contract with a DOE-approved provider • Enter into a contract with another school district or regional consortium • Develop a district-operated program • Enter into an agreement with a virtual charter school Auditor General, State of Florida

Audit Approach – District Level • An operational audit of school district VIPs is in progress • Staff from each division of our Office are assigned • Cannot comment on findings at this time • Can share items for you to consider should you plan to audit virtual education Auditor General, State of Florida

Considerations While Auditing Virtual Education • Type of audit (operational versus performance) • Auditing virtual education at the school district level can be a massive undertaking • Responsibilities at the State level (DOE) • VIPs at the school district level • Virtual education providers • Whom to audit and what is your audit period • Coordinating work for no overlaps or gaps • Statutory changes • Availability of documentation and data files Auditor General, State of Florida

Considerations While Auditing Virtual Education (Cont.) • DOE-Related Issues: • DOE compliance with relevant governing laws, rules, and policies and procedures related to the approval and denial of private providers of VIPs • The adequacy of DOE policies and procedures over the approval and denial of private providers • The effectiveness of internal controls in ensuring management’s accomplishment of control objectives • Whether DOE provided school districts with an annual list of approved providers • The effectiveness and efficiency of DOE’s current provider renewal process Auditor General, State of Florida

Considerations While Auditing Virtual Education (Cont.) • DOE-Related Issues: (Cont.) • The appropriateness and effectiveness of DOE’s oversight and guidance provided to school districts • Whether DOE has established methods for VIP data tracking, transparency, and accountability measures to ensure that online and blended learning provides opportunities and positive outcomes • Whether DOE has established a means to report on the performance of private providers and make it available to all stakeholders • Whether DOE has made efforts to create end-of-course exams to provide course-specific outcome data until national assessments are available • Whether DOE has established internal controls that address the prevention and detection of fraud Auditor General, State of Florida

Considerations While Auditing Virtual Education (Cont.) • School District-Related Issues: • School district compliance with relevant governing laws, rules, and policies and procedures related to the VIPs • The adequacy of school district VIP policies and procedures • The effectiveness of internal controls in ensuring management’s accomplishment of control objectives • The appropriateness of provider contract provisions to promote effective VIP processes • Whether students reported for funding successfully completed their courses • Whether teachers were properly certified Auditor General, State of Florida

Considerations While Auditing Virtual Education (Cont.) • Provider-Related Issues: • The effectiveness of selected IT controls in achieving management’s control objectives in the categories of compliance with law, rules, and other guidelines and the confidentiality, integrity, availability, relevance, and reliability of provider data • The adequacy of selected logical controls implemented to protect school district data • The adequacy of backup, disaster recovery, and performance monitoring controls to ensure continuity of operations Auditor General, State of Florida

Considerations While Auditing Virtual Education (Cont.) • Provider-Related Issues: (Cont.) • Whether teachers and other staff have undergone appropriate background checks before being allowed contact with students or access to student data • Other provider-related issues included obtaining and analyzing provider data for selected performance metrics Auditor General, State of Florida

Questions? Auditor General, State of Florida

Contact Information Nancy Reeder nancyreeder@aud.state.fl.us 850.487.9294 Auditor General, State of Florida

Considerations While Auditing Virtual Education