Performance audits in Washington

1. 17 June 2009 Performance audits in Washington I-900 Produces ResultsNSAA Annual Conference

2. Today’s agenda • Performance audit overview • Washington state legislation and who is subject to performance audits • Initial challenges for new program • Current challenges for program • Observations to date • State partners with contractors • Ernst & Young project methodology • Performance audit criteria • Results to date — Ernst & Young audits with SAO • Results to date — all completed SAO performance audits • New challenges for state auditors • Questions and answers

3. Performance audit overview • Generally Accepted Auditing Standards do not require an auditor to evaluate the reasonableness of business operating decisions or matters of management and operational efficiency that directly impact operating costs. • The Yellow Book does require the auditor to consider such matters; considered during a performance audit. • Performance audits encompass a wide variety of objectives, including: • Objectives related to assessing program effectiveness and results. • Economy and efficiency. • Internal control. • Compliance with legal or other requirements. • And objectives related to providing prospective analyses, guidance, or summary information.

4. Initiative 900 Passed in 2005, specifies that performance audits be conducted in accordance with Yellow Book Standards. Requires every performance audit to look for opportunities in the following areas: • Cost savings • Services that can be reduced or eliminated • Programs or services that can be transferred to private sector • Best practices • Regulatory changes that allow entity to carry out its functions • Roles, functions and recommend changes, eliminations • Gaps, overlaps in programs or services • Feasibility of pooling technology systems • Departmental performance data, performance measures and self-assessment systems

5. Who is subject to SAO performance audits in Washington? • More than 163 state agencies • Nearly 3,000 local governments

6. SAO audit selection process • Outreach efforts shape scope, objectives for performance audits • Citizens: Hotline, outreach • Interested parties • I-900 directs SAO to pursue the largest, costliest government entities first • SAO’s own audit research (opportunity, probability, citizen support, audit-ability) • Areas and issues SAO identified through prior audit work • Readily visible in RFP

7. Initial challenges State agencies and local governments: • Were unfamiliar with performance audits and new SAO audit authority • Are more familiar with financial and compliance audits • Show a desire to negotiate audit criteria • Some believe statutes are the sole performance audit criteria • Want to limit criteria to those used within their government type • Want to set criteria at the middle of the bell curve • Are surprised by what can be said in a performance audit report • Are unclear about the role of their attorneys in relation to performance audits

8. Current challenges Program funding in the current economic environment: • Program funding from I-900 resulting in approximately $1 million revenue each month • Total funding for program approximately $40 million early this year • Legislation introduced to cut funding for performance audits leaving only approximately $12 million for the next two years • Negotiations resulted in agreement to restore funding to approximately $26 million over the next two years

9. Observations on I-900 performance audits to date • Citizens and officials appreciate narrower scopes and objectives. • On occasion, public policy and citizen expectations differ. • On occasion, one public policy competes with another (and becomes part of the report). • Examples include the following audits: • How effective are WSDOT investments at minimizing congestion? • How efficient and economical are WSF operations?

10. SAO partners with contractors (including Ernst & Young) • SAO conducts performance audits with staff & vendors • SAO uses vendors for more than 80% of its performance audits (including Ernst & Young) • Successful partnering model includes: • SAO and Ernst & Young communicate regularly • Ernst & Young provides opportunity for SAO to share input • SAO supports Ernst & Young interaction with the audited agency or local government • SAO supports/champions contractor’s independence • SAO and Ernst & Young support the use of risk-based audit approach in the context of a fixed price proposal • Ernst & Young develops audit methodology to address fixed price engagement considerations

11. Ernst & Young project methodology • Phase One — Identify: Conduct a broad view performance and risk assessment to identify improvement opportunities. • Phase Two — Diagnose: Diagnose and prioritize process, initiative and or control issues that have the greatest opportunity to reduce costs and improve efficiency. • Phase Three — Design: Develop and execute work plan to identify and quantify root causes of the highest risk areas. Develop the draft report of our findings to discuss with auditee management and obtain management’s input and feedback. Iterative process • Phase Four — Deliver: Issue the final audit report to the SAO and assist the SAO in making presentations to State Legislators or Legislative Committees.

12. Audit criteria sources • Criteria for financial or compliance audits: • Generally accepted accounting principles - GAAP • Statutes • Contractual requirements • Budget appropriations • Criteria for performance audit: almost unlimited!

13. Criteria defined Chapter 7.37 of Yellow Book reads: “Criteria represent the laws, regulations, contracts, grant agreements, standards, measures, expected performance, defined business practices, and benchmarks against which performance is compared or evaluated. Criteria provide a context for evaluating evidence and understanding the findings, conclusions, and recommendations included in the report. Auditors should use criteria that are relevant to the audit objectives and permit consistent assessment of subject matter.”

14. The Auditor’s search for criteria(creativity, pragmatism, remember the citizen) • With 7 billion people on the planet, auditors may never know with certainty they have identified the best practice. But if the criteria is valid and demonstrates higher performance is possible, you may want to run with it. • Consider citizen expectations in establishing criteria and measuring performance • How Ernst & Young identified criteria on its audits? • Efficiency and economy of administrative operations at the washington state department of transportation • Efficiency and economy of Washington state ferries operations • Impact fees • Hospitals

15. Possible criteria The following are examples of possible criteria: • Purpose or goals prescribed by law or regulation or set by officials of the audited entity • Policies and procedures established by officials of the audited entity • Technically developed standards or norms • Expert opinions • Prior periods’ performance • Defined business practices • Contract or grant terms • Performance in the private sector • Performance of other entities or sectors used as defined benchmarks • What is possible • Citizen expectations

16. Other criteria observations General observations about the dialogue between auditors and governments • Government sometimes prefer criteria that comes from within the same government type • Government sometimes prefers the of use standard practices as criteria (standard practices may not correspond with best practices) • Government sometimes seeks to negotiate criteria (SAO wants to know what criteria government believes is most appropriate but SAO may not always agree)

17. Support for the SAO performance audit program • Tremendous public support for SAO performance audits. • Tremendous support by the media. • Support clarified by pressure surrounding potential legislative funding cuts earlier this year.

18. Ernst & Young Results on SAO performance audits Completed Audits • Washington State Ferries (see supplemental slides) • Washington State Dept. of Transportation Administrative and Overhead Operations (see supplemental slides) • Use of Impact Fees at Five Cities Audits Underway • Administrative operations, capital projects and citizen accountability at three public hospitals • King County Utilities • Seattle City Light operations

19. Summary of results through August 2007 Completed SAO performance audits • SAO had completed 10 performance audits of state agencies and local governments. • Collectively, the audits made 454 recommendations to improve the efficiency and effectiveness of government. • Ernst & Young’s audit of WSDOT Admin and Overhead operations identified approximately 300 issues during Phase 1. These results should be considered as SAO continues performance audits at WSDOT. • Considering the $9 million cost of conducting the audits, the findings and recommendations represent a healthy return on investment. • Overall financial impacts on next slide

20. Resultssince2007 Financial Impacts • Improved collections: $160 million • Economic benefits: $3 billion • Other financial impacts: $280.6 million • Total cost savings and financial benefits: $3.6 billion

21. Results since 2007 Department of General Administration Motor Pool; February 2007 • Identified potential savings: $2.3 million over five years • Audit cost: $255,285 • Findings and recommendations: Found 116 underused vehicles that should be sold or reassigned. Recommended changes in the method of buying new vehicles to save interest on auto loans. Also recommended raising the Motor Pool’s rental rates that were too low to cover operating expenses.

22. Results since 2007 Department of Health Office of Health Professions Quality Assurance; August 2007 • Identified potential savings: The audit emphasized protecting patients and the public from abusive and predatory practitioners rather than identifying cost savings. • Audit cost: Slightly more than $1 million • Findings and recommendations: Found inconsistent practices and lax oversight among the 57 different health care professions, putting patients at risk. Made 67 recommendations to change how health professionals are credentialed and disciplined. The Department has put in place all the recommendations it is legally authorized to do using existing resources. Recommendations requiring additional resources or policy changes are pending before the Legislature.

23. Results since 2007 Department of Transportation - Washington State Ferries; September 2007 • Identified potential savings: $50.2 million over five years • Audit cost: $947,682 • Findings and recommendations: Found an inadequate number of manager positions to supervise staff and other weaknesses at the system’s main maintenance facility. Recommended standardizing business practices across the system and consolidating low-ridership ferry runs to achieve significant savings in fuel and operating costs.

24. Results since 2007 Educational Service Districts; September 2007 • Identified potential savings: $23.5 million over five years • Audit cost: $1.7 million • Findings and recommendations: Found most districts are operating efficiently and are providing needed services to individual school districts. Made 215 recommendations spread among the state’s nine ESDs. Recommended system wide strategic planning to better coordinate services and avoid duplication. Also recommended restructuring of staff in some districts to reduce administrative costs.

25. Results since 2007 Sound Transit’s Link Light Rail; October 2007 • Identified unnecessary spending: $5.1 million • Audit cost: $557,759 • Findings and recommendations: Found inefficient and ineffective construction management leading to cost overruns and project delays. Also found inadequate assessments of contaminated sites, resulting in unnecessary cleanup costs. Audit revealed that Sound Transit improved construction planning and management processes since its early years. Recommended further improvements and refinements to construction management.

26. Results since 2007 Department of Transportation Congestion Management; October 2007 • Identified potential benefit: $3 billion over five years in economic impact to citizens and businesses as well as environment impacts • Audit cost: $1.6 million • Findings and recommendations: Found the Department does not consider reducing congestion a priority in planning transportation improvements. Concluded that congestion could be reduced 20 percent by carrying out several near-term measures at little or no cost. Identified long-term investments to significantly reduce congestion.

27. Results since 2007 Department of Transportation Administration & Overhead; November 2007 • Identified potential savings: $18.1 million over five years • Audit cost: $672,833 • Findings and recommendations: Found inefficient, decentralized and inconsistent systems and practices in various administrative functions. Recommended centralizing functional areas as well as major upgrades in programs.

28. Results since 2007 Port of Seattle Construction Management; December 2007 • Identified unnecessary spending: $97.2 million • Audit cost: $758,940 • Findings and recommendations: Found serious and pervasive issues in Port management of $1.5 billion in construction projects. Found a lack of adequate oversight by the Port Commission and inadequate controls that put taxpayer dollars at risk and raised the probability that fraud existed. Made 51 recommendations calling for the Commission to reassert oversight responsibility. Also recommended hiring an executive to oversee all procurement activity and strengthening the Port’s internal audit function.

29. Results since 2007 Department of Transportation Highway Maintenance and Construction Management; January 2008 • Identified savings and unnecessary costs: $41.9 million • Audit cost: $438,130 • Findings and recommendations: Found that while the Department follows several industry best practices, it has opportunities to improve inventory and supply management, purchase of hot-mix asphalt project management practices. Made 34 recommendations calling for improved oversight of inventory management and tracking project costs.

30. Results since 2007 Open Public Records Practices at 30 Government Entities; May 2008 • Identified savings and unnecessary costs: Unquantifiable • Audit cost: $600,000 • Findings and recommendations: Our audit work revealed that, by and large, most of the 30 entities we audited are providing good customer service in responding to public records requests. We tested the entities’ performance by submitting 10 public records requests to each entity like a citizen would and identified some trouble spots in which entities need training on the Public Records Act; have problems tracking requests; or are unable to receive them due to e-mail filters or other issues with their mail systems.

31. Results since 2007 Collection of State Debt at Six State Agencies; August 2008 • Identified cost savings: One-time cost savings of $15.6 million to nearly $160 million (50 percent of $319.4 million in total uncollected debt) • Audit cost: $701,882 • Findings and recommendations: We identified 14 issues and 45 recommendations to help six state agencies increase collection of overdue money owed to the state. Most of our recommendations require little to no investment of state resources. • Two programs − one at the Department of Revenue and one at Employment Security Department − were commended for their performance in collecting state debt.

32. Results since 2007 Administrative and Support Services at the 10 largest K-12 School Districts; September 2008 • Identified savings and unnecessary costs: $54 million over five years in eight school districts • Audit cost: $1,390,674 • Findings and recommendations: We identified 13 issues and 49 recommendations to help the districts identify opportunities to achieve greater efficiency and economy to free up resources to meet other pressing needs.

33. Results since 2007 Use of Impact Fees in Five Cities; October 2008 • Identified financial impacts: $1.18 to $1.34 million • Audit cost: $726,466 • Findings and recommendations: We identified nine issues and 17 recommendations to help cities more efficiently and effectively calculate and use impact fees. We also made recommendations to the Legislature to clarify laws and ambiguous terminology within laws regarding impact fees.

34. Results since 2007 Travel Practices at 13 School Districts; November 2008 • Cost savings: $1,075,551 over five years • Audit cost: $777,564 • Findings and recommendations: We measured the economy of travel at 13 public school districts against best practices in travel and federal rate limits. We identified five issues and nine recommendations that can be used at all school districts to achieve greater economy for their travel.

35. Results since 2007 Washington State Parks and Recreation Commission; December 2008 • Cost savings: Unquantifiable • Audit cost: $1,053,351 • Findings and recommendations: Our audit focused on whether the Commission’s strategic plan was designed to help the Commission achieve its goals; whether the Commission has appropriately assessed and obtained the human, financial and information technology resources it needs; is monitoring and appropriately measuring its performance; and is managing its operations efficiently and economically. We made recommendations to the Commission that, if instituted, could dramatically improve the agency’s operational efficiency.

36. New challenges • Current economic environment is challenging for state audit offices • Oversight of stimulus money places additional burden on state auditors • In many cases, the additional tasks are coupled with funding cuts or freezes • Ironically, performance audits arguably provide the biggest bang for the buck but may be the initial target for funding cuts

37. Questions and answers

38. Ernst & Young Michael Kucha Partner michael.kucha@ey.com +1 206 654 7741 Terry Hanson Senior Manager terry.hanson@ey.com +1 651 210 6724 Washington State Auditor’s Office Chuck Pfeil Director of Audit pfeilc@sao.wa.gov +1 360 902 0366 Chris Cortines Senior Deputy Director of Performance Audit continec@sao.wa.gov +1 360 628 7752 Contact information