Program Performance Accountability & Measurable Results

1. Program Performance Accountability& Measurable Results Kevin Keaney, Chief Pesticide Worker Safety Programs U. S. EPA 2005

2. Outline • Performance Measurement: Context, Value, Types • Accountability Drivers & Problems • GPRA & PART • Elements of a Good Measures Program

3. Where Accountability Fits Strategically Program Success is built from critical components. CLEAR GOALS Public Health & Ecological Protection Outcomes MEANINGFUL ACCOUNTABILITY Measures & Tracking GOOD PLANNING Strategic Objectives EFFECTIVE IMPLEMENTATION Multiple Levels ADEQUATE RESOURCES Acquisition & Allocation

4. Value of Performance/Results Data Meaningful Accountability = Good Management = Good Government MULTIPLE USES • Guiding Program Management: Decisions; Priorities; Resources • Performance Accountability • Tracking program success and progress • Communication • General Public • Interested Stakeholders • Decision Makers (Congress, Admin.)

5. Types of Performance Measures Performance measures vary depending on level of organization & need. Strategic Goals Broader/Long-Term Mid-Range Objectives Annual Commitments Individual Performance Stds Specific/Short-Term

6. Accountability Drivers / Mechanisms Statutes Budget Allocation 3 Year NPM Commitments Tracking Systems/Reporting Annual Work Plans Performance Standards Audits EPA 5 Year Strategic Plan Budget Formulation

7. Problems w/ Current Systems • Components:…are complex,…are created at different times,…by different people,…for different purposes,…little thought about connections or alignment among the parts. • Inevitably creates:…conflicts, …confusion, …unnecessary burdens, & …mixed messages. • Doesn’t work: …no accountability …no consequences.

8. Accountability Overload FLEXIBILITY! FOCUS! FIFRA! POLITICAL PRESSURE! SPECIAL PROJECTS! REGIONAL INITIATIVES! OMB! PART! BUDGET! STRATEGIC PLAN! GPRA! MEDIA COVERAGE! OECA! LAWSUITS!

9. Accountability Drivers/Mechanisms Statutes Budget Allocation 3 Year NPM Commitments Tracking Systems/Reporting Annual Work Plans Performance Standards Audits EPA 5 Year Strategic Plan Budget Formulation

10. Accountability Drivers/Mechanisms Statutes EPA 5 Year Strategic Plan 3 Year NPM Commitments Budget Formulation Annual Work Plans Performance Standards Budget Allocation Tracking Systems/Reporting Audits

11. GPRA GPRA = Government Performance and Results Act of 1993 • GPRA is a statute aimed at increasing accountability by requiring measurement of program results & linking those results to resource allocation. • Provides a statutory framework for strategic planning and reporting that requires federal agencies to: • Establish accountability measures to assess the outcomes of each program activity, and • Prepare performance plans that establish annual performance goals.

12. GPRA Auditing • Program Assessment Rating Tool (PART) • Office of Management & Budget assumed the role of evaluating progress against GPRA. • OMB is committed to audit all federally funded programs against GPRA requirements on a 5 year cycle. Size of individual programs receiving detailed audit varies. • This is the third year. Several Pesticide programs have been audited.

13. GPRA Audits OMB Fire is invented

14. GPRA Auditing • Program Assessment Rating Tool (PART) • A matrix of questions with a numeric value & requiring succinct answers. • Principle focus has been on the existence of Outcome Measures satisfying OMB. • OMB has not been bound by pre-existing measures (EPA’s Strategic Plan; NPM Guidance). • Moving target of changing requirements. • Short turn-around times. • Average initial scores of programs have been very low (40 - 60 out of 100) and many classified as “Results Not Demonstrated”. • “re-PARTing” is the norm. • Results are published each year with the federal budget. • Programs not demonstrating results for successive years are subject to budget cuts or elimination.

15. PART Budget Consequences OMB EPA

16. Criteria for Good Measures VALUE HIGH Outcomes Clear Wide use Meaningful Understandable Useful LOW Beans Complex Narrow use BURDEN LIGHT Few Simple Low User-friendly # of Measures Difficulty of Collection Data System Costs Expertise Required HEAVY Many Hard High Technical

17. Performance Measures Quadrant Analysis High Value 1. High Value/Light Burden 2. High Value/Heavy Burden OUTCOME NIRVANA Light Burden Heavy Burden OUTPUT HELL 3. Low Value/Light Burden 4. Low Value/Heavy Burden Low Value

18. Measures Comparison Matrix Measures Criteria Possible Measures

19. Process for Measures Improvement • Specific Problem Formulation • Baseline data collection: Current measures; current data systems, etc. • Scope of project? Strategic measures only or a universal results ethic (measures for all projects). • Participants? OPP; OECA; Regions; States/Tribes; industry; NGOs; etc. • Outputs? Measures for Strategic Plan/NPM/Annual Workplans/PART; Public Reports; Change risk assessment/management practices; Change or consolidate databases. • Timeline/Deadlines? 6 months (PART); 2 years (next 5 Year Strategic Plan).

20. Process Structure Program Specific Workgroups Gather Data; Coordinate;Develop Measures + Data Systems Regis. Regis. Review ESA C&T WP SAI Coordinate w/ Partners & Stakeholders Regions States, Tribes Industry OW,ORD, OECA USDA NGOs Senior Level Steering Committee Set Direction; Manage Process; Resolve conflicts / issues Databases & Information Systems Evaluate existing sources; Opportunities for cost savings

21. Take Away Message • All government funded activities will be held to higher standards of performance accountability to justify budgets. • Cooperative work is required to establish meaningful accountability measures and performance targets. • Well documented progress toward quantifiable public health and environmental protection goals will help ensure continued program operation and funding.

22. Program Performance Accountability& Measurable Results QUESTIONS ?