EPA and Emerging Government-Wide Quality Issues ASP 2010 Workshop

1. EPA and Emerging Government-Wide Quality IssuesASP 2010 Workshop Reggie Cheatham Director, Quality Staff, OEI September 21, 2010 Seattle, WA For Conference Purposes Only

2. Topics to Cover • Why Quality and Why Now? • What is Government Doing? • Where is Government Going? 2

3. Why Quality and Why Now? Past Future Present • Laboratory data • Data collection • Sampling • Measurement • Increasing IT investment • Increasing demand for • information • Expecting “Good Quality” • Integration • Interoperability • Usefulness • Credibility 3

4. Why Quality and Why Now? • The figure is adapted from ISO 9001 with the permission of the American National Standards Institute (ANSI) on behalf of the International Organization for Standardization (ISO). Copies of all ISO standards can be purchased from ANSI, 25 West 43rd St. New York, NY 10036, 212-642-4900, Establish standards and guidelines to address quality during the entire lifecycle of all Agency products and service 4

5. What is EPA Doing? • Quality Policy (CIO 2106) and Procedure (CIO 2106-P-01) was effective as of February 1, 2010 • EPA Standard 2106-S-01, Quality Standard for Environmental Data Collection, Production, and Use, is under development by Working Group that includes QA managers from EPA Program Offices and Regions: • Aligns with new Quality Policy and replaces Policy 2105 (formerly EPA Order 5360.1) • Retains QA/QC for traditional environmental data activities (including laboratory work) 5

6. What is EPA Doing? • The Quality Standard provides new, consistent protocols and greater clarity for addressing: • Data obtained from other sources (i.e., “secondary data”), • Methods development, • Development and use of electronic models, • More relevant Quality Management Plans (QMPs), and • Consistency in QA Project Plans (including Uniform Federal Policy QAPP) • Simplifies reporting requirements and is expected to reduce these burdens • Standard 2106-S-01 will be supported by new, simplified requirements for QA/QC in external agreements and by revised guidance on the implementation of QMPs and QAPPs 6

7. What is EPA Doing? • Participation in TNI/NELAP • EPA Drinking Water Program Accepts NELAP Accreditation • EPA Regional Labs NELAP Accredited (9 out of 10) • EPA Regions Provide Staff for the Evaluation of the NELAP Accrediting Bodies • Brenda Bettencourt (EPA Region 9) is a Member of the TNI Board of Directors • Kevin Kubik (EPA Region 2) is a Member of the NELAP Board of Directors • Many EPA Staff are TNI Members and participate on TNI Committees 7

8. What is Government Doing? • In February 2009, the President signed the American Reinvestment and Recovery Act (ARRA) • ARRA required the establishment of a public Web site where citizens could acquire information about use of funds The Board shall establish and maintain...a user-friendly, public-facing website to foster greater accountability and transparency in the use of covered funds. The website...shall be a portal or gateway to key information relating to the Act and provide connections to other government websites with related information." -- American Recovery and Reinvestment Act of 2009 • New levels of transparency, oversight, and accountability were established 8

9. What is Government Doing? Recovery.gov: Created to accomplish the ARRA’s mission and to provide information for the public to monitor progress of the stimulus package • Recovery.gov features: • Weekly updates of each agencies funding notifications and activity reports • Map presenting state-by-state funding • Graphs charting the weekly progress of agencies’ available and paid-out funds • Map showing the recipients of funds and the resulting projects • Information on Federal contracts, grants & loans • A map illustrating the estimates of jobs to be created or saved and links to job information sites • Tools for the public to report for waste, fraud and abuse of recovery funds • Oversight audits by the Inspectors General of each Agency • Web site will continue to evolve to reflect additional data as it is collected 9

10. What is EPA Doing?Management Action Plan: Quality Assuring EPA’s ARRA Grants, Contracts and Interagency Agreements (IAs) (MAP I) • MAP I • Reiterates EPA’s existing Quality Policy for QA of environmental data operations performed under grants, contracts and IAs • Explains the Quality Policy’s role under ARRA • Defines roles and responsibilities in the federal and regional programs for execution of QA policy requirements under ARRA • Includes an extensive appendix which specifies the ARRA grants, contracts and IAs that have environmental data operation components with requirements for Quality Management Plans and/or Quality Assurance Project Plans 10

11. What is EPA Doing?Management Action Plan II: Quality Assurance EPA’s Data for Performance Measures (MAP II) • MAP II • Documents a baseline of expected quality practices for ARRA performance data and information • Establishes requirements for a Data Quality Record for reported information • The Data Quality Record includes documentation about: • Data Definitions and Source Reporting • Database controls • Data Transparency in reports • Review and Certification • Describes an Error Correction Process for ARRA data 11

12. Where is Government Going?Data.gov • Created as a priority Open Government Initiative to increase the ability of the public to easily find, download, and use datasets that are generated and held by the Federal Government • Provides descriptions of the Federal datasets (metadata), information about how to access the datasets, and tools that leverage government datasets • Federal agencies make decision on which datasets to post • Datasets continue to reside on each agency’s Web site • Data Policy is posted (http://www.data.gov/datapolicy) 12

13. Where is Government Going?Data Policy • Posted datasets only contain public information • All information accessed through Data.gov is in compliance with • Federal standards for confidentiality, integrity, and availability controls. • Federal privacy requirements including OMB guidance • Agency's Information Quality Guidelines • Federal record retention requirements Secondary Use • Data accessed through Data.gov do not have controls over its end use. The US Government does not vouch for any analyses conducted with data retrieved from Data.gov • Agency's preferred citation for each dataset is included in its metadata 13

14. Where is Government Going?Government Forensic Labs • White House Web sites: • http://www.whitehouse.gov/administration/eop/ostp/nstc/committees/cos • http://www.whitehouse.gov/administration/eop/ostp/nstc • National Research Council of the National Academies of Science Report Recommendation 7 • Laboratory accreditation and individual certification of forensic science professionals should be mandatory. • No person (public or private) should be allowed to practice in a forensic science discipline or testify as a forensic science professional without certification. 14

15. Where is Government Going?Importance of Policy in Information Quality • Documents approaches and expectations to promote consistency in results • Organizes disparate Standard Operating Procedures and Quality Controls • Identifies gaps and inconsistencies • Establishes roles & responsibilities that support the business • Eliminates or reduces confusion 15

16. Where is Government Going?Milestones • March 1: OMB issued a template for the MAX system to describe the current quality of Federal spending information in USASpending .gov • April 14: Agency’s draft framework for quality document due • April 30: Comments back to Agency from OMB due • May 14: Final document due with assurance signed by Senior Accountability Official 16

17. Where is Government Going?April 14th Plan • The data quality plan focuses on. • Implementation of the Data Quality Framework • Five subsections: governance structure; risk assessment; governing principles & control activities; communications; and monitoring • USASpending.gov Data • Describes the data quality plan and control processes which will be applied specifically to Federal spending information submitted for USASpending.gov • All data will be posted to USASpending.gov 17

18. Where is Government Going? • On June 25, OMB released two new guidance documents in response to the Open Government Directive.  • The first document provides Guidance for Online Use of Web Measurement and Customization Technologies: • http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-22.pdf • The second provides Guidance for Agency Use of Third-Party Websites and Applications: • http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-23.pdf 18

19. EPA Information Quality GuidelinesUnderpins all Open Government Initiatives Under the Federal-wide Information Quality Act, the EPA IQG provide policy and procedural guidance for how EPA ensures and maximizes the quality of its disseminated information. • The EPA IQG • Adopt a basic standard of quality (including objectivity, utility and integrity) • Develop a process for reviewing the quality (including objectivity, utility and integrity) of information before it is disseminated. • Establish administrative mechanisms allowing affected persons to seek and obtain, where appropriate timely correction of information disseminated by the agency. • Apply to information EPA disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information. 19

20. OBJECTIVITY – The disseminated information is accurate, reliable and unbiased and it is presented in a clear, complete and unbiased manner. UTILITY – The disseminated information is usefulness to the intended users. INTEGRITY – The disseminated information is protected from unauthorized access or revision, to ensure it is not compromised through corruption or falsification. REPRODUCIBILITY - The disseminated information is capable of being replicated, subject to an acceptable degree of imprecision. TRANSPARENCY – Sufficient metadata is provided to allow users to evaluate the quality of the disseminated information. Basic Standard Of QualityIQG Terms and Definitions 20 EPA IQG: Section 5.1

21. EPA RFCs (as of 8/20/10) Source: http://epa.gov/quality/informationguidelines/iqg-list.html 21

22. Requests by Category 22

23. Final Thoughts • The open government initiative, when combined with access to federal spending of taxpayers’ money and the resulting programmatic performance and programmatic data measures, will greatly enhance timeliness in the successful implementation of sound quality assurance principles. 23

24. Questions? Reggie Cheatham OEI/Quality Staff Director cheatham.reggie@epa.gov 202-564-6830 24