Preparing Quality Assurance Project Plans

1. Preparing Quality Assurance Project Plans Presented By: Denise L. Goddard, Chemist Quality Assurance Section Athens, Georgia

2. EPA DISCLAIMER • This Presentation is for Training Purposes Only.

3. EPA - QA Documents for Preparing Quality Assurance Project Plans • Guidance on Systematic Planning Using the Data Quality Objectives Process, EPA QA/G-4, EPA/240/B-06/001 (February 2006) • Requirements for Quality Assurance Project Plans, EPA QA/R-5, EPA/240/B-01/003 (March 2001) • Guidance for Quality Assurance Project Plans, EPA QA/G-5, EPA/240/R-02/009 (December 2002)

4. Are QAPPs Really Required??? YES!! • Quality System Requirements – “Approved Quality Assurance Project Plans (QAPPs) or equivalent documents defined by your organization’s QMP, for all applicable projects and/or studies that will involve environmental data collection or where environmental decisionswill be made for a particular site. QAPP must be approved prior to any data gathering work or activities, except under circumstances requiring immediate action (emergency response) to protect human health and the environment or operations conducted under police powers.”

5. Organizational Applicability?? • EPA Organizations – Covered under Executive Order 5360.1, A2: “The Agency-wide Quality System requirements defined by this Order apply to all EPA organizations, and components thereof, in which the environmental programs conducted involve the scope of activities described in Section 5.a above. The authority of this Order applies only to EPA organizations except as addressed by Section 5.d(2) below.”

6. External Organizations Requirements • Extramural Agreements: Agency-wide Quality System requirements may also apply to non-EPA organizations. These requirements are defined in the applicable regulations governing extramural agreements. Agency-wide Quality System requirements may also be invoked as part of negotiated agreements such as memoranda of understanding (MOUs). Non-EPA organizations that may be subject to quality system requirements include:

7. Extramural Agreements • (a) Any organization or individual under direct contract to EPA to furnish services or items or perform work (i.e. contractor) under the authority of 48 CFR 46, (including applicable work assignments, delivery orders, and task orders); • 40 CFR 31 – Grants & Cooperative Agreements with State & Local Governments • 40 CFR 35 – State & Local Assistance

8. The Purpose of a Quality Assurance Project Plan • As a planning document, the QAPP should contain a detailed description of environmental data collection activities and operations, the problems associated with a site, the sampling and analysis requirements, the decisions to be made, and the necessary QA/QC activities governing this effort.

9. Issues Addressed by a QAPP • The QAPP must provide sufficient detail such as: • The project’s technical and quality objectives – these must be well defined and agreed upon by all affected parties and stakeholders • The program-specific and site-specific requirements (stipulated in consent decrees, records of decision, regulations, statutes, etc.). • The intended measurements, data generation or data acquisition methodsthatare appropriate for achieving project goals/objectives.

10. Issues Addressed by a QAPP – Con’t • A summary of the assessment proceduresfor confirming that data of the type, quantity and quality required and expected were obtained, and • A description of the process for evaluating the limitations on the use of the information or data obtained that includes identifying, documenting and communicating the limitations to all affected parties and stakeholders.

11. Overview of Content Requirements • To be effective, the QAPP must clearly state: • The purposeof the environmental data operation (e.g., enforcement, research and development, rulemaking), • The type of work to be done (e.g., pollutant monitoring, site characterization, risk characterization, bench level proof of concept experiments), and • The intended use of the results (e.g., compliance determination, selection of remedial technology, site closure, development of environmental regulations).

12. Before We Start - Some Preliminaries – Format/Content Requirements • Because the QAPP is a formal document – it should contain: • A Title Page containing the title of the document, the Identification of the Organization that Prepared the QAPP, the Preparation Date and the Version Number – The document should bePaginated • An Approval Page – Containing Signature and Date Blocks for each of the individuals/organizations responsible for approving this document.

13. Some Preliminaries – Format and Content Requirements • A Distribution List – Containing the Names, Mailing Addresses, Phone Numbers, and Email Addresses for each of the individuals and organizations requiring copies of the approved QAPP. • Table of Contents – For Text, Tables, Figures, Maps & Appendices. If there are numerous Tables, Figures & Maps– Place these items in the Appendix to reduce breakup of the text.

14. Some Cautionary Tips!!! • Some Cautions: • Avoid using generic language that does not provide the required information or level of detail required. • For projects requiring the generation of chemical or biological data, make sure that you produce a list of contaminants of concern – or identify the biological parameters of interest. • Make sure the approved QAPP is distributedto project personnel, laboratory staff and if you are using CLP, identify the COCs in project log (unless there are numerous contaminants).

15. Let’s Start - Components of a QAPP • A QAPP is composed of approximately 25 elements that are grouped into four classes or categories as follows: • Class A – Project Management • Class B – Measurement/Data Acquisition • Class C – Assessment/Oversight • Class D – Data Validation/Data Usability

16. Class A Topics - Overview • The elements in this group address: • Project Management • Project History/Site History • Goals & Objectives of the Project • Project Outputs

17. Class A Topics • The following topicsmust be addressed as part of the Class A components/elements: • A1 – Title/Approval Page • A2 – Table of Contents • A3 – Distribution List • A4 – Project/Task Description • A5 – Problem Definition/Background Info • A6 – Project/Task Description • A7 – Quality Objectives & Criteria – DQOs/DQIs • A8 - Documents & Records

18. A4 Project/Task Organization • The following information is required: • Identify the individuals/organizations that will participate in the project/study – discuss their roles/responsibilities – identify the principal data users, decision makers, QA Manager, stakeholders and end data users. • QA Manager – should be iidentified in the QAPP –this individual should be independent of data collection operations, should have direct access to senior management, have overall authority over data collection activities when non-conformance with the QAPP is encountered. • An organizational chart depicting the lines of communication and authorities between senior management, the QAM and project personnel – should also include principal and end data users, decision makers, stakeholders, contractors & and any subcontractors.

19. Organizational Chart 1 Senior Management Laboratory Analysis Field Sampling Staff Data Validation

20. Organizational Chart 2 Senior Management Laboratory Analysis Data Validation Data Quality Assessment Field Sampling Staff QA Manager

21. Organizational Chart 3 Senior Management QA Manager Data Validation Data Quality Assessment Field Sampling Staff Laboratory Analysis

22. Organizational Chart 4 Senior Management QA Manager Project Management Field Sampling Staff Laboratory Analysis Data Validation & Data Quality Assessment

23. Organizational Chart 5 QA Manager Senior Management Project Manager Decision Makers Field Sampling Staff Laboratory Analysis Data Validation & Data Quality Assessment Organic Analysis Inorganic Analysis John WU DQA Linda Good D. Val. Joe Smo Jane Doe

24. A5 Problem Definition/Background • Summarize the problem to be solved • The decision to be made • Or outcometo be achieved • Include background/historicalinformation • Includescientific and regulatory perspectives

25. A6 Project/Task Description • Summarize all work to be done • Specify all measurements that must be taken – identify which measurements are critical or non-critical - critical measurements will be used to make site decisions – non-critical measurements won’t be used during the decision making process • Provide a list of all of the equipment required • Identify any products that will be produced • Provide Maps, Charts, Figures & Tables

26. A7 Quality Objectives & Criteria • Describe the quality goals/objectives for the project – provide the performance criteria for achieving these goals/objectives, etc. • Provide the project-specific data quality objectives (both qualitative and quantitative) and the specific data quality indicators (precision, bias, sensitivity, comparability, completeness and representativeness) relevant to the project/study.

27. Brief Overview of the Systematic Planning Process • Data Quality Objectives Process: • Step 1 – State the Problem • Step 2 – Identify the Goals of the Study • Step 3 – Identify the information inputs • Step 4 – Define the Boundaries of the Study • Step 5 – Develop the Analytical Approach • Step 6 – Specify Performance or Acceptance Criteria • Step 7 – Develop the Plan for Obtaining Data

28. Additional Thoughts on the DQO Process • Include any and all assumptions concerning site contamination, contaminant pathways, remedial techniques, clean-up design, monitoring strategies, etc., as part of the DQO process. • Identify any suspected potential departures from assumptions in support of the DQO process.

29. A8 Special Training/Certifications • Identify and describe any specialized training (including QA training) needed by project personnel required to successfully complete the project or task. • Discuss how such training will be provided – discuss who is responsible for obtaining internal training for staff. • Discuss where training documentationwill be maintained. • Specify whether professional certifications, accreditations or licenses are required for staff to perform their designated tasks/duties.

30. A9 Documents & Records • Describe the process and responsibilities for ensuring the appropriate project personnel have the most current approved version of the QAPP, including version control, updates, distribution and disposition. • Itemize the information required in project documents, records and reports. The type of information required for analytical data reports must be specified for both hard-copy and electronic formats. Data deliverables can and do include raw data, data from other sources such as computer databases, literature searches, field logs, sample preparation logs, analysis logs, instrument printouts, model inputs and outputs files, and the results of calibrations and QA checks.

31. A9 Documents & Records • Specify whether status/progress reports and final reports are required. • Specify or reference all applicable requirements for the final disposition of records/documents, including location and retention time. • Identify the individualswho are responsible for preparing project documents, records and reports – also identify who within EPA will receive this information.

32. Class B Topics - Overview • Discuss all aspects of data collection and generation • Describe sampling design and providerationale for your approach • Specify the analytical measurements both field and fixed laboratory • Describe sample handling and chain-of-custody requirements • Specify QA/QC samples with acceptance criteria

33. Class B Topics • B1 – Sampling Process Design • B2 – Sampling Methods • B3 – Sample Handling & Custody • B4 – Analytical Methods • B5 – Quality Control • B6 – Instrument/Equipment Testing, Inspection & Maintenance • B7 – Instrument/Equipment Calibration & Frequency • B8 - Inspection/Acceptance of Supplies & Consumables • B9 – Non-Direct Measurements • B10 – Data Management

34. B1 Sampling Process Design & Experimental Design • Describe the experimental data generation or data collection design for the project, including as appropriate: • The types & numbers of samples required • The design of the sampling network • The sampling locations, frequency of collection at each location and sample matrices • The measurement parameters of interest, and • The rationale for the sampling design chosen.

35. Sampling Designs Should be Consistent with your Conceptual Models!! • Evaluate your underlying assumptions -whether they are conscious or unconscious • Use a statistical tool or sampling tool such as Visual Sample Plan to test your sampling design. • Use historical data if available to determine the actual distribution of contaminants.

36. B1 Sampling Designs • Directed Sampling Designs • Judgmental Sampling • Probability Sampling Designs • Simple Random • Systematic/Grid • Stratified • Composite • Adaptive • Collaborative ( Double) • Hot Spot

37. Judgmental Sampling Design - Pros • Judgmental sampling is the subjective selection of sampling locations in space & time by an individual analyst or expert. • Consistent with intuitive feeling • Easy to direct, easy to do • May be cost effective if the conceptual site model for the project is correct • Great if you know absolutely everything there is to know about the site and your conceptual site model is absolutely correct.

38. Judgmental Sampling Design - Cons • Inference from sample to population questionable • Use of incorrect conceptual model can lead to incorrect decisions – can be a disaster. • Not suitable for estimating underlying population parameters (e.g., mean) with specified confidence – Cannot use statistics to evaluate distribution of data with any degree of confidence – with this sampling design this is no underlying assumption that the data are normally distributed. • Not suitable for testing hypothesis about underlying populations with specified decision error rates

39. Simple Random Sampling - Pros • Simple in concept and provides proper (theoretical support) data for statistical data analysis – representative sampling locations are chosen using the theory of random chance probabilities • Protects against bias in estimating parameters (e.g., means) and testing hypothesis • Is the basic building block of more complicated (and efficient) sampling designs.

40. Simple Random Sampling - Cons • Ignores available information that could be used to develop more cost-effective sampling designs • Not as effective as other designs for delineating patterns of contamination or finding hot spots • Difficult to find randomly selected sampling locations • Tends to demand large numbers of samples

41. Systematic (Grid) Sampling • Systematic (grid) sampling consists of collecting samples according to a specified pattern at regular intervals in space or time within a grid pattern: • Square or rectangular grid patterns over space • Equal-interval sampling along a straight line

42. Systematic Sampling - Pros • Easy to explain and implement and provides uniform coverage of site or project • Good for estimating boundaries, trends, or patterns of contamination over space or time. • May yield more precise estimates of population parameters than other sampling designs • Required for statistical data analysis to estimate trends and spatial patterns

43. Systematic Sampling - Cons • Systematic sampling can cause estimated means to be biased if the sampling grid pattern lines up with any pattern of contamination. • More information is needed (than for simple random sampling) about the population to estimate the variance of the estimated mean.

44. Stratified Sampling • The target population is divided meaningfully into contiguous sub-populations called strata • Sampling locations are selected independently within each stratausing some sampling design

45. Stratified Sampling - Pros • Dramatically reduces the variability present in the population and hence improves precision • Enables estimates of individual areas to be made • Assists in providing good coverage of the project • Allows for increased samples from policy or project sensitive areas

46. Stratified Sampling - Cons • Requires advanced knowledge in order to divide the study area into roughly homogeneous strata before sampling • The number of samples to be taken in each stratum must be determined • If strata boundaries are inaccurate, what appears to be outlier data can appear due to being in the wrong strata

47. Composite Sampling • Many individual (grab) samples are combined and thoroughly mixed to make a homogeneous whole. • At random, sub-samples (composite samples) are made and sent to the laboratory for analysis. • The physical size of composite samples are the same size as those obtained at random.

48. Composite Sampling - Pros • Allows for estimating the mean concentration with the same precision at a lower cost • Provides better coverage of the study site without increasing the number of chemical analyses • Allows for a more representative sample from a basic area of sample support (sampling unit). • Can be used in combination with other sampling designs.

49. Composite Sampling - Cons • Information on individual samples used to form composite samples is lost in compositing • Potential for loss of contaminants (volatiles) during the mixing and handling phase • Potential for reactions and interactions among analytes during compositing • Need to make decision on how many grab samples to be composited and how many composite samples to send for analysis

50. WHY IS YOUR SAMPLING DESIGN IMPORTANT!! • UNCERTAINTY!!! • UNCERTAINTY!!! • UNCERTAINTY!!! Due to the Variability Between Analytical Results Within a Given Data Set??? OR Due to Sampling Issues???