Quality Control

1. Quality Control what to compare against the validation templates (see templates in course webpage: http://itep68.itep.nau.edu/itep_downloads/QA101_Resources/Validation%20Templates%20from%20Redbook%20QA%20Handbook%20vol%20II.pdf )

2. Overall Course Overview: • April 7: QA Systems, EPA definitions, PQAOs and common sense – Mike Papp • April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks) – Travis Maki • April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) – Jeremy Howe • April 28: Calculating Bias and Precision and AQS reports – Angie Shatas • May 5: 40 CFR 58 App. A- Gaseous Pollutants – Glenn Gehring • May 12: 40 CFR 58 App. A- Ozone – Brenda Jarrell • May 19: 40 CFR 58 App. A- PM filter and continuous methods –Brandy Toft

3. QC: An Ongoing System • Measuring • Comparing with MQO • Graphing it/ watching for its drift • Fixing it when needed Everything must be documented and, when significant, reported

4. Objectives of QC: • Instruments operating stably • Measure what you can, verify it is: • Within limits • Not slowly drifting up or down • Verify you are meeting your QAPP specs • QAPP specs assess bias and imprecision • Ideal is to estimate bias and imprecision, so that you can know and limit your overall error • Write it down!

5. Evaluate where “Things Can Go Wrong”—plan how to check and what to document • Preparing for the field • Checking your equipment • Analyzer working, QC within limits • Transfer standards also need to be working • Supporting stuff: • Site, power, weather, frequency of your checks ok? Look to the future • Data recording system working • Routine data • QC data • Downloading the data

6. Error …the difference between your answer and the “truth” Two components of total error (accuracy): • Bias (jump) • Precision (wiggle)

7. Bias Error • Minimized by calibrating your equipment against a standard • Make sure the standard has TRACEABILITY to a NIST standard • Compare result of the standard(s) to what your instruments read (pollutant, flow rate, temp, pressure) Measures any bias in your equipment at that time

8. Precision Error • Some imprecision is unavoidable • Sometimes up, sometimes down–“random” • Estimated by measuring the same thing several times • Or by side-by-side collocated • Minimized by carefully following procedures

9. QC Checks are made using transfer standards: • The term “transfer” means that you are “transferring” the reliability of a measurement from a primary calibration standard to your field instrument • The primary calibration standard can be the place where you send your transfer standard to be recalibrated annually • Pressure, temperature, and flow rate of your instruments must be checked with your transfer standards

10. Transfer Standards: • For example, from CFR 50 Appendix L: “An authoritative flow rate standard shall be used for calibrating or verifying the sampler's flow rate measurement device with an accuracy of ±2 percent. The flow rate standard shall be a separate, stand-alone device ... This flow rate standard must have its own certification and be traceable to a National Institute of Standards and Technology (NIST) primary standard for volume or flow rate.

11. Transfer standards needed:

12. Example of QC check (verification) • Perform a check of your equipment: • flow rate for PM • concentration for gas • internal mass for TEOM • This may be single-point (one flow rate) or multi-point (several flow rates) • If this is within specifications, record this and continue

13. Calibrations (covered in more detail in webinars #4-7) • If results are NOT within specs, then the instrument must be adjusted • This adjustment means that the response of the instrument is changed, which is defined as a CALIBRATION • Calibrations must be multi-point • After calibrating, do QC check again

14. Example of QC checks for PM2.5 • Corrective Action if the flow rate difference between the transfer standard and the sampler is > 4% • Check sampler for internal and external leaks • Ensure that temp. and pressure are within normal ranges • Run check a 2nd time • If still > 4%, perform a multipoint calibration followed by a single-point verification

15. QC checks for gaseous-every 2 weeks: • 1-point QC check must show within 7% for O3, 10% for others (if starting out, every week) • Zero must not drift more than 2% of full scale • Span concentration must not drift more than 7% since last check

16. Gaseous Verifications • Gaseous instrument verifications can be conducted: • Through-the-probe (is what you will be audited on—do this if possible) • Through-the-manifold • Through the back of the analyzer • To evaluate the sampling system it is recommended that at least some verifications be conducted through-the-probe

17. QC Checks (and audits):

18. Who can do these, and with what?

19. Gaseous Standards All calibrations, verifications, and audits should be conducted with certified gas standards or a certified transfer standard (ozone). Gas standards are certified by the vendor and can be recertified (EPA Protocol 600/R-97/121) Ozone transfer standards should be annually certified and traceable to a Standard Reference Photometer (SRP)

20. What do I do with QC results? • DASC (Data Assessment Statistical Calculator) • In Excel, easy to use • Calculates what EPA calculates in AMP255 • Can be used with any parameters • Cites equation #s from CFR • AQS: Data Quality Indicators Report (AMP255) • Covers all of the Requirements • http://www.epa.gov/ttnamti1/qareport.html

21. Calculations: • Consider the standard to be the “truth” for your equipment • Difference between the “truth” and your equipment is the instrument error (this may include bias [error either usually high or usually low], and imprecision [wiggle sometimes high and sometimes low]) • How is instrument error quantified?

22. Percent Difference used for both Precision and Bias Basis for All Calculations (divide by best estimate of the truth, to get a %-off-from-truth): • For standard gas • For Collocated Samples

23. Next steps of QC If you are outside the limits: • Review procedures & logs to identify problem • Go back and review data—may have to throw out (invalidate—never delete) data to the last passing check (see validation templates for critical criteria that require invalidation—may not have to invalidate data if it’s a systematic or operational criterion) • Fix the problem, document it and report (when significant)

24. Accuracy = Total Error • “Accuracy” for EPA means “total error” • Comparison conducted with completely different system can be used to estimate total error • For PM2.5, this is a performance evaluation conducted with a side-by-side FRM device, with the filter weighed by a different lab • Total error includes both precision and bias errors • Accuracy estimates are verified thru audits

25. Resources • AMTIChttp://www.epa.gov/ttn/amtic/ • Transfer Standards for Calibration of Air Monitoring Analyzers for Ozone:Technical Assistance Document http://www.epa.gov/ttnamti1/files/ambient/qaqc/OzoneTransferStandardGuidance.pdf • U. S. EPA Ambient Air Monitoring Protocol Gas Verification Program: Implementation Plan 2nd DRAFT 7/2009 http://www.epa.gov/ttnamti1/files/ambient/qaqc/aapgvpimpplan.pdf EPA NPAP TTP PE SOP COMPENDIUM (DRAFT FINAL, 1-12-05) http://www.epa.gov/ttn/amtic/files/ambient/npapsop/npapttpsop.pdf

26. Resources • Field Standard Operating Procedures for the Federal PM2.5 Performance Evaluation Program http://www.epa.gov/ttn/amtic/files/ambient/pm25/qa/PEP_Field_SOP.pdf • Pb Performance Evaluation Program http://www.epa.gov/ttn/amtic/files/ambient/pb/Pb-PEPImplementationPlan_7_09.pdf • Model Quality Assurance Project Plan For the National Air Toxics Trends Stations http://www.epa.gov/ttn/amtic/files/ambient/airtox/NATTS_Model_QAPP.pdf Resources—National QA Workgroup • QA National Meeting Presentations http://www.epa.gov/ttn/amtic/qaqcrein.html

27. Conclusions: • QC checks are necessary to both do, and • Track results of • Subsequent webinars will cover pollutant specifics QC checks • Course website: http://itep68.itep.nau.edu/itep_downloads/QA101_Resources/ • Our emails: • Travis Maki: tmaki@itcmi.org • Melinda.ronca-battista@nau.edu • Christopher.lee@nau.edu