2011 DOE ASP – Pleasanton, CA

1. Understanding MARLAP Decision Making at Detection and Quantitation Limits 2011 DOE ASP – Pleasanton, CA

2. Robert P. Di Rienzo ALS Environmental

3. There may be excessive use of acronyms greater than 5 letters and statistical calculations in this presentation. Psychotherapy Guidance is suggested

4. Do you know what the acronym MARLAP stands for?

5. Multi-Agency Radiological Laboratory Analytical Protocols Manual

6. Where to Now? • We will talk about several principles recommended by MARLAP pertaining to uncertainty and detection capability • MARLAP approach to detection and uncertainty • Statistics again • Sources of uncertainty • Type B vs. Type A evaluations of uncertainty • How does quality control relate to uncertainty and detection statistics? • Decisionmaking

7. MARLAP Critical Value and MDA The MARLAP Glossary defines: Critical Value as: In the context of analyte detection, the minimum measured value (e.g., of the instrument signal or the analyte concentration) required to give confidence that a positive (nonzero) amount of analyte is present in the material analyzed. The critical value is sometimes called the critical level or decision level. and Minimum Detectable Concentration (MDA) as: the smallest (true) value of the net state variable that gives a specified probability that the value of the response variable will exceed its critical value—, i.e., that the material analyzed is not blank.

8. MARLAPMinimum Quantifiable Concentration (MQC) The MARLAP Glossary defines the MQC as: Minimum Quantifiable Concentration (MQC) (3.3.7): The minimum quantifiable concentration, or the minimum quantifiable value of the analyte concentration, is defined as the smallest concentration of analyte whose presence in a laboratory sample ensures the relative standard deviation of the measurement does not exceed a specified value, usually 10 percent.

9. Critical Value, MDA and MQC Critical Value MDA MQC

10. In General MARLAP recommends: • Use a performance-based approach where possible • Use validated approaches to measurements • Use measurement uncertainty as basis for decisionmaking • Ensure that data quality is sufficient for decisionmaking by linking QC to MQOs

11. MARLAP Approach to Detection MARLAP 19.3.9 • Detection in radiochemistry based on the uncertainty of the instrument signal obtained by counting analyte-free sources; • Should be based on the uncertainty obtained when analyte-free samples are analyzed.

12. MARLAP Recommendations on Uncertainty MARLAP 19.3.9 • Adopt terminology and methods of the GUM for evaluating and reporting uncertainty. • Account for both random and systematic effects • Exclude blunders and spurious errors

13. MARLAP Recommendations on Uncertainty MARLAP 19.3.9 • Evaluate and propagate uncertainty from all sources believed to be potentially significant in the final result. • Follow QC procedures that ensure a state of statistical control, which is a prerequisite for uncertainty evaluation.

14. GUM Law of Propagation of Uncertainty

15. Special Form of the Uncertainty Propagation Formula (MARLAP 19.4.3.3)

16. Sources of Uncertainty Reasonable estimates of uncertainty include all factors that contribute uncertainty.

17. Sources of Uncertainty Not all sources of uncertainty may be significant but they should at least be considered: radiation counting instrument calibration (e.g., counting efficiency) tracers, carriers contamination of reagents and tracers yield measurement subsampling instrument backgrounds crosstalk and spillover Volume measurements (e.g., pipetting) Mass (i.e., weighing)

18. GUM Type B Evaluation of Uncertainty • Type B estimations are based on experience, knowledge of process – not on measurements • Different distributions–trapezoidal, rectangular, others • Imported values (e.g., branching ratios) • Radiation counting - Poisson Uncertainty (a.k.a. count uncertainty) • May be limited or inappropriate due to an incomplete or faulty understanding of the process

19. GUM Type B Evaluation of Uncertainty Radiation Counting • Square root of the counts as an estimate of Poisson uncertainty of a count • A significant and variable component of uncertainty • Represents the minimum uncertainty due to the random nature of radioactivity counting • Not always sufficient since non-Poisson sources also contribute to uncertainty • Does not account for method bias

20. MARLAP Warns… Section 19.5.5 • It is inappropriate to use the Poisson model: • If the amount of blank contaminant varies between measurements • If the causes of blank contamination are not well understood. • It is usually necessary to determine the blank level and its uncertainty by replicate measurements (a Type A evaluation).

21. GUM Type A Evaluation of UncertaintyMARLAP 19.4.2.1 • Determined by a series of experimental observations • Activity of blanks or background • Activity of spiked samples • Includes all processing steps

22. GUM Type A Evaluation of UncertaintyMARLAP 19.4.2.1 We calculate the mean and standard uncertainty and use these as an input for uncertainty propagation

23. Batch QC Samples,Uncertainty & Detection Statistics • Batch QC Samples are all observed measures of method performance • Blanks • LCS • Duplicates • Other

24. Conclusions on Detection and Uncertainty • Evaluation of uncertainty per GUM • Blank samples should be used to test the process for absolute bias • Other QC samples can be used to test for relative bias and precision

25. Conclusions on Detection and Uncertainty • Uncertainty must be assessed across the entire range • Traditional type B estimates of uncertainty may be low • Estimating the uncertainty at the MDA and MQC may require samples spiked at the these levels.

26. Critical Value, MDA and MQC Reportable a posteriori value Critical Value MDA MQC MQOs to Lab

27. Do you know what the acronym UFP QAPP stands for?

29. UFP QAPP A QAPP presents the steps that should be taken to ensure that environmental data collected are of the correct type and quality required for a specific decision or use. Definitive data — Analytical data of known quality, concentration, and level of uncertainty.

30. Do you know what the acronym MARSSIM stands for?

32. MARSSIM MARSSIM presents a systemized approach for designing surveys to collect data needed for making decisions such as whether or not to release a site. The guidance given in MARSSIM is performance-based and directed towards acquiring site-specific data.

34. MARSSIM Using the DQO Process ensures that the type, quantity, and quality of environmental data used in decision making will be appropriate for the intended application.

35. action level (1.4.9): The term action level is used in this document to denote the value of a quantity that will cause the decisionmaker to choose one of the alternative actions. MARLAP decision uncertainty (1.4.7): Refers to uncertainty in the decisionmaking process due to the probability of making a wrong decision because of measurement uncertainties and sampling statistics. Decision uncertainty is usually expressed as by the estimated probability of a decision error under specified assumptions. MARLAP

36. Additional Components of decision uncertainty • Representative Sampling in the field • Sample Collection Activities • Field Procedures • FSMO Accreditation/Assessment

37. UncertaintyDefinitions • a term used in subtly different ways in a number of fields, including philosophy, physics, statistics, economics, finance, insurance, psychology, sociology, engineering, and information science. It applies to predictions of future events, to physical measurements already made, or to the unknown. • 2) If outcomes will occur with a probability that cannot even be estimated, the decision-maker faces uncertainty.

39. Conclusions for Laboratory • Use uncertainty concept in MARLAP to determine MDA and MQC. • Uses uncertainty at appropriate levels when determining MDA and MDC. • Be ready to provide project with MDA and MQC that meets DQO objectives.

40. Conclusions for Projects • Determine what levels of uncertainty are appropriate for decision making. • Develop DQOs based on uncertainty and performance. • Provide laboratory with DQOs and measurement performance criteria based on determined project objectives.

41. Acknowledgements Thanks to the support of Mr. Bob Shannon to help me understand MARLAP and to Ms. Marlene Moore for her guidance over the years on understanding uncertainty. Thanks to ALS Environmental, Mr. Ken Campbell and Mr. Brent Stephens for their allowing me to spend the time needed to prepare and present to you.

42. Thank You Very Much Robert P. Di Rienzo ALS Environmental Bob.DiRienzo@ALSGlobal.com