Simplifying Measurement Uncertainties

1. Simplifying Measurement Uncertainties Bill Hirt, Ph.D / February 2016

2. Start the Process • Light the fuse (candle) • Shed light on the way

3. Back to the beginning • Accredited calibration labs report measurements on tools and devices needing regular service … typically including uncertainties for those measurements. • Accredited testing labs, when requested or when needed to interpret Statements of Compliance, report uncertainties alongside their test measurements.

4. Framework for uncertainties • Unless calculating MDL’s or LOD’s or LOQ’s … • It is presumed that ISO 17025 accredited labs demonstrate the competence to calculate and report GUM uncertainties … estimated at roughly a 95 % confidence.

5. When do accredited labsreport MU’s ? • Every accredited calibration certificate, unless requested otherwise • Test reports : • When a customer requests it • When the uncertainty affects compliance with a specification limit • When it is relevant to the validity or application of the result

6. What do we mean by MU? • Measurement value ± uncertainty (MU) • MU usually at 95 % confidence – Why? • MU usually reported at k = 2

8. Where do we find most MUs? • On calibration certificates • On test reports • In footnotes

9. Did you know that … • A high percentage of ISO 17025 accredited calibration laboratories issue incorrect uncertainties on their certificates ?

10. ISO 17025 includes –Calibration and Testing • A good percentage of calibration labs do not report MU’s on their certificates • Most testing labs do not report MU’s on their test reports • And WHY ???

11. First Group Discussion • Scenario used in 3-day MU course outlined first • Have groups discuss other real-life scenarios where uncertainty can be critical or life-saving • Put examples on index cards on the tables

12. Uncertainty and Traceability • Metrological traceability (not sample traceability) • Two platforms of confidence • Comparison with hi-quality stds thru chain to SI through an NMI or DI • Confidence that GUM unc’s used … and actual measurement error clearly known

13. Uncertainty and Proficiency Testing (PT) • Calibration PT programs typically request that one or more devices have measurements taken … and Both the measurements and their uncertainties be reported. • The Cal PT study issues a report with standardized results including an En

14. Envalue Determination (Calibration Labs)

15. Interpretation of En values

16. Accredited Testing Labsand PT • Very few ISO 17025 accredited PT programs for testing include a requirement for MU’s • Some now request them • Many in the future will require them • In addition to z-scores, many future reports will include En values too

18. “The Big Three” in 17025 • Measurement Uncertainty • Metrological Traceability • Proficiency Testing

19. Uncertainty and Statements of Compliance • High percentage of measurements are made to ensure manufactured or natural materials meet a narrow range of specifications • Manufacturers and regulators define specs • No measurement is perfect • MU may or may not be critical to have CONFIDENCE that specs are being met

20. Group discussion - 2 • At your tables, discuss specific areas of compliance specs that you are aware of • Make a list of at least 6 different groups or types of specs at your table to report later • The group will share after your discussion

21. Uncertainty and Guard Banding • Management of manufacturing or other monitoring to assure material is safely within specifications, including consideration of uncertainty • Organization may adjust their spec to capture either more potentially defective samples or allow more acceptable product out the door

23. Definitions • Uncertainty – a property of a measurement result that defines the range of probable values of the measurand • Uncertainty budget – the systematic description of uncertainty determinations relevant to specific measurements including ranges plus all factors, assumptions and calculations included (must include both type A and type B factors)

24. Basic Steps in Uncertainty Budgets • List ALL potential factors affecting variability in measurements - make table • Determine the standard uncertainty for each factor (includes distribution) • Perform RSS for all factors to create the combined (standard) uncertainty • Multiply by distribution factor (k=2 … or ?)

25. Root Sum Squaring

26. Sample Standard Deviation(s)

27. Calibration MU example

28. Second Cal MU example

29. Testing MU • Very often much more complicated than calibration -- why ?? • Many stages in test processes • Many error types with different units of measure • Many errors not defined and require guess

31. Table A6.2: Summary of results from collaborative trial of the method and in-house repeatability check

32. Table A6.4: Combined standard uncertainties

34. Key current and future factor • SAMPLING and sample error factors

35. Nitrogen in Forage Budget

36. Nitrogen in Dry Feed Budget

37. New proposed TABLE • Handout shows old / traditional version of the Student’s T Table • Back side shows our proposed new version • Let’s review its features

38. Open quiz for the room • Using the new Student’s T table, what is the k-factor … for an MU … at 95% confidence … when the number of repeatability measurements is : • 3 ? • 5 ? • 10 ? • 30 ?

39. Open quiz for the room - 2 • For an MU … at 95% confidence … when the number of repeatability measurements is : • 3 • 5 • 10 • 30 • … and the test measurement mean is 100 mg and the combined uncertainty is 5 mg, what is each 95% MU : • With 3 repeats • With 5 repeats • With 10 repeats • With 30 repeats

40. Open quiz for the room - 3

41. Open quiz for the room – 3+

42. Example of questionable MU on commercial calibration certificate

43. Standard Deviation of the Mean • Caution – it does NOT replace repeatability SD • The equation on the left for repeatability / SD becomes 

44. Control Charts • Plots of long-term measurements of a single parameter to note trends or variability • Often the main basis for testing uncertainties • Main testing lab equivalent of repeatability and/or reproducibility

48. AAFCO example of Control Chart of Matrix Spike

49. Basic Steps in Uncertainty Budgets • List ALL potential factors affecting variability in measurements - make table • Determine the standard uncertainty for each factor (includes distribution) • Perform RSS for all factors to create the combined (standard) uncertainty • Multiply by distribution factor (k=2 … or ?)