1. The Expression of Uncertainty in Measurement Bunjob Suktat JICA Uncertainty Workshop January 16-17, 2013 Bangkok, Thailand

2. Acceptance of the Measurement Results

3. Contents • Introduction • GUM Basic Concepts • Basic Statistics • Evaluation of Measurement Uncertainty • How is Measurement Uncertainty estimated? • Reporting Result • Conclusions and Remarks

4. Introduction • Guide to the Expression of Uncertainty in Measurement was published by the International Organization for Standardization in 1993 in the name of 7 international organizations • Corrected and reprinted in 1995 • Usually referred to simply as the “GUM”

5. Guide to the Expression of Uncertainty in Measurement (1993) BIPM - International Bureau of Weights and Measures http//: www.bipm.org International Organisations IEC - International Electrotechnical Commision http//: www.iec.ch IFCC - International Federation of Clinical Chemistry http//: www.ifcc.org IUPAP - International Union of Pure and Applied Physics http//: www.iupap.org IUPAC - International Union of Pure and Applied Chemistry http//: www.iupac.org ISO - International Organisation for Standardisation http//: www.iso.ch OIML - International Organisation for legal metrology http//: www.oiml.org

6. Basic concepts • Every measurement is subject to some uncertainty. • A measurement result is incomplete without a statement of the uncertainty. • When you know the uncertainty in a measurement, then you can judge its fitness for purpose. • Understanding measurement uncertainty is the first step to reducing it

7. Introduction to GUM • When reporting the result of a measurement of a physical quantity, it is obligatory that some quantitative indication of the quality of the result be given so that those who use it can assess its reliability. • Without such an indication, measurement results can not be compared, either among themselves or with reference values given in the specification or standard. GUM 0.1

8. Stated Purposes • Promote full information on how uncertainty statements are arrived at • Provide a basis for the international comparison of measurement results

9. Benefits • Much flexibility in the guidance • Provides a conceptual framework for evaluating and expressing uncertainty • Promotes the use of standard terminology and notation • All of us can speak and write the same language when we discuss uncertainty

10. Uses of MU • QC & QA in production • Law enforcement and regulations • Basic and applied research • Calibration to achieve traceability to national standards • Developing, maintaining, and comparing international and national reference standards and reference materials • GUM 1.1

11. After uncertainty evaluation No uncertainty evaluation (only precision) value 12.5 12.0 11.5 mg kg-1 11.0 10.5 R1 R2 R1 R2 R1 R2 Are these results different?

12. En-score according to GUM “Normalized” versus ... propagated combined uncertainties • Performance evaluation: • 0 <|En|< 2 : good • 2 <|En|< 3 : warning  preventive action • |En|>3 : unsatisfactory  corrective action

13. What is Measurement? Measurement is ‘Set of operations having the object of determining a value of a quantity.’ ( VIM 2.1 ) Note: The operations may be performed automatically.

14. Basic concepts • Measurement • the objective of a measurement is to determine the value of the measurand, that is, the value of the particular quantity to be measured • a measurement therefore begins with • an appropriate specification of the measurand • the method of measurement and • the measurement procedure GUM 3.1.1

15. Principles of Measurement Method of Comparison DUT Result Standard

16. Basic concepts • Result of a measurement • is only an estimate of a true value and only complete when accompanied by a statement of uncertainty. • is determined on the basis of series of observations obtained under repeatabilityconditions • Variations in repeated observations are assumed to arise becauseinfluence quantities GUM 3.1.2 GUM 3.1.4 Gum 3.1.5

17. Influence quantity • Quantity that is not the measurand but that affects the result of measurement. • Example : temperature of a micrometer used to measure length. ( VIM 2.7 )

18. What is Measurement Uncertainty? • “parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” – GUM, VIM • Examples: • A standard deviation (1 sigma) or a multiple of it (e.g., 2 or 3 sigma) • The half-width of an interval having a stated level of confidence

19. Uncertainty • The uncertainty gives the limits of the range in which the “true” value of the measurand is estimated to be at a given probability.. • Measurement result = Estimate ± uncertainty • (22.7 ± 0.5) mg/kg • The value is between 22.2 mg/kg and 23.2 mg/kg

20. Measurement Error Measurement Error Real Number System Measured Value True Value Measured values are inexact observations of a true value. The difference between a measured value and a true value is known as the measurement error or observation error.

21. Basic concepts • The error in a measurement • Measured value – True value. • This is not known because: • The true value for the measurand • This is not known • The result is only an estimate of a true value and only complete when accompanied by a statement of uncertainty. GUM 2.2.4 GUM 3.2.1

22. Random & Systematic Errors • Error can be decomposed into random and systematic parts • The random error varies when a measurement is repeated under the same conditions • The systematic error remains fixed when the measurement is repeated under the same conditions

23. Random error • Result of a measurement minus the mean result of a large number of repeated measurement of the same measurand. ( VIM 3.13 )

24. Random Errors • Random errors result from the fluctuations in observations • Random errors may be positive or negative • The average bias approaches 0 as more measurements are taken

25. Random error • Presumably arises from unpredictable temporal and spatial variations • gives rise to variations in repeated observations • Cannot be eliminated, only reduced. GUM 3.2.2

26. Systematic Errors Mean result of a large number of repeated measurements of the same measurand minus a true value of the measurand. ( VIM 3.14 )

27. Systematic Errors • A systematic error is a consistent deviation in a measurement • A systematic error is also called a bias or an offset • Systematic errors have the same sign and magnitude when repeated measurements are made under the same conditions • Statistical analysis is generally not useful, but rathercorrections must be made based on experimental conditions.

28. Systematic error • If a systematic error arises from a recognized effect of an influence quantity • the effect can be quantified • can not be eliminated, only reduced. • if significant in size relative to required accuracy, a correction or correction factor can be applied to compensate • then it is assumed that systematic error is zero. GUM 3.2.3

29. Basic concepts Systematic error • It is assumed that the result of a measurement has been corrected for all recognised significant systematic effects GUM 3.2.4

30. Measurement Error Systematic error Random error

31. Correcting for Systematic Error • If you know that a substantial systematic error exists and you can estimate its value, include a correction (additive) or correction factor (multiplicative) in the model to account for it • Correction - Value that , added algebraically to the uncorrected result of a measurement , compensates for an assumed systematic error (VIM 3.15) • Correction Factor- numerical factor by which the uncorrected result of a measurement is multiplied to compensate for systematic error.  [VIM 3.16]

32. Uncertainty • The result of a measurement after correction for recognized systematic effects is still only an estimateof the value of the measurand because of the uncertainty arising; • from random effects and • from imperfect correction of the result for systematic effects GUM 3.3.1

33. Classification of effects and uncertainties • Random effects • Unpredictable variations of influence quantities • Lead to variations in repeated measurements • Expected value : 0 • Can be reduced by making many measurement • Systematic effects • Recognized variations of influence quantities • Lead to BIAS in repeated measurements • Expected value : unknown • Can be reduced by applying a correction which carries an uncertainty

37. Error versus uncertainty • It is important not to confuse the terms error and uncertainty • Error is the difference between the measured value and the “true value” of the thing being measured • Uncertaintyis a quantification of the doubt about the measurement result • In principle errors can be known and corrected • But any error whose value we do not know is a source of uncertainty.

38. Blunders • Blunders in recording or analysing data can introduce a significant unknown error in the result of a measurement. • Measures of uncertainty are not intended to account for such mistakes GUM 3.4.7

39. Basic Statistics

40. Population and Sample Samples • Parent Population The set of all possible measurements. • Sample A subset of the population - measurements actually made. Handful of marbles from the bag Population Bag of Marbles Slide 7

41. Histograms • When making many measurements, there is often variation between readings. Histogram plots give a visual interpretation of all measurements at once. • The x-axis displays a given measurement and the height of each bar gives the number of measurements within the given region. • Histograms indicate the variability of the data and are useful for determining if a measurement falls outside of “specification”.

42. For a large number of experiment replicates the results approach an ideal smooth curve called theGAUSSIAN or NORMAL DISTRIBUTION CURVE Characterised by: The mean value –x gives the center of the distribution The standard deviation – s measures the width of the distribution

43. “Sum of” Individual measurement Number of measurements Average • The most basic statistical tool to analyze a series of measurements is the average or mean value : The average of the three values 10, 15and 12.5 is given by:

44. Deviation Deviation = individual value – avg value Need to calculate an average or “standard” deviation To eliminate the possibility of a zero deviation, we square di

45. Standard Deviation • The average amount that each measurement deviates from the average is called standard deviation (s) and is calculated for a small number of measurements as: Sum of deviation squared xi = each measurement = average n = number of measurements Note this is called root mean square: square root of the mean of the squares

46. Standard Deviation

47. Standard Deviation For example, calculate the standard deviation of the following measurements: 10, 15 and 12.5 (avg = 12.5) The values deviate on average plus or minus 2.5 :12.5 ± 2.5 10.0 12.5 15.0

48. Other ways of expressing the precision of the data: • Variance Variance = s2 • Relative standard deviation • Percent RSD or Coefficient of Variation (CV)

49. Standard Deviation of the Mean The uncertainty in the best measurement is given by the standard deviation of the mean (SDOM)

50. Gaussian Distribution • Given a set of repeated measurements which have random error. • For the set of measurements there is a mean value. • If the deviation from the mean for all the measurements follows a Gaussian probability distribution, they will form a “bell-curve” centered on the mean value. • Sets of data which follow this distribution are said to have a normal (statistical) distribution of random data.