Some Metrological Aspects of Ordinal Quality Data Treatment

1. Some Metrological Aspects of Ordinal Quality Data Treatment *Emil Bashkansky Tamar Gadrich ORT Braude College of Engineering, Israel ENBIS-11 Coimbra, Portugal, September 2011, 11:50 – 13:20

2. Presentation Outline-stage I

3. Presentation Outline-stage II

4. Presentation Outline-stage III

5. Examples of ordinal scale usage

6. Why revision of MC for ordinal measurements is needed? * ISO/IEC Guide 99: International vocabulary of metrology — “Basic and general concepts and associated terms (VIM)”

7. Error description Classic continual: The probability density function pdf (Y/X) of receiving result Y, given the true value of the measurandX , in it's simplest form: pdf (Y/X) = Normal(X+bias, ) Ordinal: The conditional probabilities that an object will be classified as level j, given that its actual/true level is i.

8. Error -free ordinal measurement

9. Error –binary case

10. Some examples

11. Uncertainty-general case • The likelihood that a measured level jis received, whereas the true level is i

12. Uncertainty matrix- binary case

13. Inaccuracy- Error matrix :

15. Repeatability Blair & Lacy (2000)

16. Repeatability - the expected cumulative frequency of data/items classified up to the k-th category, given that its actual/true level is i

17. ORDANOVA:DECOMPOSITION OF TOTAL DISPERSION AFTER MEASUREMENT/CLASSIFICATION - the expected cumulative frequency of items belonging up to the k-th category after measurement

18. Reproducibility

19. Some definitions - conditional joint probability of sorting the measured object to the a-th level by the first MS (called A), and the b-th level by the second MS (called B), given the actual/true category i

20. A & B MSs classification matrices

21. Joint probability matrix pi- the probability that an object being measured relates to category i, ( ) - the joint probability of sorting the item as a by the first measurement system (A) and b by the second measurement system (B).

22. Modified Kappa measure of Agreement When a half of all items are correctly classified:

23. 1. Reproducibility - reference standard is known

24. 2. Reproducibility - reference standard is unknown

25. Binary case example

26. Typical relation between quality level and commonly used chemical/physical features for yellow-flesh nectarines

27. Ternary scale example (fruit quality classification) weighted total kappa equals 0.734

28. Repeated measurements-general case Let's consider arbitrary ordinal scale with m categories and suppose, that nrepeated measurements of the same object were performed resulting in vector: (n=n1+n2+…+nm)

29. Repeated measurements-general case The maximum likelihood estimation must be made in favor of such, most plausiblei , that maximizes the scalar product:

30. General case example-single measurement

31. General case example: after 10 repititions

32. SUMMARY On an ordinal measurement scale the essential for evaluating the error, repeatability and uncertainty of the measurement result base knowledge must be the classification/measurement matrix. Given this matrix, authors introduced a way to calculate the classification/measurement system’s accuracy, precision (repeatability & reproducibility) and uncertainty matrix. In order to estimate comparability and equivalence between measurement results received on an ordinal scale basis, the modified kappa measure is suggested. Three of the most suitable usages of the measure were thoroughly analyzed. The advantage of the proposed measure vs. the traditional one lies in the fact that the former follows the superposition principle: the total measure equals the weighted sum of partial measures for every ordinal category. As it is well known, repeated measurements may improve the quality of the measurement result. When decisions are ML based, one can find how many repetitions are necessary in order to achieve the desired accuracy level using the algorithm suggested by the authors,.

33. Thank Youfor your attention! E-mail: ebashkan@braude.ac.il