Computer Vision Chapter 4

1. Computer VisionChapter 4 Statistical Pattern Recognition Presenter: 小彼得 E-mail: itspeter@gmail.com Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.

2. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

3. Introduction • Units: Image regions and projected segments • Each unit has an associated measurement vector • Using decision rule to assign unit to class or category optimally DC & CV Lab. CSIE NTU

4. Introduction (Cont.) • Feature selection and extraction techniques • Decision rule construction techniques • Techniques for estimating decision rule error DC & CV Lab. CSIE NTU

5. Simple Pattern Discrimination • Also called pattern identification process • A unit is observed or measured • A category assignment is made that names or classifies the unit as a type of object • The category assignment is made only on observed measurement (pattern) • This is called Fair Game Assumption DC & CV Lab. CSIE NTU

6. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

7. Simple Pattern Discrimination (cont.) • a: assigned category from a set of categories C • t: true category identification from C • d: observed measurement from a set of measurements D • (t, a, d): event of classifying the observed unit • P(t, a, d): probability of the event (t, a, b) DC & CV Lab. CSIE NTU

8. Economic Gain Matrix • e(t, a): economic gain/utility with true category t and assigned category a • A mechanism to evaluate a decision rule • Identity gain matrix DC & CV Lab. CSIE NTU

9. Identity Gain Matrix DC & CV Lab. CSIE NTU

10. An Instance DC & CV Lab. CSIE NTU

11. Another Instance P(g, g): probability of true good, assigned good, P(g, b): probability of true good, assigned bad, ... e(g, g): economic consequence for event (g, g), … e positive: profit consequence e negative: loss consequence DC & CV Lab. CSIE NTU

12. Another Instance (cont.) DC & CV Lab. CSIE NTU

13. Another Instance (cont.) DC & CV Lab. CSIE NTU

14. Another Instance (cont.) • Fraction of good objects manufactured P(g) = P(g, g) + P(g, b) • Fraction of good objects manufactured P(b) = P(b, g) + P(b, b) • Expected profit per object E =P(g, g)*e(g,g) + P(g, b)*e(g,b) + P(b, g)*e(b,g) + P(b, b)*e(b,b) DC & CV Lab. CSIE NTU

15. Conditional Probability DC & CV Lab. CSIE NTU

16. Conditional Probability (cont.) • P(b|g): false-alarm rate • P(g|b): misdetection rate • Another formula for expected profit per object DC & CV Lab. CSIE NTU

17. Example 4.1 P(g) = 0.95, P(b) = 0.05 DC & CV Lab. CSIE NTU

18. Example 4.1 (cont.) DC & CV Lab. CSIE NTU

19. Example 4.2 P(g) = 0.95, P(b) = 0.05 DC & CV Lab. CSIE NTU

20. Example 4.2 (cont.) DC & CV Lab. CSIE NTU

21. Decision Rule Construction • (t, a): summing (t, a, d) on every measurements d • Therefore, • Average economic gain DC & CV Lab. CSIE NTU

22. Decision Rule Construction (cont.) DC & CV Lab. CSIE NTU

23. Decision Rule Construction (cont.) • We can use identity matrix as the economic gain matrix to compute the probability of correct assignment: DC & CV Lab. CSIE NTU

24. Fair Game Assumption • Decision rule uses only measurement data in assignment; the nature and the decision rule are not in collusion • In other words, P(a| t, d) = P(a| d) DC & CV Lab. CSIE NTU

25. Fair Game Assumption (cont.) • From the definition of conditional probability DC & CV Lab. CSIE NTU

26. Fair Game Assumption (cont.) • Derive P(t,a|d) = P(a|d)P(t|d) • By def., P(t,a|d) = P(t,a,d) / P(d) ----(1) • By def., P(a|t,d) = P(t,a,d) / P(t,d) ----(2) • By Fair Game Assumption P(a|t,d) = P(a|d) ----(3) • (1)+(2)可得 P(t,a|d)*P(d) = P(a|t,d)*P(t,d) • (3)代入得 P(t,a|d)*P(d) = P(a|d)*P(t,d) Q.E.D DC & CV Lab. CSIE NTU

27. DC & CV Lab. CSIE NTU

28. Deterministic Decision Rule • We use the notation f(a|d) to completely define a decision rule; f(a|d) presents all the conditional probability associated with the decision rule • A deterministic decision rule: • Decision rules which are not deterministic are called probabilistic/nondeterministic/stochastic DC & CV Lab. CSIE NTU

29. Expected Value on f(a|d) • Previous formula • By and => DC & CV Lab. CSIE NTU

30. Expected Value on f(a|d) (cont.) DC & CV Lab. CSIE NTU

31. Bayes Decision Rules • Maximize expected economic gain • Satisfy • Constructing f DC & CV Lab. CSIE NTU

32. Bayes Decision Rules (cont.) DC & CV Lab. CSIE NTU

33. Bayes Decision Rules (cont.) + + DC & CV Lab. CSIE NTU

34. Continuous Measurement • For the same example, try the continuous density function of the measurements: and • Measurement lie in the close interval [0,1] • Prove that they are indeed density function DC & CV Lab. CSIE NTU

35. Continuous Measurement (cont.) • Suppose that the prior probability of is 1/3 and the prior probability of is 2/3 • When , a Bayes decision rule will assign an observed unit to t1, which implies => DC & CV Lab. CSIE NTU

36. Continuous Measurement (cont.) • .805 > .68, the continuous measurement has larger expected economic gain than discrete DC & CV Lab. CSIE NTU

37. Break DC & CV Lab. CSIE NTU

38. Prior Probability • The Bayes rule: • Replace with • The Bayes rule can be determined by assigning any categories that maximizes DC & CV Lab. CSIE NTU

39. Equal-probability of ignorance • Equal-probability of ignorance assumption • P(t) is likely equal. • Put identity gain matrix together. • Maximum likelihood decision rule. DC & CV Lab. CSIE NTU

40. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

41. Economic Gain Matrix • Do they have different decision rules ? DC & CV Lab. CSIE NTU

42. Economic Gain Matrix • Identity matrix • Incorrect loses 1 • A more balanced instance DC & CV Lab. CSIE NTU

43. Economic Gain Matrix • Suppose are two different economic gain matrix with relationship • According to the construction rule. Given a measurement d, • Because • We then got DC & CV Lab. CSIE NTU

44. Economic Gain Matrix • Summary • Under positive multiplicative constant and additive constantwill not change the optimal rule. DC & CV Lab. CSIE NTU

45. Economic Gain Matrix • In some cases, non regular matrix can derive optimal rule. • Revisit the continuous example • Let the economic matric be arbitrary, say it is • Optimal rule is to assign t1 when • Substituting the density for P(t1,x) , P(t2,x) • Finally, reach assigns category t1 DC & CV Lab. CSIE NTU

46. Overview of this chapter • Introduction • Bayes Decision Rules • Economic Gain Matrix • Maximin Decision Rule • Decision Rule Error • Reserving Judgment • Nearest Neighbor Rule • Binary Decision Tree Classifier • Neural Networks

47. Maximin Decision Rule • Recall, we have done that… • Finding function f(a|d) under… • given d, pick “a” that maximize • What if we don’t know P(t,d) and P(t) is not reasonable to be assume even. • Equal-probability-of-ignorance assumption is not true DC & CV Lab. CSIE NTU

48. Maximin Decision Rule • Maximizes average gain over worst prior probability DC & CV Lab. CSIE NTU

49. Maximin Decision Rule • Simplify using the assumption that • Thus, reaching worst prior of • When and others have zero probability • Formula became , instead of DC & CV Lab. CSIE NTU

50. Example 4.3 DC & CV Lab. CSIE NTU