 Download Download Presentation Measuring Uncertainty in Graph Cut Solutions

# Measuring Uncertainty in Graph Cut Solutions

Download Presentation ## Measuring Uncertainty in Graph Cut Solutions

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Measuring Uncertainty in Graph Cut Solutions Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University

2. st-mincut Most Probable Solution s t Labelling problem Objective Graph Cut No uncertainty measure associated with the solution Belief or Confidence

3. Outline • Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results

4. Outline • Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results

5. Inference in Graphical Models x: Set of latent variables D: Observed Data x*: Most Probable (MAP) Solution Pr: Joint Posterior Probability x* = arg maxPr(x|D) x E(x|D) = -log Pr(x|D) + constant x* = arg minE(x|D) x

6. Inference in Graphical Models MAP Solution arg max Pr(A,B) {A=1, B=0} B 0 1 2 0 Joint Distribution A 1 Pr(A,B) 2

7. Inference in Graphical Models • Marginal • Sum the joint probability over all other variables. B 0 1 2 0 A P(A=1) = 0.6 1 2

8. Inference in Graphical Models • Max- Marginals (µ) • Maximum joint probability over all other variables. B 0 1 2 0 A µA,1 = 0.35 1 2

9. Inference in Graphical Models • Confidence or Belief (σ) • Normalized max-marginals σA,1 = µA,1 / Σx µA,x = 0.35/ 0.58 = 0.603 B 0 1 2 µA,0 = 0.2 0 A µA,1 = 0.35 1 2 µA,2 = 0.03

10. Inference in Graphical Models • Min-Marginals Energies(ψ) • Minimize joint energy over all other variables. • Related to max-marginals as: µj = (1/z)*exp(-ψj)

11. Inference in Graphical Models • Min-Marginals Energies(ψ) • Minimize joint energy over all other variables. • Related to max-marginals as: µj = (1/z)*exp(-ψj) - Can be used to compute confidence as: σj = µj / Σaµa = exp(-ψi) / Σaexp(-ψa)

12. Inference in Graphical Models • Min-Marginals Energies(ψ) • Minimize joint energy over all other variables. • Related to max-marginals as: µj = (1/z)*exp(-ψj) - Can be used to compute confidence as: σj = µj / Σaµa = exp(-ψi) / Σaexp(-ψa) How to compute min-marginal energies using Graph Cuts?

13. Inference in Graphical Models

14. Inference in Graphical Models

15. Outline • Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results

16. Inference using Graph cuts • Given energy functions E(x|D), • Compute: arg min E(x|D) x

17. Inference using Graph cuts • Given energy functions E(x|D), • Compute: arg min E(x|D) x • Certain E(x|D) can be minimized using graph cuts exactly.

18. Inference using Graph cuts • Given energy functions E(x|D), • Compute: arg min E(x|D) x • Certain E(x|D) can be minimized using graph cuts exactly. • Class of energy function and graph contruction • - Binary random variables • - Submodular functions (Kolmogorov & Zabih, ECCV 2002) • - Multi-valued variables • Convex Pair-wise Terms (Ishikawa, PAMI 2003)

19. Inference using Graph cuts Graph Construction for Binary Random Variables EMRF(a1,a2) Source (1) a1 a2 Sink (0)

20. Inference using Graph cuts EMRF(a1,a2) =2a1 Source (1) 2 t-edges (unary terms) a1 a2 Sink (0)

21. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1 Source (1) 2 a1 a2 5 Sink (0)

22. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 Source (1) 2 9 a1 a2 5 4 Sink (0)

23. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 +2a1ā2 Source (1) 2 9 a1 a2 2 5 4 n-edges (pair-wise term) Sink (0)

24. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 1 a1 a2 2 5 4 Sink (0)

25. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 1 a1 a2 2 5 4 Sink (0)

26. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 Cost of st-cut = 11 1 a1 a2 a1 = 1 a2 = 1 2 5 4 EMRF(1,1) = 11 Sink (0)

27. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 Cost of st-cut = 8 1 a1 a2 a1 = 1 a2 = 0 2 5 4 EMRF(1,0) = 8 Sink (0)

28. Inference using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 Cost of st-cut = 8 1 a1 a2 a1 = 1 a2 = 0 2 5 4 MAP Solution EMRF(1,0) = 8 a1,map = 1 a2,map = 0 Sink (0)

29. Outline • Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results

30. Computing Min-marginals using Graph Cuts

31. Computing Min-marginals using Graph Cuts Instead of minimizing E(.), minimize a projection of E(.) where the value of latent variable xv is fixed to label j. All projections of a sub-modular function are sub-modular [Kolmogorov and Zabih, ECCV 2002]

32. Computing Min-marginals using Graph Cuts Instead of minimizing E(.), minimize a projection of E(.) where the value of latent variable xv is fixed to label j. Problem Solved? Not Really!

33. Computing Min-marginals using Graph Cuts • Computing a min-marginal requires computation of a st-cut. • Typical Segmentation problem • 640x480 image, 2 labels • Variables = 640x480 = 307200 • Number of Min-marginals = 307200x2 = 614400 • Time taken for 1 graph cut = .3 seconds • Total computation time = 614400x0.3= 184320 sec= 51.2 hours!!!

34. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 1 a1 a2 2 5 4 Sink (0)

35. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2+ 4ā2+ 2a1ā2 +ā1a2 a2= 1 Source (1) EMRF(a1,1) = 2a1 + 5ā1+ 9+ ā1 2 9 1 a1 a2 2 5 4 Sink (0)

36. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2+ 4ā2+ 2a1ā2 +ā1a2 a2= 1 Source (1) EMRF(a1,1) = 2a1 + 5ā1+ 9+ ā1 2 EMRF(a1,1) = 2a1 + 6ā1+ 9 9 a1 a2 6 Sink (0)

37. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 5 4 K ∞ Sink (0)

38. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ Sink (0)

39. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ Sink (0) K> Sum of Outgoing/Incoming edges

40. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ • Results in a small change in the graph. Sink (0)

41. Energy Projections and Graph Construction EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ • Results in a small change in the graph. Sink (0) Solve using Dynamic Graph Cuts 

42. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Energy function Graph 2 Projection of Energy function

43. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Energy function Graph 1 and 2 are similar Graph 2 Projection of Energy function

44. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Graph 2

45. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 Compute Max-flow Computationally Expensive Procedure Graph 2 Residual Graph 2 st-cut 2 Compute Max-flow

46. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 Compute Max-flow Re-parameterized Graph 2 Graph 2 Residual Graph 2 st-cut 2

47. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 same solution Re-parameterized Graph 2 Graph 2 Residual Graph 2 st-cut 2

48. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 similar edge weights Re-parameterized Graph 2 Graph 2 Residual Graph 2 st-cut 2

49. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 Compute Max-flow Re-parameterized Graph 2 Compute Max-flow Extremely Fast Operation Graph 2 Residual Graph 2 st-cut 2

50. Dynamic Graph Cuts - Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 300 msec Re-parameterized Graph 2 0.002 msec Extremely Fast Operation Graph 2 Residual Graph 2 st-cut 2