Create Presentation
Download Presentation

Download Presentation
## Measuring Uncertainty in Graph Cut Solutions

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Measuring Uncertainty in Graph Cut Solutions**Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University**st-mincut**Most Probable Solution s t Labelling problem Objective Graph Cut No uncertainty measure associated with the solution Belief or Confidence**Outline**• Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results**Outline**• Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results**Inference in Graphical Models**x: Set of latent variables D: Observed Data x*: Most Probable (MAP) Solution Pr: Joint Posterior Probability x* = arg maxPr(x|D) x E(x|D) = -log Pr(x|D) + constant x* = arg minE(x|D) x**Inference in Graphical Models**MAP Solution arg max Pr(A,B) {A=1, B=0} B 0 1 2 0 Joint Distribution A 1 Pr(A,B) 2**Inference in Graphical Models**• Marginal • Sum the joint probability over all other variables. B 0 1 2 0 A P(A=1) = 0.6 1 2**Inference in Graphical Models**• Max- Marginals (µ) • Maximum joint probability over all other variables. B 0 1 2 0 A µA,1 = 0.35 1 2**Inference in Graphical Models**• Confidence or Belief (σ) • Normalized max-marginals σA,1 = µA,1 / Σx µA,x = 0.35/ 0.58 = 0.603 B 0 1 2 µA,0 = 0.2 0 A µA,1 = 0.35 1 2 µA,2 = 0.03**Inference in Graphical Models**• Min-Marginals Energies(ψ) • Minimize joint energy over all other variables. • Related to max-marginals as: µj = (1/z)*exp(-ψj)**Inference in Graphical Models**• Min-Marginals Energies(ψ) • Minimize joint energy over all other variables. • Related to max-marginals as: µj = (1/z)*exp(-ψj) - Can be used to compute confidence as: σj = µj / Σaµa = exp(-ψi) / Σaexp(-ψa)**Inference in Graphical Models**• Min-Marginals Energies(ψ) • Minimize joint energy over all other variables. • Related to max-marginals as: µj = (1/z)*exp(-ψj) - Can be used to compute confidence as: σj = µj / Σaµa = exp(-ψi) / Σaexp(-ψa) How to compute min-marginal energies using Graph Cuts?**Outline**• Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results**Inference using Graph cuts**• Given energy functions E(x|D), • Compute: arg min E(x|D) x**Inference using Graph cuts**• Given energy functions E(x|D), • Compute: arg min E(x|D) x • Certain E(x|D) can be minimized using graph cuts exactly.**Inference using Graph cuts**• Given energy functions E(x|D), • Compute: arg min E(x|D) x • Certain E(x|D) can be minimized using graph cuts exactly. • Class of energy function and graph contruction • - Binary random variables • - Submodular functions (Kolmogorov & Zabih, ECCV 2002) • - Multi-valued variables • Convex Pair-wise Terms (Ishikawa, PAMI 2003)**Inference using Graph cuts**Graph Construction for Binary Random Variables EMRF(a1,a2) Source (1) a1 a2 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) =2a1 Source (1) 2 t-edges (unary terms) a1 a2 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1 Source (1) 2 a1 a2 5 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 Source (1) 2 9 a1 a2 5 4 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 +2a1ā2 Source (1) 2 9 a1 a2 2 5 4 n-edges (pair-wise term) Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 1 a1 a2 2 5 4 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 1 a1 a2 2 5 4 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 Cost of st-cut = 11 1 a1 a2 a1 = 1 a2 = 1 2 5 4 EMRF(1,1) = 11 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 Cost of st-cut = 8 1 a1 a2 a1 = 1 a2 = 0 2 5 4 EMRF(1,0) = 8 Sink (0)**Inference using Graph cuts**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 Cost of st-cut = 8 1 a1 a2 a1 = 1 a2 = 0 2 5 4 MAP Solution EMRF(1,0) = 8 a1,map = 1 a2,map = 0 Sink (0)**Outline**• Inference in Graphical Models • Inference using Graph Cuts • Computing Min-marginals using Graph Cuts • Flow Potentials and Min-marginals • Results**Computing Min-marginals using Graph Cuts**Instead of minimizing E(.), minimize a projection of E(.) where the value of latent variable xv is fixed to label j. All projections of a sub-modular function are sub-modular [Kolmogorov and Zabih, ECCV 2002]**Computing Min-marginals using Graph Cuts**Instead of minimizing E(.), minimize a projection of E(.) where the value of latent variable xv is fixed to label j. Problem Solved? Not Really!**Computing Min-marginals using Graph Cuts**• Computing a min-marginal requires computation of a st-cut. • Typical Segmentation problem • 640x480 image, 2 labels • Variables = 640x480 = 307200 • Number of Min-marginals = 307200x2 = 614400 • Time taken for 1 graph cut = .3 seconds • Total computation time = 614400x0.3= 184320 sec= 51.2 hours!!!**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (1) 2 9 1 a1 a2 2 5 4 Sink (0)**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2+ 4ā2+ 2a1ā2 +ā1a2 a2= 1 Source (1) EMRF(a1,1) = 2a1 + 5ā1+ 9+ ā1 2 9 1 a1 a2 2 5 4 Sink (0)**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2+ 4ā2+ 2a1ā2 +ā1a2 a2= 1 Source (1) EMRF(a1,1) = 2a1 + 5ā1+ 9+ ā1 2 EMRF(a1,1) = 2a1 + 6ā1+ 9 9 a1 a2 6 Sink (0)**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 5 4 K ∞ Sink (0)**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ Sink (0)**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ Sink (0) K> Sum of Outgoing/Incoming edges**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ • Results in a small change in the graph. Sink (0)**Energy Projections and Graph Construction**EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 + Kā2 Alternative Construction Source (1) A high unary term (t-edge) can be used to constrain the solution of the energy to be the solution of the energy projection. 2 9 1 a1 a2 2 • The minimum value of the energy projection can be calculated by using the same graph. 5 4 K ∞ • Results in a small change in the graph. Sink (0) Solve using Dynamic Graph Cuts **Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Energy function Graph 2 Projection of Energy function**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Energy function Graph 1 and 2 are similar Graph 2 Projection of Energy function**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Graph 2**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 Compute Max-flow Computationally Expensive Procedure Graph 2 Residual Graph 2 st-cut 2 Compute Max-flow**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 Compute Max-flow Re-parameterized Graph 2 Graph 2 Residual Graph 2 st-cut 2**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 same solution Re-parameterized Graph 2 Graph 2 Residual Graph 2 st-cut 2**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 similar edge weights Re-parameterized Graph 2 Graph 2 Residual Graph 2 st-cut 2**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 Compute Max-flow Re-parameterized Graph 2 Compute Max-flow Extremely Fast Operation Graph 2 Residual Graph 2 st-cut 2**Dynamic Graph Cuts**- Kohli and Torr [ICCV 2005] Graph 1 Residual Graph 1 st-cut 1 300 msec Re-parameterized Graph 2 0.002 msec Extremely Fast Operation Graph 2 Residual Graph 2 st-cut 2