SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Combinatorial Optimization and Computer Vision

trygg avatar
trygg
125 diapositives · 1.4K Views · Aug 20, 2014

This paper explores the multifaceted challenges of unsupervised object segmentation in computer vision, specifically targeting the segmentation of cows in images. Traditional methods often require extensive user intervention, such as specifying seed pixels or bounding boxes, to achieve satisfactory results. This study delves into the variabilities of shape and appearance, self-occlusion, and the potential for developing more autonomous techniques. By leveraging maximum a posteriori (MAP) and energy minimization methods using graph cuts, the research aims to produce more reliable and efficient solutions for the segmentation process.

Combinatorial Optimization and Computer Vision - slide 1
01 / 125Combinatorial Optimization and Computer Vision
1 / 17

Télécharger la présentation

Combinatorial Optimization and Computer Vision
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Combinatorial Optimization andComputer Vision Philip Torr

  2. Story • How an attempt to solve one problem lead into many different areas of computer vision and some interesting results.

  3. Aim Object Category Model • Given an image, to segment the object Segmentation Cow Image Segmented Cow • Segmentation should (ideally) be • shaped like the object e.g. cow-like • obtained efficiently in an unsupervised manner • able to handle self-occlusion

  4. Challenges Shape Variability Appearance Variability Self Occlusion

  5. Motivation • Current methods require user intervention • Object and background seed pixels (Boykov and Jolly, ICCV 01) • Bounding Box of object (Rother et al. SIGGRAPH 04) Object Seed Pixels Cow Image

  6. Motivation • Current methods require user intervention • Object and background seed pixels (Boykov and Jolly, ICCV 01) • Bounding Box of object (Rother et al. SIGGRAPH 04) Object Seed Pixels Background Seed Pixels Cow Image

  7. Motivation • Current methods require user intervention • Object and background seed pixels (Boykov and Jolly, ICCV 01) • Bounding Box of object (Rother et al. SIGGRAPH 04) Segmented Image

  8. Motivation • Current methods require user intervention • Object and background seed pixels (Boykov and Jolly, ICCV 01) • Bounding Box of object (Rother et al. SIGGRAPH 04) Object Seed Pixels Background Seed Pixels Cow Image

  9. Motivation • Current methods require user intervention • Object and background seed pixels (Boykov and Jolly, ICCV 01) • Bounding Box of object (Rother et al. SIGGRAPH 04) Segmented Image

  10. Motivation • Problem • Manually intensive • Segmentation is not guaranteed to be ‘object-like’ Non Object-like Segmentation

  11. MRF for Image Segmentation Boykov and Jolly [ICCV 2001] EnergyMRF = Unary likelihood Contrast Term Pair-wise terms (Potts Model) Maximum-a-posteriori (MAP) solution x*= arg min E(x) x Data (D) Unary likelihood Pair-wise Terms MAP Solution

  12. GraphCut for Inference Source Foreground Cut Image Background Sink Cut:A collection of edges which separates the Source from the Sink MinCut:The cut with minimum weight (sum of edge weights) Solution:Global optimum (MinCut) in polynomial time

  13. Energy Minimization using Graph cuts Graph Construction for Boolean Random Variables EMRF(a1,a2) Source (0) a1 a2 Sink (1)

  14. Energy Minimization using Graph cuts EMRF(a1,a2) =2a1 Source (0) 2 t-edges (unary terms) a1 a2 Sink (1)

  15. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1 Source (0) 2 a1 a2 5 Sink (1)

  16. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 Source (0) 2 9 a1 a2 5 4 Sink (1)

  17. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 +2a1ā2 Source (0) 2 9 a1 a2 2 5 4 n-edges (pair-wise term) Sink (1)

  18. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 1 a1 a2 2 5 4 Sink (1)

  19. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 1 a1 a2 2 5 4 Sink (1)

  20. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 Cost of st-cut = 11 1 a1 a2 a1 = 1 a2 = 1 2 5 4 EMRF(1,1) = 11 Sink (1)

  21. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 Cost of st-cut = 8 1 a1 a2 a1 = 1 a2 = 0 2 5 4 EMRF(1,0) = 8 Sink (1)

  22. Computing the st-mincut from Max-flow algorithms Source (0) • The Max-flow Problem • Edge capacity and flow balance constraints 2 9 • Notation • Residual capacity • (edge capacity – current flow) 1 a1 a2 2 5 4 • Simple Augmenting Path based Algorithms • Repeatedly find augmenting paths and push flow. • Saturated edges constitute the st-mincut. • [Ford-Fulkerson Theorem] Sink (1)

  23. Minimum s-t cuts algorithms • Augmenting paths [Ford & Fulkerson, 1962] • Push-relabel [Goldberg-Tarjan, 1986]

  24. “source” “sink” T S A graph with two terminals “Augmenting Paths” • Find a path from S to T along non-saturated edges • Increase flow along this path until some edge saturates

  25. “source” “sink” T S A graph with two terminals “Augmenting Paths” • Find a path from S to T along non-saturated edges • Increase flow along this path until some edge saturates • Find next path… • Increase flow…

  26. “source” “sink” T S A graph with two terminals  MIN CUT “Augmenting Paths” • Find a path from S to T along non-saturated edges • Increase flow along this path until some edge saturates Iterate until … all paths from S to T have at least one saturated edge MAX FLOW

  27. MRF, Graphical Model • Probability for a labellingconsists of • Likelihood Unary potential based on colour of pixel • Prior which favours same labels for neighbours (pairwise potentials) mx m(labels) Prior Ψxy(mx,my) my Unary Potential Φx(D|mx) x y D(pixels) Image Plane

  28. Example Cow Image Object Seed Pixels Background Seed Pixels Φx(D|obj) x … x … Φx(D|bkg) Ψxy(mx,my) y … y … … … … … Prior Likelihood Ratio (Colour)

  29. Example Cow Image Object Seed Pixels Background Seed Pixels Pair-wise Terms Likelihood Ratio (Colour)

  30. Contrast-Dependent MRF • Probability of labelling in addition has • Contrast term which favours boundaries to lie on image edges mx m(labels) my x Contrast Term Φ(D|mx,my) y D(pixels) Image Plane

  31. Example Cow Image Object Seed Pixels Background Seed Pixels Φx(D|obj) x … x … Φx(D|bkg) Ψxy(mx,my)+ Φ(D|mx,my) y … y … … … … … Pair-wise Term Likelihood Ratio (Colour)

  32. Example Cow Image Object Seed Pixels Background Seed Pixels Prior + Contrast Likelihood Ratio (Colour)

  33. Object Graphical Model • Probability of labelling in addition has • Unary potential which depend on distance from Θ (shape parameter) Θ (shape parameter) Unary Potential Φx(mx|Θ) mx m(labels) my Object Category Specific MRF x y D(pixels) Image Plane

  34. Example Cow Image Object Seed Pixels Background Seed Pixels ShapePriorΘ Prior + Contrast Distance from Θ

  35. Example Cow Image Object Seed Pixels Background Seed Pixels ShapePriorΘ Prior + Contrast Likelihood + Distance from Θ

  36. Example Cow Image Object Seed Pixels Background Seed Pixels ShapePriorΘ Prior + Contrast Likelihood + Distance from Θ

  37. Thought • We can imagine rather than using user input to define histograms we use object detection.

  38. Shape Model • BMVC 2004

  39. Yuille, ‘91 • Brunelli & Poggio, ‘93 • Lades, v.d. Malsburg et al. ‘93 • Cootes, Lanitis, Taylor et al. ‘95 • Amit & Geman, ‘95, ‘99 • Perona et al. ‘95, ‘96, ’98, ‘00 Pictorial Structure Fischler & Elschlager, 1973

  40. Layered Pictorial Structures (LPS) • Generative model • Composition of parts + spatial layout Layer 2 Spatial Layout (Pairwise Configuration) Layer 1 Parts in Layer 2 can occlude parts in Layer 1

  41. Layered Pictorial Structures (LPS) Cow Instance Layer 2 Transformations Θ1 P(Θ1) = 0.9 Layer 1

  42. Layered Pictorial Structures (LPS) Cow Instance Layer 2 Transformations Θ2 P(Θ2) = 0.8 Layer 1

  43. Layered Pictorial Structures (LPS) Unlikely Instance Layer 2 Transformations Θ3 P(Θ3) = 0.01 Layer 1

  44. How to learn LPS • From video via motion segmentation see Kumar Torr and Zisserman ICCV 2005. • Graph cut based method.

  45. Examples

  46. LPS for Detection • Learning • Learnt automatically using a set of examples • Detection • Matches LPS to image using Loopy Belief Propagation • Localizes object parts

  47. Detection • Like a proposal process.

  48. Pictorial Structures (PS) Fischler and Eschlager. 1973 PS = 2D Parts + Configuration Aim: Learn pictorial structures in an unsupervised manner Layered Pictorial Structures (LPS) Parts + Configuration + Relative depth • Identify parts • Learn configuration • Learn relative depth of parts

  49. P2 (x,y,,) P1 P3 MRF Image Motivation Matching Pictorial Structures - Felzenszwalb et al - 2001 Outline Texture Part likelihood Spatial Prior

  50. YES NO 2 1 P2 (x,y,,) P1 P3 MRF Image Motivation Matching Pictorial Structures - Felzenszwalb et al - 2001 • Unary potentials are negative log likelihoods Valid pairwise configuration Potts Model