Multiplicative Bounds for Metric Labeling

1. Multiplicative Boundsfor Metric Labeling M. Pawan Kumar ÉcoleCentrale Paris Joint work with Phil Torr, Daphne Koller

2. Metric Labeling Variables V= { V1, V2, …, Vn}

3. Metric Labeling Variables V= { V1, V2, …, Vn}

4. Metric Labeling wabd(f(a),f(b)) θb(f(b)) wab ≥ 0 θa(f(a)) d is metric Va Vb minf E(f) + Σ(a,b)wabd(f(a),f(b)) = Σaθa(f(a)) Labels L= { l1, l2, …, lh} Variables V= { V1, V2, …, Vn} Labeling f: { 1, 2, …, n}  {1, 2, …, h}

5. Metric Labeling Va Vb minf E(f) + Σ(a,b)wabd(f(a),f(b)) = Σaθa(f(a)) NP hard Low-level vision applications

6. Approximate Algorithms • Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001 • Murphy et al. Loopy Belief Propagation: An Empirical Study, UAI, 1999 • Winn et al. Variational Message Passing, JMLR, 2005 • Yedidiaet al. Generalized Belief Propagation, NIPS, 2001 • Besag. On the Statistical Analysis of Dirty Pictures, JRSS, 1986 • Boykov et al. Fast Approximate Energy Minimization via Graph Cuts, PAMI, 2001 • Komodakis et al. Fast, Approximately Optimal Solutions for Single and Dynamic • MRFs, CVPR, 2007 • Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010 • Chekuri et al. Approximation Algorithms for Metric Labeling, SODA, 2001 • Goemanset al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995 • Muramatsuet al. A New SOCP Relaxation for Max-Cut, JORJ, 2003 • Ravikumar et al. QP Relaxations for Metric Labeling, ICML, 2006 • Alahariet al. Dynamic Hybrid Algorithms for MAP Inference, PAMI 2010 • Kohliet al. On Partial Optimality in MultilabelMRFs, ICML, 2008 • Rotheret al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007 . . .

7. Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves

8. Integer Linear Program Minimize a linear function over a set of feasible solutions Indicator xa(i)  {0,1} for each variable Va and label li Number of facets grows exponentially in problem size

9. Linear Programming Relaxation Indicator xa(i)  {0,1} for each variable Va and label li Schlesinger, 1976; Chekuri et al., 2001; Wainwright et al., 2003

10. Linear Programming Relaxation Indicator xa(i)  [0,1] for each variable Va and label li Schlesinger, 1976; Chekuri et al., 2001; Wainwright et al., 2003

11. Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves

12. Move-Making Algorithms Space of All Labelings f

13. Expansion Algorithm Variables take label lα or retain current label Boykov, Veksler and Zabih, 2001 Slide courtesy PushmeetKohli

14. Expansion Algorithm Variables take label lα or retain current label Tree Ground House Status: Initialize with Tree Expand Ground Expand House Expand Sky Sky Boykov, Veksler and Zabih, 2001 Slide courtesy PushmeetKohli

15. Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves

16. Multiplicative Bounds f*: Optimal Labeling f: Estimated Labeling Σaθa(f(a)) + Σ(a,b)sabd(f(a),f(b)) ≥ Σaθa(f*(a)) + Σ(a,b)sabd(f*(a),f*(b))

17. Multiplicative Bounds f*: Optimal Labeling f: Estimated Labeling Σaθa(f(a)) + Σ(a,b)sabd(f(a),f(b)) ≤ B Σaθa(f*(a)) + Σ(a,b)sabd(f*(a),f*(b))

18. Multiplicative Bounds M = ratio of maximum and minimum distance

19. Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves • Complete Rounding • Interval Rounding • Hierarchical Rounding

20. Complete Rounding Treat xa(i)  [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤ixa(j) r ya(2) ya(i) ya(k) 0 ya(1) ya(h) = 1 Generate a random number r  (0,1] Assign the label next to r

21. Complete Move Va Vb θab(i,k) = sabd(i,k) NP-hard

22. Complete Move Va Vb d’(i,k) ≥ d(i,k) d’ is submodular θab(i,k) = sabd’(i,k)

23. Complete Move Va Vb d’(i,k) ≥ d(i,k) d’ is submodular θab(i,k) = sabd’(i,k)

24. Complete Move New problem can be solved using minimum cut Same multiplicative bound as complete rounding Multiplicative bound is tight

25. Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves • Complete Rounding • Interval Rounding • Hierarchical Rounding

26. Interval Rounding Treat xa(i)  [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤ixa(j) ya(2) ya(i) ya(k) 0 ya(1) ya(h) = 1 Choose an interval of length h’

27. Interval Rounding Treat xa(i)  [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤ixa(j) r ya(i) ya(k) REPEAT Choose an interval of length h’ Generate a random number r  (0,1] Assign the label next to r if it is within the interval

28. Interval Move Choose an interval of length h’ Va Vb θab(i,k) = sabd(i,k)

29. Interval Move Choose an interval of length h’ Add the current labels Va Vb θab(i,k) = sabd(i,k)

30. Interval Move Choose an interval of length h’ Add the current labels d’(i,k) ≥ d(i,k) d’ is submodular Solve to update labels Va Vb Repeat until convergence θab(i,k) = sabd’(i,k)

31. Interval Move Each problem can be solved using minimum cut Same multiplicative bound as interval rounding Multiplicative bound is tight Kumar and Torr, NIPS 2008

32. Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves • Complete Rounding • Interval Rounding • Hierarchical Rounding

33. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Hierarchical clustering of labels (e.g. r-HST metrics)

34. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels L1, L2 or L3 Move down the hierarchy until the leaf level

35. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels l1, l2 or l3

36. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels l4, l5 or l6

37. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels l7, l8 or l9

38. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Hierarchical clustering of labels (e.g. r-HST metrics)

39. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Obtain labeling f1 restricted to labels {l1,l2,l3}

40. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Obtain labeling f2 restricted to labels {l4,l5,l6}

41. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Obtain labeling f3 restricted to labels {l7,l8,l9}

42. Hierarchical Move L1 L2 L3 f3(a) f3(b) f2(a) f2(b) f1(a) f1(b) Va Vb Move up the hierarchy until we reach the root

43. Hierarchical Move Each problem can be solved using minimum cut Same multiplicative bound as hierarchical rounding Multiplicative bound is tight Kumar and Koller, UAI 2009

44. Conclusion M = ratio of maximum and minimum distance

45. Open Problems • Moves for general rounding schemes • Higher-order energy functions • Better comparison criterion

46. Questions? http://www.centrale-ponts.fr/personnel/pawan