Submodular Function Minimization

1. Submodular Function Minimization Collette Coullard IMA and Northwestern University November 25, 2002

2. Outline • Introduction to Submodular Function Minimization (SFM) • Definitions • Applications- Why is SFM interesting? • History • Separation-optimization equivalence • Developments • Recent developments • Iwata, Fleischer, Fugishige algorithm • Lex Schrijver’s algorithm • Schrijver’s Algorithm • Conclusions

3. subsets of E E A,B E A B  f(A) + f(B)  f(AB) + f(AB) Equivalently: SR E, e E E R S e  f(S {e}) – f(S)  f(R  {e}) – f(R) marginal value of e wrt S (cost) marginal value of e wrt R (cost) Submodular Function Definitions f2Eℝ is submodular if

4. Example: Number of Different Colors e a marginal value of g g d k 0 b i f({a,b,d,e,k})= 5 < c 1 h f({e,k})= 2 j E f(E)=5 f({a,b,d,e,k,g})= 5 f({e,k,g})= 3

5. Example: Number of Connections (Edges)) marginal value of g e a g d k 3 f’({a,b,d,e,k})= 3 b i < c 2 h f’({e,k})= 0 j E f’(E)= 11 f’({a,b,d,e,k,g})= 6 f’({e,k,g})= 2

6. marginal value of g Example: - Number of Connections (Edges)) e a g d f’({a,b,d,e,k,g})= -6 k -3 f’({a,b,d,e,k})= -3 b i < f’({e,k,g})= -2 c -2 h f’({e,k})= 0 j E f’(E)= -11

7. marginal value of g Example: Number of Colors - Number of Edges: f’’ = f + f’ e a g d f’’({a,b,d,e,k,g})= 5-6=-1 k -3 f’’({a,b,d,e,k})= 5-3=2 b i < f’’({e,k,g})= 3-2=1 c -1 h f’’({e,k})= 2-0=2 j E f’’(E)= 5 -11= -6

8. G = digraph with node set V A A A  V\{s,t} s f(A) + f(B)f(AB) + f(AB) t f(A) := number of arcs leaving node set A  {s} B A B Example: Directed Cuts

9. DC DC DC S R Benefit term Risk term Net cost marginal risk-marginal benefit wrt S marginal risk-marginal benefit wrt R Example: Benefit and Risk Tradeoff E: customers i

10. Example: Find minimum (cardinality or capacity) (s,t)-cut (Note: ƒnot monotone) s t Example: Find minimum net value (colors-edges) node set Submodular Function Minimization:Find S E with minimum f(S)

11. Example: Find minimum (cardinality or capacity) (s,t)-cut DC (Note: ƒnot monotone) s t Example: Find minimum net cost (max net profit) customer set Submodular Function Minimization:Find S E with minimum f(S)

12. x* x0 P P Optimizationand Separation Given c and P, find x*P such that cx* cx, for every x P. Given x0 and P, either conclude x0P or return valid inequality axb with ax0 > b. Theorem(Grötschel,Lovasz, Schrijver 1981) The optimization problem for Pcan be solved in polynomial time if and only if the separation problem can be solved for Pin polynomial time.

13. Greedy Algorithm (Edmonds): ce1 ce2 x*e1 = f({e1}) x*e2= f({e1,e2})-f({e1}) x* x0 xe1 P P xe2 P: x(A)  f(A), A  E x(E) = f(E) Optimization and Separation Given x0 and P, either conclude x0P or return valid inequality x(A)  f(A), with x0(A) > f(A) or min{ f(A) - x0(A) : A  E } submodular submodular function minimization

14. Challenge(Grötschel,Lovasz, Schrijver 1981) Since Optimizationand Separation are polynomially equivalent, and the optimization problem for Phas a strongly polynomial combinatorial algorithm (GA), there should be a strongly polynomial combinatorial algorithm for the separation problem for P. There should be a strongly polynomial algorithm for Submodular Function Minimization (SFM).

15. Developments • Submodular function and matroids • Rado (1942) • Ingleton (1959) • Submodular functions and optimization: Greedy algorithm maximizes linear functions over polymatroids • Edmonds (1970) • Polynomial solvability of submodular function minimization (SFM) via ellipsoid algorithm (separation-optimization equivalence) • Grötschel, Lovász, Schrijver (1981,1988) • Partial order of polymatroid extreme points +finite combinatorial algorithm for SFM • Bixby, Cunningham, Topkis (1985) • Pseudopolynomial combinatorial algorithm for SFM • Cunningham (1985) • Strongly polynomial algorithm for symmetric SFM • Queyranne (1998) • Strongly polynomial combinatorial algorithms for SFM • Iwata, Fleischer, Fugishige (1999), Iwata, Fleischer (2000), Fugishige, Iwata(2001) • Schrijver (1999)

16. a collection of subsets of E (called independent sets) satisfying r is a submodularfunction on E Submodular function and matroids a finite set A matroid M=(E,I) I 1: empty set is independent I 2: subsets of independent sets are independent I 3: All maximal independent subsets of A  E have same size, called rank of A, r(A)

17. submodular Matroid Examples Forest Matroid of a Graph • G=(V,E) • M=(E,I) edge sets with no cycles Rank functionon 2E:

18. submodular Matroid Examples • G=(V,E) Cutset Matroid of a Graph • M=(E,I) edge sets with no cutsets Rank functionon 2E:

19. a submodularfunction on E Matroid Partition • G=(V,E) • Can E be partitioned • into at most k forests? • (k independent sets) • This is a special case • of SFM.

20. a submodularfunction on E Matroid Intersection • Given matroids M1=(E,I1) and M2=(E,I2), • max {|I|: II1 I2} = min {f1(A) + f2(E\A) : A  E} • Special case of SFM. Maximum matching in bipartite graph is a special case. • G=(UV,E)

21. Theorem. (Edmonds) This system is TDI, which implies D always has Integral optimal solutions.  • D models the submodular function minimization problem • Feasible solutions to P provide lower bounds onf(S*) • Optimal solutions to P provide certificate of optimality for S* Mathematical Programming Formulationof SFM P: D:

22. Schrijver’s Algorithm for SFM • Maintains a feasible solution x to P’: • Maintains feasibility of x by keeping: b is an extreme point of P’ • Iteratively improves:

23. choice of A submodularity of f Extreme Points b is an extreme point of P’: Feasibility proof for b : Suppose smallest last j Then Lower ideals of the ordering are tight sets for b.

24. Show examples in Excel.

25. Optimality Condition: We have an xRE and an R E with x is feasible for P’ R has all the negative components and no positive components of x Ris the set of initial elements in the order, for each bi

26. Optimality condition examples 2 4 1

27. + + - - + R + - Auxiliary Network G=(E,A) case 1: no path from + to -.

28. Auxiliary Network G=(E,A) + + - s - + t - + case 2: There is a path from + to -. Use last arc (s,t) to “improve x”.

29. x = 1b1 + 2b2 + ... + kbk; 1 , ... , k  0; 1 + ... + k = 1 s -1 : . . +1 = 1 ( ’1b’1+ ... + ’kb’l) + 2b1 + ... + kbk x’ = x+ 1 s -1 : . +1 t = ’1b’1 + ’2b’2 + ... + ’kb’l; b1 +  t Improving x: An Iteration, algebraically

30. s -1 : . +1 = ’1b’1 + ’2b’2 + ... + ’kb’l; b1 +  t Schrijver’s clever idea

32. x = 1b1 + 2b2 + ... + kbk; 1 , ... , k  0; 1 + ... + k = 1 s -1 : . +1 = 1 ( ’1b’1+ ... + ’kb’l) + 2b1 + ... + kbk x+ 1 t x’ = ... s -1 : . +1 = ’1b’1 + ’2b’2 + ... + ’kb’l; b1 +  t An Iteration, geometrically

33. Complexity of O(n6)   n n n n

34. Conclusions and future work • Submodular Function Minimization (SFM) generalizes several combinatorial optimization problems. • Schrijver’s elegant algorithm • known basic ideas from ‘80s • new idea on step direction • clever tie-breaking for O(n6) complexity bound • C++ implementations in the works • Schrijver Algorithm • IFF Algorithms • possibly Queyranne’s column-generation idea • others?