Randomization in Graph Optimization Problems

1. Randomization in Graph Optimization Problems David Karger MIT http://theory.lcs.mit.edu/~karger

2. Randomized Algorithms • Flip coins to decide what to do next • Avoid hard work of making “right” choice • Often faster and simpler than deterministic algorithms • Different from average-case analysis • Input is worst case • Algorithm adds randomness

3. Methods • Random selection • if most candidate choices “good”, then a random choice is probably good • Random sampling • generate a small random subproblem • solve, extrapolate to whole problem • Monte Carlo simulation • simulations estimate event likelihoods • Randomized Rounding for approximation

4. Cuts in Graphs • Focus on undirected graphs • A cut is a vertex partition • Value is number (or total weight) of crossing edges

5. Optimization with Cuts • Cut values determine solution of many graph optimization problems: • min-cut / max-flow • multicommodity flow (sort-of) • bisection / separator • network reliability • network design Randomization helps solve these problems

6. Presentation Assumption • For entire presentation, we consider unweighted graphs (all edges have weight/capacity one) • All results apply unchanged to arbitrarily weighted graphs • Integer weights = parallel edges • Rational weights scale to integers • Analysis unaffected • Some implementation details

7. Basic Probability • Conditional probability • Pr[AÇB] = Pr[A] × Pr[B | A] • Independent events multiply: • Pr[AÇB] = Pr[A] × Pr[B] • Union Bound • Pr[X ÈY] £ Pr[X] + Pr[Y] • Linearity of expectation: • E[X + Y] = E[X] + E[Y]

8. Random Selection forMinimum Cuts Random choices are good when problems are rare

9. Minimum Cut • Smallest cut of graph • Cheapest way to separate into 2 parts • Various applications: • network reliability (small cuts are weakest) • subtour elimination constraints for TSP • separation oracle for network design • Nots-t min-cut

10. Max-flow/Min-cut • s-t flow: edge-disjoint packing of s-t paths • s-t cut: a cut separating s and t • [FF]: s-t max-flow = s-t min-cut • max-flow saturates all s-t min-cuts • most efficient way to find s-t min-cuts • [GH]: min-cut is “all-pairs” s-t min-cut • find using n flow computations

11. Flow Algorithms • Push-relabel [GT]: • push “excess” around graph till it’s gone • max-flow in O*(mn)(note: O* hides logs) • recent O*(m3/2)[GR] • min-cut in O*(mn2) --- “harder” than flow • Pipelining [HO]: • save push/relabel data between flows • min-cut in O*(mn) --- “as easy” as flow

12. Contraction • Find edge that doesn’t cross min-cut • Contract (merge) endpoints to 1 vertex

13. Contraction Algorithm • Repeat n - 2 times: • find non-min-cut edge • contract it (keep parallel edges) • Each contraction decrements #vertices • At end, 2 vertices left • unique cut • corresponds to min-cut of starting graph

15. Picking an Edge • Must contract non-min-cut edges • [NI]: O(m)time algorithm to pick edge • n contractions: O(mn)time for min-cut • slightly faster than flows If only could find edge faster…. Idea: min-cut edges are few

16. Randomize Repeat until 2 vertices remain pick a random edge contract it (keep fingers crossed)

17. Analysis I • Min-cut is small---few edges • Suppose graph has min-cut c • Then minimum degree at least c • Thus at least nc/2 edges • Random edge is probably safe Pr[min-cut edge] £c/(nc/2) = 2/n (easy generalization to capacitated case)

18. Analysis II • Algorithm succeeds if never accidentally contracts min-cut edge • Contracts #vertices from n down to 2 • When k vertices, chance of error is 2/k • thus, chance of being right is 1-2/k • Pr[always right] is product of probabilities of being right each time

19. Analysis III …not too good!

20. Repetition • Repetition amplifies success probability • basic failure probability 1 - 2/n2 • so repeat 7n2 times

21. How fast? • Easy to perform 1 trial in O(m) time • just use array of edges, no data structures • But need n2 trials: O(mn2) time • Simpler than flows, but slower

22. An improvement [KS] • When k vertices, error probability 2/k • big when k small • Idea: once k small, change algorithm • algorithm needs to be safer • but can afford to be slower • Amplify by repetition! • Repeat base algorithm many times

23. (50-50 chance of avoiding min-cut) Recursive Algorithm Algorithm RCA (G, n ) {G has n vertices} repeat twice randomly contract G to n/2½ vertices RCA(G,n/21/2)

24. Main Theorem • On any capacitated, undirected graph, Algorithm RCA • runs in O*(n2) time with simple structures • finds min-cut with probability ³ 1/log n • Thus, O(log n) repetitions suffice to find the minimum cut (failure probability 10-6) in O(n2 log2n) time.

25. Proof Outline • Graph has O(n2)(capacitated) edges • So O(n2) work to contract, then two subproblems of size n/2½ • T(n) = 2 T(n/2½) + O(n2) = O(n2log n) • Algorithm fails if both iterations fail • Iteration succeeds if contractions and recursion succeed • P(n)=1 - [1 - ½ P(n/2½)]2 = W (1 / log n)

26. Failure Modes • Monte Carlo algorithms always run fast and probably give you the right answer • Las Vegas algorithms probably run fast and always give you the right answer • To make a Monte Carlo algorithm Las Vegas, need a way to check answer • repeat till answer is right • No fast min-cut check known (flow slow!)

27. How do we verify a minimum cut?

28. Enumerating Cuts The probabilistic method, backwards

29. Cut Counting • Original CA finds any given min-cut with probability at least 2/n(n-1) • Only one cut found • Disjoint events, so probabilities add • So at most n(n-1)/2 min-cuts • probabilities would sum to more than one • Tight • Cycle has exactly this many min-cuts

30. Enumeration • RCA as stated has constant probability of finding any given min-cut • If run O(log n) times, probability of missing a min-cut drops to 1/n3 • But only n2 min-cuts • So, probability miss any at most 1/n • So, with probability 1-1/n, find all • O(n2 log3n) time

31. Generalization • If G has min-cut c, cut £ac is a-mincut • Lemma: contraction algorithm finds any given a-mincut with probability W (n-2a) • Proof: just add a factor to basic analysis • Corollary: O(n2a)a-mincuts • Corollary: Can find all in O*(n2a) time • Just change contraction factor in RCA

32. Summary • A simple fast min-cut algorithm • Random selection avoids rare problems • Generalization to near-minimum cuts • Bound on number of small cuts • Probabilistic method, backwards

33. Random Sampling

34. Random Sampling • General tool for faster algorithms: • pick a small, representative sample • analyze it quickly (small) • extrapolate to original (representative) • Speed-accuracy tradeoff • smaller sample means less time • but also less accuracy

35. A Polling Problem • Population of size m • Subset of c red members • Goal: estimate c • Naïve method: check whole population • Faster method: sampling • Choose random subset of population • Use relative frequency in sample as estimate for frequency in population

36. Analysis: Chernoff Bound • Random variables XiÎ [0,1] • Sum X = åXi • Bound deviation from expectation Pr[|X-E[X]|³e E[X]] < exp(-e2E[X]/ 4) • “Probably, X Î(1±e) E[X]” • If E[X] ³ 4(ln n)/e2, “tight concentration” • Deviation by eprobability < 1 / n

37. Application to Polling • Choose each member with probability p • Let X be total number of reds seen • Then E[X]=pc • So estimate ĉ by X/p • Note ĉ accurate to within 1±e iff X is within 1±e of expectation: ĉ = X/p Î(1±e) E[X]/p = (1±e) c

38. Analysis • Let Xi=1 if ith red item chosen, else 0 • Then X= åXi • Chernoff Bound applies • Pr[deviation bye] <exp(-e2pc/ 4) • < 1/nifpc > 4(log n)/e2 • Pretty tight • if pc < 1,likely no red samples • so no meaningful estimate

39. Sampling for Min-Cuts

40. Min-cut Duality • [Edmonds]: min-cut=max tree packing • convert to directed graph • “source” vertex s (doesn’t matter which) • spanning trees directed away from s • [Gabow] “augmenting trees” • add a tree in O*(m) time • min-cut c (via max packing) in O*(mc) • great if m and c are small…

41. Example min-cut 2 2 directed spanning trees directed min-cut 2

42. Random Sampling • Gabow’s algorithm great if m, c small • Random sampling • reduces m, c • scales cut values (in expectation) • if pick half the edges, get half of each cut • So find tree packings, cuts in samples Problem: maybe some large deviations

43. Sampling Theorem • Given graph G, build a sample G(p)by including each edge with probability p • Cut of value vin Ghas expected value pv in G(p) • Definition:“constant” r= 8 (ln n)/e2 • Theorem: With high probability, all exponentially many cuts in G(r/ c) have (1±e) times their expected values.

44. A Simple Application • [Gabow] packs trees in O*(mc) time • Build G(r/ c) • minimum expected cut r • by theorem, min-cut probably near r • find min-cut in O*(r m) timeusing [Gabow] • corresponds to near-min-cut in G • Result: (1+e)times min-cut in O*(m/e2) time

45. Proof of Sampling: Idea • Chernoff bound says probability of large deviation in cut value is small • Problem: exponentially many cuts. Perhaps some deviate a great deal • Solution: showed few small cuts • only small cuts likely to deviate much • but few, so Chernoff bound applies

46. Proof of Sampling • Sampled with probability r/c, • a cut of value ac has mean ar • [Chernoff]: deviates from expected size by more than e with probability at most n-3a • At most n2acuts have value ac • Pr[any cut of value ac deviates] = O(n-a) • Sum over alla ³ 1

47. Las Vegas Algorithms Finding Good Certificates

48. Approximate Tree Packing • Break edges into c /r random groups • Each looks like a sample at rate r/ c • O*( rm / c) edges • each has min expected cut r • so theorem says min-cut (1 – e) r • So each has a packing of size (1 – e) r • [Gabow] finds in time O*(r2m/c) per group • so overall time is (c/r ) × O*(r2m/c) = O*(rm)

49. Las Vegas Algorithm • Packing algorithm is Monte Carlo • Previously found approximate cut (faster) • If close, each “certifies” other • Cut exceeds optimum cut • Packing below optimum cut • If not, re-run both • Result: Las Vegas, expected time O*(rm)

50. Exact Algorithm • Randomly partition edges in two groups • each like a ½ -sample: e =O*(c-½) • Recursively pack trees in each half • c/2 - O*(c½) trees • Merge packings • gives packing of size c - O*(c½) • augment to maximum packing: O*(mc½) • T(m,c)=2T(m/2,c/2)+O*(mc½) = O*(mc½)