Geometry and Expansion: A survey of recent results

1. Geometry and Expansion: A survey of recent results Sanjeev Arora Princeton ( touches upon: S. A., Satish Rao, Umesh Vazirani, STOC’04; S. A., Elad Hazan, and Satyen Kale, FOCS’04; S. A., James Lee, and Assaf Naor, STOC’05 + papers that are not mine)

2. | E(S, Sc)| (G) = min S S S µ V |S| |S| < |V|/2 | E(S, Sc)| c(G) = min S µ V |S| c |V| < |S| < |V|/2 Sparsest Cut / Edge Expansion G = (V, E) c- balanced separator Both NP-hard

3. Why these problems are important • Analysis of random walks, PRAM simulation, packet routing, clustering, VLSI layout etc. • Underlie many divide-and-conquer graph algorithms (surveyed by Shmoys’95) • Discrete analog of isoperimetry; useful in Riemannian geometry (via 2nd eigenvalue of Laplacian (Cheeger’70) • Graph-theoretic parameters of inherent interest (cf. Lipton-Tarjan planar separator theorem)

4. The three main characters Expansion Isoperimetry (continuous analog of expansion) Geometry (and geometric embeddings of finite metric spaces) Outcome: New plog n –approximations for various NP-hard problems; Derived using geometric insights, & which led to new geometry thms.

5. 3) Embeddings of finite metric spaces into l1 • Geometric approach; more general result (but still O(log n) approximation) Previous approximation algorithms for expansion problems • Eigenvalue approaches (Cheeger’70, Alon’85, Alon-Milman’85)Only yield factor n approximation. 2c(G) ¸ (G) ¸ c(G)2 /2 2) O(log n) -approximationvia LP (multicommodity flows) (Leighton-Rao’88) • Approximate max-flow mincut theorems • Region-growing argument (Linial, London, Rabinovich’94, AR’94)

6. log n log n New results of [ARV’04] • O( ) -approximation to sparsest cut and conductance • O( )-pseudoapproximation to c-balanced separator (algorithm outputs a c’-balanced separator, c’ < c) • Existence of expander flows in every graph (approximate certificates of expansion) Disparate approaches from previous slide get “unified”

7. Outline: • Graph partitioning problems: intro and history • New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani] • Uses of “S. T.” in geometric embeddings • Introduction to expander flows and O(n2) time algorithms • Outline of proof of “S. T.” • Open problems Next: Semidefinite relaxations for c-balanced separator (and how to round the solution)

8. S | E(S, Sc)| c(G) = min S µ V |S| c |V| < |S| < |V|/2 Semidefinite relaxation for c-balanced separator |vi –vj|2/4 =1 |vi –vj|2 =0 +1 S -1 “cut semimetric” Find unit vectors in <n Assign {+1, -1} to v1, v2, …, vn to minimize (i, j) 2 E |vi –vj|2/4 Subject to i < j |vi –vj|2/4 ¸ c(1-c)n2 Triangle inequality |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k

9. Vi Vj Vk Unit l22 space Unit vectors v1, v2,… vn2<d |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k non obtuse ! Example: Hypercube {-1, 1}k |u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1 In fact, l2 and l1 are subcases of l22

10. log n 1 p log n Structure Theorem for l22 spaces [ARV’04] Subsets S and T are -separated if for every vi2 S, vj2 T |vi –vj|2¸ <d G = Graph in which (i,j) is an edge iff |vi –vj|2· ¸  Thm: If i< j |vi –vj|2 = (n2) then 9S, T of size (n) that are  -separated for  = ( 1 ) Equiv: G is an “expander” )·

11. log n ) |E(R, Rc)| · SDPopt / · O( SDPopt) Main thm ) O( )-approximation log n v1, v2,…, vn2<d is optimum SDP soln; SDPopt = (i, j) 2 E |vi –vj|2 S, T :  –separated sets of size (n) Do BFS from S until you hit T. Take the level of the BFS tree with the fewest edges and output the cut (R, Rc) defined by this level  (i, j) 2 E |vi –vj|2¸ |E(R, Rc)| £

12. log n Other new -approximation algorithms • MIN-2-CNF deletion and several graph deletion problems. [Agarwal, Charikar, Makarychev, Makarychev’05] • MIN-LINEAR ARRANGEMENT [Charikar, Karloff, Rao’05] • General SPARSEST CUT [A., Lee, Naor ’05] • Min-ratioVERTEX SEPARATORS and Balanced VERTEX SEPARATORS[ Feige, Hajiaghayi, Lee, ’05] Example: Structure Theorem (Agarwal, Charikar, Makarychev2 ‘05) d : directed version of l22 metric; w: weight function on the nodes G = (V, E): any graph on the nodes. S There exists a subset S that contains 1/10 of the total weight and such that e leaves S d(e) is at Most p log n £e 2 E d(e). All use the Structure Theorem (+ other ideas) (Useful in rounding SDP for MIN-2CNF-DELETION.)

13. Outline: • Graph partitioning problems: intro and history • New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani] • Geometric embeddings of metric spaces • Introduction to expander flows and O(n2) time algorithms • Outline of proof of “S. T.” • Open problems

14. <k(with l2 norm) Finite metric space (X, d) f(x) y f d(x,y) x f(y) distortion of f is minimum C>1 such that d( x, y) · |f(x ) – f( y)|2·C d( x, y) 8 x, y Thm (Bourgain’85): For every n-point metric space, a map exists with distortion O(log n) [LLR’94]: Efficient algorithm to find the map; Proof that O(log n) cannot be improved in general Qs: Improve O(log n) for X = l22 (say) or l1 ?

15. Embeddings and Cuts (LLR’94, AR’94) Recall: Cut semi-metric Fact: Metric (X, d) embeds isometricallyin l1 iff it can be written as a positive combination of cut semimetrics 1 0 Embedding l22 into l1 gives a way to produce cuts from SDP solution

16. Status report of this area Best upperbound Best lowerbound Disproves Goemans-Linial conjecture log n [Bourgain’85] Uses fourier techniques developed for PCPs! log0.75 n [Chawla,Gupta,Racke ’04] Exactly the integrality gap of SDP for general SPARSEST CUT [LLR’94, AR’94] log0.5 n log log n [A., Lee, Naor’04] Note: l2µ l1µ l22

17. x Ai Embedding Upperbounds:Frechet’s recipe to embed metric space (X, d) into Rk Pick k suitable subsets A1, A2, …, Ak of X Map x 2 X to (d(x, A1), d(x, A2), … , d(x, Ak)) Note: d(x, A1) – d(y, A1) · d(x, y) Why S.T. useful: If Sobtained from S.T., then in the mapping x ! d(x, S), “many” x’s (namely, all those in T) map far from 0. In recent embeddings, Ai’s are chosen using S.T.and “Measured descent” idea of [Krauthgamer, Lee, Naor, and Mendel’04]

18. Embedding lowerbounds (Khot-Vishnoi’05) Explicit unit- l22 space (X, d) that requires distortion log log log n into l1 Main observation: Need good handle on cut structure of X Use hypercube as building block ! Cut ´ Boolean Function Number of cut edges = average sensitivity (Fourier analysis a la KKL, Friedgut, Hastad, Bourgain etc. ) isoperimetric theorems) [Khot-Naor]: Lowerbounds for embedding earth-mover & edit metrics into l1

19. Outline: • Graph partitioning problems: intro and history • New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani] • Outline of proof of “S. T.” • Uses of “S. T.” in geometric embeddings • Introduction to expander flows and O(n2) time algorithms • Open problems

20. log n S Our Thm: If G has expansion , then a D-regular expander flow exists in it where D=  Expander flows: Motivation “Expander” G = (V, E) Idea: Embed a D-regular (weighted) graph such that 8 S w(S, Sc) = (D |S|) (*) S (certifies expansion = (D) ) Weighted Graph w satisfies (*) iff L(w) = (1) [Cheeger] Cf. Jerrum-Sinclair, Leighton-Rao(embed a complete graph) Can be found in O(n2) time (A., Hazan, Kale ’04)

21. Example of expander flow n-cycle Take any 3-regular expander on n nodes Put a weight of 1/3n on each edge Embed this into the n-cycle Routing of edges does not exceed any capacity ) expansion =(1/n)

22. T S • Outline: • Graph partitioning problems: intro and history • New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani] • Uses of “S. T.” in geometric embeddings • Introduction to expander flows and O(n2) time algorithms • Outline of proof of “S. T.” • Open problems Outline of proof of S. T. (Algorithm to produce  -separatedsets S, T, of size (n) )

23. 0.01 d “Stretched pair”: vi, vj such that |vi –vj|2· and | h vi –vj, u i | ¸ 0.01 d Algorithm to produce two  –separated sets <d Easy: Su and Tu likely to have size (n) u Tu Delete any vi2 Su, vj2 Tu s.t. |vi –vj|2 < . (till no such pair remains) Su If Su, Tu still have size (n), output them Main difficulty: Show that whp only o(n) points get deleted Obs: Deleted pairs are stretched and they form a matching.

24. -t2 /2 e 1 1 d d = O( 1 ) Stretched pair: |vi –vj|2 < ; |<vi –vj, u>| > 0.01 d  Naïve analysis of random projection fails v <d u <u, v> ?? standard deviations E[# of stretched pairs] = n2 exp(-) À n

25. Vi 0.01 d Proof by contradiction: Suppose matching of (n) size exists with probability (1)… ….stretched pairs are almost everywhere you look! Vj u Ball (vi , ) Idea: Put stretched pairs together; derive very improbable event

26. Vi Vj Vk s s s s Walks in unit l22 space Unit vectors v1, v2,… vn2<d |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k Angles are non obtuse Taking r steps of length s only takes you squared distance rs2 (i.e. distance r s)

27. r steps of length s ) squared distance rs2 (distance r s) s s 0.01 <vfinal –v0, u> ¸ r 0.01 0.01 d d d s s r Projection = £ standard deviation  Proof by contradiction (contd.) Claim: 9walk on stretched edges VERY UNLIKELY IF r large enough) Walk impossible (CONTRADICTION) Stretched pair: |vi –vj|2 < ; |<vi –vj, u> ¸ 0.01 d    …. u How to produce walk: delicate argument; measure concentration |vfinal –v0|· r 

28. OPEN PROBLEMS • Better approximation factor than O( )? (For general SPARSEST CUT, log log n “lowerbound” ) • Better distortion bound for embedding l22 into l1?( upperbound v/s loglog n lowerbound.) • Remove need for solving SDPs (i.e., design combinatorial algorithms) (similar to one for SPARSEST CUT from [A., Hazan, Kale] ) • O(m) time algorithm for SPARSEST CUT instead of O(n2)? (not known even for Leighton-Rao’88 O(log n) approximation) • Other applications of expander flows? (Useful in some geometric results [Naor, Rabani, Sinclair’04])

29. Looking forward to more progress… Thanks !

30. ) (D) ·(G) ·O(D ) log n log n New Result (A., Hazan, Kale;FOCS’04) O(n2) time algorithm that given any graph G finds for some D >0 • a D-regular expander flow • a cut of expansion O( D ) Ingredients: Approximate eigenvalue computations; Approximate flow computations (Garg-Konemann; Fleischer) Random sampling (Benczur-Karger + some more) Idea: Define a zero-sum game whose optimum solution is an expander flow; solve approximately using Freund-Schapire approximate solver.

31. Expander flows: LP view · 1 LP feasible )¸(D) · D Thm [ARV]:90 s.t. the LP is feasible with D = /√log n G

32. Open problems (circa April’04) O(n2) time; [A., Hazan, Kale] • Better running time/combinatorial algorithm? • Improve approximation ratio to O(1); better rounding??(our conjectures may be useful…) • Extend result to other expansion-like problems (multicut, general sparsest cut; MIN-2CNF deletion) • Resolve conjecture about embeddability of l22 into l1; of l1 into l2 • Any applications of expander flows? Integrality gap is (log n) [Charikar] log3/4 n distortion; [Chawla,Gupta, Racke] Yes [Naor,Sinclair,Rabani] Better embeddings of lp into lq [Lee]

33. Various new results O(n2) time combinatorial algorithm for sparsest cut (does not use semidefinite programs) [A., Hazan, Kale’04] New results about embeddings: (i) lp into lq[J. Lee’04] (ii) l22 and l1 into l2[CGR’04] (approx for general sparsest cut) Clearer explanation of expander flows and their connection to embeddings [NRS’04]

34. log n Formal statement : 90 >0 s.t. foll. LP is feasible for d = (G) Pij = paths whose endpoints are i, j 8i jp 2 Pij fp = d (degree) 8e 2 E p 3 e fp· 1 (capacity) 8S µ V i 2 S j 2 Scp 2 Pij fp¸0 d |S| (demand graph is an expander) fp¸ 0 8 paths p in G

35. A concrete conjecture (prove or refute) G = (V, E);  = (G) For every distribution on n/3 –balanced cuts {zS} (i.e., SzS =1) there exist (n) disjoint pairs (i1, j1), (i2, j2), ….. such that for each k, • distance between ik, jk in G is O(1/ ) • ik, jk are across (1) fraction of cuts in {zS} (i.e., S: i 2 S, j 2 Sc zS = (1) ) Conjecture ) existence of d-regular expander flows for d = 

36. log n log n

37. Example of l22 space: hypercube {-1, 1}k |u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1 In fact, every l1 space is also l22 Conjecture (Goemans, Linial): Every l22 space is l1 up to distortion O(1)

38. 1 0 0 1 1 Semidefinite LP Relaxations for c-balanced separator Min (i, j) 2 E Xij 0 · Xij· 1 Motivation: Every cut (S, Sc) defines a (semi) metric Xij2 {0,1} Xij + Xj k¸ Xik  i< j Xij¸ c(1-c)n2 There exist unit vectors v1, v2, …, vn2<n such that Xij = |vi - vj|2 /4

39. Semidefinite relaxation (contd) Min (i, j) 2 E |vi –vj|2/4 |vi|2 = 1 |vi –vj|2 + |vj –vk|2¸ |vi –vk|28 i, j, k i < j |vi –vj|2¸ 4c(1-c)n2 Unit l22 space Many other NP-hard problems have similar relaxations.

40. If any vi2 Su and vj2 Tu satisfy |vi –vj|2·, delete them and repeat until no such vi, vj remain 0.01 d “Stretched pair”: vi, vj such that |vi –vj|2· and | h vi –vj, u i | ¸ 0.01 d Algorithm to produce two  –separated sets <d Check if Su and Tu have size (n) u Tu Su If Su, Tu still have size (n), output them Main difficulty: Show that whp only o(n) points get deleted Obs: Deleted pairs are stretched and they form a matching.

41. T S Next 10-12 min: Proof-sketch of Structure Thm ( algorithm to produce  -separated S, T of size (n);  = 1/ )

42. “Stretched pair”: vi, vj such that |vi –vj|2· and | h vi –vj, u i | ¸ 0.01 d O( 1 ) £ standard deviation  = exp( - ) log n ) PrU [ vi, vj get stretched] = exp( - 1 )  E[# of stretched pairs] = O( n2 ) £ exp(- ) logn “Matching is of size o(n) whp” : naive argument fails

43. |vfinal - vi| < r  = O( r ) x standard dev. | <vfinal – vi, u>| ¸ 0.01r 0.01 0.01 0.01 r d d d  d Generating a contradiction: the walk on stretched pairs Contradiction if r is large enough! Vj vfinal    Vi r steps u

44. Reason: Isoperimetric inequality for spheres  Measure concentration (P. Levy, Gromov etc.) <d A : measurable set with (A) ¸ 1/4 A : points with distance · to A A (A) ¸ 1 – exp(-2 d) A

45. Expander flows (approximate certificates of expansion)