1 / 69

Discrete Optimization Lecture 2

Discrete Optimization Lecture 2. M. Pawan Kumar pawan.kumar@ecp.fr. Slides available online http:// cvc.centrale-ponts.fr /personnel/ pawan /. Outline. Chow-Liu Tree Minimum Spanning Tree Problem Kruskal’s Method Prim’s Method. Distribution.

zahina
Télécharger la présentation

Discrete Optimization Lecture 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete OptimizationLecture 2 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvc.centrale-ponts.fr/personnel/pawan/

  2. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  3. Distribution Estimate a joint probability P(x) from the data P(x) = P(x0,x1,…,xn-1) = P(x0) P(x1|x0) … P(xn-1|x1,…,xn-2) Images courtesy of Pedro Felzenszwalb

  4. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x3) PT(x4|x1) PT(x3|x0) PT(x2|x0) PT(x1|x0) PT(x0) Estimate PT(xa|xp(a)) = P(xa|xp(a))

  5. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x2) PT(x4|x2) PT(x3|x2) PT(x2|x0) PT(x1|x2) PT(x0) Which tree? Estimate PT(xa|xp(a)) = P(xa|xp(a))

  6. Kullback-Leibler Divergence KL(P1 || P2) = -Σx P1(x) log P2(x) +ΣxP1(x) log P1(x) Constant KL(P1 || P2) ≥ 0 KL(P1 || P1) = 0 Substitute P1 = P and P2 = PT. Minimize KL(P || PT)

  7. Estimating the Tree Structure min -Σx P(x) log PT(x)

  8. Estimating the Tree Structure min -Σx P(x) log ΠaPT(xa|xp(a))

  9. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa|xp(a))

  10. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))

  11. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a))P(xa) • PT(xp(a))P(xa)

  12. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σx P(x) Σalog P(xa)

  13. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σa ΣxaP(xa) log P(xa) Independent of the tree structure

  14. Estimating the Tree Structure min -Σa ΣxaΣxp(a)P(xa,xp(a)) log PT(xa,xp(a)) • PT(xp(a))P(xa) min -Σa l(a,p(a)) Mutual Information

  15. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  16. Undirected Connected Simple Graph v0 G = (V, E) 2 1 v1 v4 l: E R 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4

  17. Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 ET is a subset of E Graph T is a tree

  18. Weight of a Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 w(T) = Sum of the length of all edges in ET w(T) = 2+1-1-5+1-2 = -4

  19. Minimum Spanning Tree Problem v0 G = (V, E) 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 T* = argminT w(T) Find a tree with the minimum weight

  20. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  21. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Start with a forest where every vertex is a tree

  22. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  23. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  24. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 YES Add the edge to the forest Does this edge connect two different trees?

  25. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  26. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  27. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  28. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  29. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  30. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  31. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  32. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  33. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 NO Discard the edge Does this edge connect two different trees?

  34. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  35. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  36. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  37. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  38. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  39. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  40. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  41. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  42. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  43. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Spanning Tree !!! Algorithm terminates.

  44. Summary Given G = (V, E). Define T = (V, {}) and S = E. While T is not a spanning tree Select the minimum length edge e from S If e connects two different trees Add e to T End Remove e from S End

  45. Time Complexity O(mlog(m)) where m = |E|

  46. Proof Sketch Let ET(t) be the edges added by iteration t There exists an MST T containing all edges in ET(t). Trivial for t = 0. Assume it is true for t Edge e is added at iteration t+1. ET(t+1) = ET(t) U e If T does not contain e, adding e to T creates a cycle Remove the edge from the cycle that is not in ET(t+1) Resulting spanning tree T’ must have minimum weight

  47. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  48. Prim’s Method v0 v0 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 Initialize T = (VT, {}) where VT = {v0}

  49. Prim’s Method v0 v0 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 Choose (u*,v*) = min l(u,v), u  VT, v  V \ VT

  50. Prim’s Method v0 v0 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 Choose (u*,v*) = min l(u,v), u  VT, v  V \ VT

More Related