Download
discrete optimization lecture 2 part 1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Discrete Optimization Lecture 2 – Part 1 PowerPoint Presentation
Download Presentation
Discrete Optimization Lecture 2 – Part 1

Discrete Optimization Lecture 2 – Part 1

170 Vues Download Presentation
Télécharger la présentation

Discrete Optimization Lecture 2 – Part 1

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Discrete OptimizationLecture 2 – Part 1 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvn.ecp.fr/personnel/pawan/

  2. Time Complexity of Djikstra’s Set d(v) = ∞ for all v  V. Set U = V. Set d(s) = 0. Set u = s. While u is not equal to t Choose u = argminvU d(v) For each v such that (u,v)  A d(v) = min { d(v), d(u) + l(u,v)} End Set U = U \ {u} End

  3. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  4. Distribution Estimate a joint probability P(x) from the data P(x) = P(x0,x1,…,xn-1) = P(x0) P(x1|x0) … P(xn-1|x1,…,xn-2) Images courtesy of Pedro Felzenszwalb

  5. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x3) PT(x4|x1) PT(x3|x0) PT(x2|x0) PT(x1|x0) PT(x0) Estimate PT(xa|xp(a)) = P(xa|xp(a))

  6. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x2) PT(x4|x2) PT(x3|x2) PT(x2|x0) PT(x1|x2) PT(x0) Which tree? Estimate PT(xa|xp(a)) = P(xa|xp(a))

  7. Kullback-Leibler Divergence KL(P1 || P2) = -Σx P1(x) log P2(x) +ΣxP1(x) log P1(x) Constant KL(P1 || P2) ≥ 0 KL(P1 || P1) = 0 Substitute P1 = P and P2 = PT. Minimize KL(P || PT)

  8. Estimating the Tree Structure min -Σx P(x) log PT(x)

  9. Estimating the Tree Structure min -Σx P(x) log ΠaPT(xa|xp(a))

  10. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa|xp(a))

  11. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))

  12. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a))P(xa) • PT(xp(a))P(xa)

  13. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σx P(x) Σalog P(xa)

  14. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σa ΣxaP(xa) log P(xa) Independent of the tree structure

  15. Estimating the Tree Structure min -Σa ΣxaΣxp(a)P(xa,xp(a)) log PT(xa,xp(a)) • PT(xp(a))P(xa) min -Σa l(a,p(a)) Mutual Information

  16. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  17. Undirected Connected Simple Graph v0 G = (V, E) 2 1 v1 v4 l: E R 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4

  18. Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 ET is a subset of E Graph T is a tree

  19. Weight of a Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 w(T) = Sum of the length of all edges in ET w(T) = 2+1-1-5+1-2 = -4

  20. Minimum Spanning Tree Problem v0 G = (V, E) 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 T* = argminT w(T) Find a tree with the minimum weight

  21. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  22. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Start with a forest where every vertex is a tree

  23. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  24. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  25. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 YES Add the edge to the forest Does this edge connect two different trees?

  26. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  27. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  28. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  29. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  30. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  31. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  32. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  33. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  34. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 NO Discard the edge Does this edge connect two different trees?

  35. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  36. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  37. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  38. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  39. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  40. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  41. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  42. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  43. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  44. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Spanning Tree !!! Algorithm terminates.

  45. Summary Given G = (V, E). Define T = (V, {}) and S = E. While T is not a spanning tree Select the minimum length edge e from S If e connects two different trees Add e to T End Remove e from S End

  46. Time Complexity O(mlog(m)) where m = |E|

  47. Proof Sketch Let ET(t) be the edges added by iteration t There exists an MST T containing all edges in ET(t). Trivial for t = 0. Assume it is true for t Edge e is added at iteration t+1. ET(t+1) = ET(t) U e If T does not contain e, adding e to T creates a cycle Remove the edge from the cycle that is not in ET(t+1) Resulting spanning tree T’ must have minimum weight

  48. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  49. Prim’s Method v0 v0 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 Initialize T = (VT, {}) where VT = {v0}

  50. Prim’s Method v0 v0 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 Choose (u*,v*) = min l(u,v), u  VT, v  V \ VT