1 / 61

CPSC 311 Analysis of Algorithms

CPSC 311 Analysis of Algorithms. More Graph Algorithms Prof. Jennifer Welch Fall 2009. Minimum Spanning Tree. 16. 5. 4. 11. 12. 7. 3. 14. 9. 6. 2. 8. 10. 15. 17. 13. 18. find subset of edges that span all the nodes, create no cycle, and minimize sum of weights.

ddiaz
Télécharger la présentation

CPSC 311 Analysis of Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CPSC 311Analysis of Algorithms More Graph Algorithms Prof. Jennifer Welch Fall 2009 CPSC 311, Fall 2009

  2. Minimum Spanning Tree 16 5 4 11 12 7 3 14 9 6 2 8 10 15 17 13 18 find subset of edges that span all the nodes, create no cycle, and minimize sum of weights CPSC 311, Fall 2009

  3. Facts About MSTs • There can be many spanning trees of a graph • In fact, there can be many minimum spanning trees of a graph • But if every edge has a unique weight, then there is a unique MST CPSC 311, Fall 2009

  4. Uniqueness of MST • Suppose in contradiction there are 2 MSTs, M1 and M2. • Let e be edge with minimum weight that is one but not the other (say it is in M1). • If e is added to M2, a cycle is formed. • Let e' be an edge in the cycle that is not in M1 CPSC 311, Fall 2009

  5. Uniqueness of MST e: in M1 but not M2 M2: e': in M2 but not M1; wt is less than wt of e Replacing e with e' creates a new MST M3 whose weight is less than that of M2 contradiction CPSC 311, Fall 2009

  6. Generic MST Algorithm • input: weighted undirected graph G = (V,E,w) • T := empty set • while T is not yet a spanning tree of G • find an edge e in E s.t. T U {e} is a subgraph of some MST of G • add e to T • return T (as MST of G) CPSC 311, Fall 2009

  7. Kruskal's MST algorithm 16 5 4 11 12 7 3 14 9 6 2 8 10 15 17 13 18 consider the edges in increasing order of weight, add in an edge iff it does not cause a cycle CPSC 311, Fall 2009

  8. Kruskal's Algorithm as a Special Case of Generic Alg. • Consider edges in increasing order of weight • Add the next edge iff it doesn't cause a cycle • At any point, T is a forest (set of trees); eventually T is a single tree CPSC 311, Fall 2009

  9. Kruskal's Algorithm • input: G = (V,E,w) // weighted graph • T := Ø // subset of E • sort E by increasing weights • for each (u,v) in E in sorted order do • if T U {u,v} has no cycle then T := T U {(u,v)} • return (V,T) CPSC 311, Fall 2009

  10. Why is Kruskal's Greedy? • Algorithm manages a set of edges s.t. • these edges are a subset of some MST • At each iteration: • choose an edge so that the MST-subset property remains true • subproblem left is to do the same with the remaining edges • Always try to add cheapest available edge that will not violate the tree property • locally optimal choice CPSC 311, Fall 2009

  11. Correctness of Kruskal's Alg. • Let e1, e2, …, en-1 be sequence of edges chosen • Clearly they form a spanning tree • Suppose it is not minimum weight • Let eibe the edge where the algorithm goes wrong • {e1,…,ei-1} is part of some MST M • but {e1,…,ei} is not part of any MST CPSC 311, Fall 2009

  12. Correctness of Kruskal's Alg. M: ei, forms a cycle in M wt(e*) > wt(ei) replacing e* w/ eiforms a spanning tree with smaller weight than M, contradiction! e* : min wt. edge in cycle not in e1 to ei-1 white edges are part of MST M, which contains e1 to ei-1, but not ei CPSC 311, Fall 2009

  13. Note on Correctness Proof • Argument on previous slide works for case when every edge has a unique weight • Algorithm also works when edge weights are not necessarily correct • Modify proof on previous slide: contradiction is reached to assumption that ei is not part of any MST CPSC 311, Fall 2009

  14. Implementing Kruskal's Alg. • Sort edges by weight • efficient algorithms known • How to test quickly if adding in the next edge would cause a cycle? • use Disjoint Sets data structure… CPSC 311, Fall 2009

  15. Disjoint Sets in Kruskal's Alg. • Use Disjoint Sets data structure to test whether adding an edge to a set of edges would cause a cycle • Each set represents nodes that are connected using edges chosen so far • Initially put each node in a set by itself using Make-Set • When testing edge (u,v), call Find-Set to check if u and v are in the same set • if so, then a cycle would result if (u,v) is chosen • When edge (u,v) is chosen, call Union(u,v) CPSC 311, Fall 2009

  16. Kruskal's Algorithm #2 • input: G = (V,E,w) // weighted graph • T := Ø // subset of E • sort E by increasing weights • for each v in V do MakeSet(v) • for each (u,v) in E in sorted order do • a := FindSet(u) • b := FindSet(v) • if a ≠ b then • T := T U {(u,v)} • Union(a,b) • return (V,T) CPSC 311, Fall 2009

  17. Running Time of Kruskal's MST Algorithm • Sorting the edges takes O(E log E) time • The rest of the time is proportional to the total time taken by all the calls to MakeSet, FindSet, and Union. • Number of calls to MakeSet is |V| • If we unravel the for loop, we can see: • total number of calls to FindSet is O(E) • total number of calls to Union is O(V) • So number of operations is O(V+E), with |V| MakeSet ops. • If the Disjoint Sets data structure is implemented using union by rank and path compression, the time for all the Disjoint Sets operations is O((V+E) log*V) • Total time is O(E log E), since graph is connected, so V ≤ E + 1, and this equals O(E log V), since E is O(V2) and log V2 = 2log V CPSC 311, Fall 2009

  18. Another Greedy MST Alg. • Kruskal's algorithm maintains a forest that grows until it forms a spanning tree • Alternative idea is keep just one tree and grow it until it spans all the nodes • Prim's algorithm • At each iteration, choose the minimum weight outgoing edge to add • greedy! CPSC 311, Fall 2009

  19. Idea of Prim's Algorithm • Instead of growing the MST as possibly multiple trees that eventually all merge, grow the MST from a single node, so that there is only one tree at any point. • Also a special case of the generic algorithm: at each step, add the minimum weight edge that goes out from the tree constructed so far. CPSC 311, Fall 2009

  20. Prim's Algorithm • input: weighted undirected graph G = (V,e,w) • T := empty set • S := {any node in V} • while |T| < |V| - 1 do • let (u,v) be a min wt. outgoing edge (u in S, v not in S) • add (u,v) to T • add v to S • return (S,T) (as MST of G) CPSC 311, Fall 2009

  21. Prim's Algorithm Example 8 7 b c d 4 9 2 11 a i 4 e 14 7 6 8 10 h g f 1 2 CPSC 311, Fall 2009

  22. Correctness of Prim's Algorithm • Let Ti be the tree represented by (S,T) at the end of iteration i. • Show by induction on i that Ti is a subtree of some MST of G. • Basis: i = 0 (before first iteration). T0 contains just a single node, and thus is a subtree of every MST of G. CPSC 311, Fall 2009

  23. u v Correctness of Prim's Algorithm • Induction: Assume Ti is a subtree of some MST M. We must show Ti+1 is a subtree of some MST. • Let (u,v) be the edge added in iteration i+1. Case 1: (u,v) is in M. Then Ti+1 is also a subtree of M. Ti Ti+1 CPSC 311, Fall 2009

  24. Correctness of Prim's Algorithm Case 2: (u,v) is not in M. • There is a path P in M from u to v, since M spans G. • Let (x,y) be the first edge in P with one endpoint in Ti and the other not in Ti. y P x Ti u v CPSC 311, Fall 2009

  25. Correctness of Prim's Algorithm • Let M' = M - {(x,y)} U {(u,v)} • M' is also a spanning tree of G. • w(M') = w(M) - w(x,y) + w(u,v) ≤ w(M) since (u,v) is min wt outgoing edge • So M' is also an MST and Ti+1 is subtree M' y x Ti u v Ti+1 CPSC 311, Fall 2009

  26. Implementing Prim's Algorithm • How do we find minimum weight outgoing edge? • First cut: scan all adjacency lists at each iteration. • Results in O(VE) time. • Try to do better. CPSC 311, Fall 2009

  27. Implementing Prim's Algorithm • Idea: have each node not yet in the tree keep track of its best (cheapest) edge to the tree constructed so far. • To find min wt. outgoing edge, find minimum among these values • use a priority queue to store the best edge info (insert and extract-min operations) CPSC 311, Fall 2009

  28. u v w x Implementing Prim's Algorithm • When a node v is added to T, some other nodes might have their best edges affected, but only neighbors of v • add decrease-key operation to the priority queue v's best edge to Ti check if this edge is cheaper for w Ti Ti+1 x's best edge to Ti w's best edge to Ti CPSC 311, Fall 2009

  29. Details on Prim's Algorithm Associate with each node v two fields: • best-wt[v] : if v is not yet in the tree, then it holds the min. wt. of all edges from v to a node in the tree. Initially infinity. • best-node[v] : if v is not yet in the tree, then it holds the name of the node u in the tree s.t. w(v,u) is v's best-wt. Initially nil. CPSC 311, Fall 2009

  30. Details on Prim's Algorithm • input: G = (V,E,w) // initialization • initialize priority queue Q to contain all nodes, using best-wt values as keys • let v0 be any node in V • decrease-key(Q,v0,0) // last line means change best-wt[v0] to 0 and adjust Q accordingly CPSC 311, Fall 2009

  31. Details on Prim's Algorithm • while Q is not empty do • u := extract-min(Q) // node w/ smallest best-wt • if u is not v0 then add (u,best-node[u]) to T • for each neighbor v of u do • if v is in Q and w(u,v) < best-wt[v] then • best-node[v] := u • decrease-key(Q,v,w(u,v)) • return (V,T) // as MST of G CPSC 311, Fall 2009

  32. Running Time of Prim's Algorithm Depends on priority queue implementation. Let • Tins be time for insert • Tdec be time for decrease-key • Tex be time for extract-min Then we have • |V| inserts and one decrease-key in the initialization: O(VTins+Tdec) • |V| iterations of while • one extract-min per iteration: O(VTex) total CPSC 311, Fall 2009

  33. Running Time of Prim's Algorithm • Each iteration of while includes a for loop. • Number of iterations of for loop varies, depending on how many neighbors the current node has • Total number of iterations of for loop is O(E). • Each iteration of for loop: • one decrease key, so O(ETdec) total CPSC 311, Fall 2009

  34. Running Time of Prim's Algorithm • O(V(Tins + Tex) + ETdec) • If priority queue is implemented with a binary heap, then • Tins = Tex = Tdec = O(log V) • total time is O(E log V) • (Think about how to implement decrease-key in O(log V) time.) CPSC 311, Fall 2009

  35. Shortest Paths in a Graph • We already saw the Floyd-Warshall algorithm to compute all pairs of shortest paths • Let's review two important single-source shortest path algorithms: • Dijkstra's algorithm • Bellman-Ford algorithm CPSC 311, Fall 2009

  36. s t Single Source Shortest Path Problem • Given: directed or undirected graph G = (V,E,w) and source node s in V • Find: For each t in V, a path in G from s to t with minimum weight • Warning! Negative weights are a problem: 4 5 CPSC 311, Fall 2009

  37. Shortest Path Tree • Result of a SSSP algorithm can be viewed as a tree rooted at the source • Why not use breadth-first search? • Works fine if all weights are the same: • weight of each path is (a multiple of) the number of edges in the path • Doesn't work when weights are different CPSC 311, Fall 2009

  38. Dijkstra's SSSP Algorithm • Assumes all edge weights are nonnegative • Similar to Prim's MST algorithm • Start with source node s and iteratively construct a tree rooted at s • Each node keeps track of tree node that provides cheapest path from s (not just cheapest path from any tree node) • At each iteration, include the node whose cheapest path from s is the overall cheapest CPSC 311, Fall 2009

  39. 4 5 1 s 6 Prim's MST Prim's vs. Dijkstra's 4 5 1 s 6 Dijkstra's SSSP CPSC 311, Fall 2009

  40. Implementing Dijkstra's Alg. • How can each node u keep track of its best path from s? • Keep an estimate, d[u], of shortest path distance from s to u • Use d as a key in a priority queue • When u is added to the tree, check each of u's neighbors v to see if u provides v with a cheaper path from s: • compare d[v] to d[u] + w(u,v) CPSC 311, Fall 2009

  41. Dijkstra's Algorithm • input: G = (V,E,w) and source node s // initialization • d[s] := 0 • d[v] := infinity for all other nodes v • initialize priority queue Q to contain all nodes using d values as keys CPSC 311, Fall 2009

  42. Dijkstra's Algorithm • while Q is not empty do • u := extract-min(Q) • for each neighbor v of u do • if d[u] + w(u,v) < d[v] then • d[v] := d[u] + w(u,v) • decrease-key(Q,v,d[v]) • parent(v) := u CPSC 311, Fall 2009

  43. Dijkstra's Algorithm Example iteration 2 a b 8 12 4 c 10 9 6 3 2 d e 4 source is node a CPSC 311, Fall 2009

  44. Correctness of Dijkstra's Alg. • Let Ti be the tree constructed after i-th iteration of while loop: • nodes not in Q • edges indicated by parent variables • Show by induction on i that the path in Ti from s to u is a shortest path and has distance d[u], for all u in Ti. • Basis: i = 1. s is the only node in T1 and d[s] = 0. CPSC 311, Fall 2009

  45. Ti Ti-1 s x u Correctness of Dijkstra's Alg. • Induction: Assume Ti-1 is a correct shortest path tree and show for Ti. • Let u be the node added in iteration i. • Let x = parent(u). Need to show path in Ti from s to u is a shortest path, and has distance d[u] CPSC 311, Fall 2009

  46. P', another path from s to u a s x u b Correctness of Dijkstra's Alg P, path in Ti from s to u Ti-1 Ti (a,b) is first edge in P' that leaves Ti-1 (i.e., a is in Ti-1 but b is not) CPSC 311, Fall 2009

  47. Correctness of Dijkstra's Alg Let P1 be part of P' before (a,b). Let P2 be part of P' after (a,b). w(P') = w(P1) + w(a,b) + w(P2) ≥ w(P1) + w(a,b) (nonneg wts) ≥ (wt of path in Ti-1 from s to a) + w(a,b) (inductive hypothesis) ≥ (wt of path in Ti-1 from s to x) + w(x,u) (alg chose u in iteration i and d-values are accurate, by inductive hyp.) = w(P). So P is a shortest path, and d[u] is accurate after iteration i. Ti-1 P Ti s u x a b P' CPSC 311, Fall 2009

  48. Running Time of Dijstra's Alg. • initialization: insert each node once • O(V Tins) • O(V) iterations of while loop • one extract-min per iteration => O(V Tex) • for loop inside while loop has variable number of iterations… • For loop has O(E) iterations total • one decrease-key per iteration => O(E Tdec) • Total is O(V (Tins + Tex) + E Tdec) CPSC 311, Fall 2009

  49. Using Fancier Heap Implementations • O(V(Tins + Tex) + E Tdec) • If priority queue is implemented with a binary heap, then • Tins = Tex = Tdec = O(log V) • total time is O(E log V) • There are fancier implementations of the priority queue, such as Fibonacci heap: • Tins = O(1), Tex = O(log V), Tdec = O(1) (amortized) • total time is O(V log V + E) CPSC 311, Fall 2009

  50. Using Simpler Heap Implementations • O(V(Tins + Tex) + E Tdec) • If graph is dense, so that |E| = (V2), then it doesn't help to make Tins and Tex to be at most O(V). • Instead, focus on making Tdec be small, say constant. • Implement priority queue with an unsorted array: • Tins = O(1), Tex = O(V), Tdec = O(1) • total is O(V2) CPSC 311, Fall 2009

More Related