Minimum Spanning Trees • Definition • Two properties of MST’s • Prim and Kruskal’s Algorithm • Proofs of correctness • Boruvka’s algorithm • Verifying an MST • Randomized algorithm in linear time
Problem Definition • Input • Weighted, connected undirected graph G=(V,E) • Weight (length) function w on each edge e in E • Task • Compute a spanning tree of G of minimum total weight • Spanning tree • If there are n nodes in G, a spanning tree consists of n-1 edges such that no cycles are formed
1 4 8 A 1 B 4 C D 8 A B C D 7 Output: 6 2 9 3 7 6 2 5 9 3 E F G 5 E F G Example Input:
Two Properties of MST’s • Cycle Property: For any cycle C in a graph, the heaviest edge in C does not appear in the minimum spanning tree • Used to rule edges out • Cut Property: For any proper non-empty subset X of the vertices, the lightest edge with exactly one endpoint in X belongs to the minimum spanning forest • Used to rule edges in
1 4 8 A B C D 7 6 2 9 3 5 E F G 1 4 8 A B C D 7 6 2 9 3 5 E F G Cycle Property Illustration/Proof • Proof by contradiction: • Suppose T is an MST with such an edge e. • Derive a contradiction showing that T is not an MST
1 4 8 A B C D 7 6 2 9 3 5 E F G 1 4 8 A B C D 7 6 2 9 3 5 E F G Cut Property Illustration/Proof • Proof by contradiction: • Suppose T is an MST without such an edge e. • Derive a contradiction showing that T is not an MST
Three Classic Greedy Algorithms • Kruskal’s approach • Select the minimum weight edge that does not form a cycle • Prim’s approach • Choose an arbitrary start node v • At any point in time, we have connected component N containing v and other nodes V-N • Choose the minimum weight edge from N to V-N • Boruvka’s approach • Prim “in parallel”
1 4 8 A B C D 7 6 2 9 3 5 E F G Example • Illustrate the execution of Kruskal and Prim’s algorithms on the following input graph. Let D be the arbitrary start node for Prim’s algorithm.
1 4 8 A B C D 7 6 2 9 3 5 E F G Prim Implementation • Use a priority queue to organize nodes in V-N to facilitate finding closest node. • Extract-Min operation • Decrease-key operation • Describe how we could implement Prim using a priority queue.
Running Time of Prim • How many extract-min operations will we perform? • How many decrease-key operations will we perform? • How much time to build initial priority queue? • Given binary heap implementation, what is the running time? • Fibonacci heap: • Decrease-key drops to O(1) amortized time
1 4 8 A B C D 7 6 2 9 3 5 E F G Kruskal Implementation • Kruskal’s Algorithm • Adding edge (u,v) to the current set of edges T forms a cycle if and only if u and v are in the same connected component.
1 4 8 A B C D 7 6 2 9 3 5 E F G Disjoint Set Data Stucture (Ch 21) • Given a universe U of objects • Maintain a collection of sets Si such that • Unioni Si = U • Si intersect Sj is empty • Find-set(x): Returns set Si that contains x • Merge(Si, Sj): Returns new set Sk = Si union Sj • Describe how we can implement cycle detection with this data structure.
Running Time of Kruskal • How many merges will we perform? • How many Find-set operations will we perform? • Each can be implemented in amortized a(V) time where a is a very slow growing function. • What other operations do we need to implement? • Overall running time?
Proofs of Correctness • Why do we know each edge that is added in Kruskal’s algorithm is part of an MST? • Why do we know that each edge added in Prim’s algorithm is part of an MST?
Boruvka’s Algorithm • Prim “in parallel” • Boruvka Step: • We have a graph of vertices • For each v in V, select the minimum weight edge connected to v • Update • Contract all selected edges, replacing each connected component by a single vertex • Delete loops, and keep only the lowest weight edge in a multi-edge • Run Boruvka steps until we have a single node
1 9 8 A B C D 7 6 2 4 3 5 E F G 1 9 8 A B C D 7 7 A’ B’ 6 2 4 3 5 E F G Boruvka Step Illustration
Boruvka’s Algorithm Analysis • Correctness: • How can we verify that each edge we add is part of an MST? • Running time: • What is the running time of a Boruvka Step? • How many Boruvka steps must we implement in the worst case?
Verification Problem • Input • Weighted, connected undirected graph G=(V,E) • Weight (length) function w on each edge e in E • A spanning tree T of G • Task • Answer yes/no if T is a minimum spanning tree for G • Two key concepts • Decision problem: problem with yes/no answer • Verification: Is it easier to verify an answer as correct as compared to generating an answer?
1 4 8 A B C D 7 6 2 9 3 5 E F G Key idea: T(u,v) • Suppose we have a tree T • For each edge (u,v) not in T, let T(u,v) be the heaviest edge on the (u,v) path in T • If w(u,v) > w(T(u,v)), then (u,v) should not be in T. • Consider edge (B,F) with weight 7. • T(B,F) = (C,G) which has weight 6. • (B,F) is appropriately not in T as w(B,F) > w(C,G).
Verification Algorithm • For each edge (u,v) not in T, find T(u,v). • Compare w (u,v) to w(T(u,v)) • If all are ok, then return yes, else return no. • Running time? • O(E) time to perform comparisons • Sophisticated techniques to find all the T(u,v) in O(E) time.
Randomized Algorithm • Run Baruvka’s Step 2 times • If there were originally n nodes, how many in reduced problem • Random Sampling • Make a smaller subgraph by choosing each edge with probability ½ • Recursively compute minimum spanning tree (or perhaps forest) F on this reduced graph • Use verification algorithm to eliminate any edges (u,v) not in F whose weight is more than weight of F(u,v). • Apply algorithm recursively to the remaining graph to compute a spanning tree T’ • Knit together tree from step 1 with F’ to form spanning tree
Contracted Graph G1 List of edges E1 Step 2: Construct G2 by sampling ½ edges Step 2: Recursively Construct T2 for graph G2 Step 2: Use T2 to identify edges E2 that cannot be in MST for G1 Step 3: Recursively Construct T3 for graph G3 Step 3: Return T3 melded with edges E1 as final tree T4 Visualization Graph Go Step 1: Run 2 Baruvka Steps: Graph G3 = G1 – E2 Tree T4 = T3 union E1
Contracted Graph G1 List of edges E1 Step 3: Recursively Construct T3 for graph G3 Step 3: Return T3 melded with edges E1 as final tree T4 Analysis Intuition Graph Go Step 1: Run 2 Baruvka Steps: G2 has at most ½ the edges of G0 Step 2: Construct G2 by sampling ½ edges Graph G3 = G1 – E2 Step 2: Recursively Construct T2 for graph G2 Step 2: Use T2 to identify edges E2 that cannot be in MST for G1 G3 edges bounded by ½ nodes of G0 Tree T4 = T3 union E1
