Data Structures Graphs

1. Data StructuresGraphs

2. Han Luke Leia Graph… ADT? • Not quite an ADT…operations not clear • A formalism for representing relationships between objects Graph G = (V,E) • Set of vertices:V ={v1,v2,…,vn} • Set of edges:E = {e1,e2,…,em} where each ei connects twovertices (vi1,vi2) V = {Han, Leia, Luke} E = {(Luke, Leia), (Han, Leia), (Leia, Han)}

3. Examples of Graphs • The web • Vertices are webpages • Each edge is a link from one page to another • Call graph of a program • Vertices are subroutines • Edges are calls and returns • Social networks • Vertices are people • Edges connect friends

4. Han Luke Leia Han Luke Leia Graph Definitions In directed graphs, edges have a direction: In undirected graphs, they don’t (are two-way): v is adjacent to u if (u,v)  E

5. Weighted Graphs Each edge has an associated weight or cost. 20 Clinton Mukilteo 30 Kingston Edmonds 35 Bainbridge Seattle 60 Bremerton

6. Paths and Cycles • A path is a list of vertices {v1, v2, …, vn} such that (vi, vi+1)  E for all 0  i < n. • A cycle is a path that begins and ends at the same node. Chicago Seattle Salt Lake City San Francisco Dallas • p = {Seattle, Salt Lake City, Chicago, Dallas, San Francisco, Seattle}

7. Path Length and Cost • Path length: the number of edges in the path • Path cost: the sum of the costs of each edge Chicago 3.5 Seattle 2 2 Salt Lake City 2 2.5 2.5 2.5 3 San Francisco Dallas length(p) = 5 cost(p) = 11.5

8. More Definitions:Simple Paths and Cycles A simple path repeats no vertices (except that the first can also be the last): p = {Seattle, Salt Lake City, San Francisco, Dallas} p = {Seattle, Salt Lake City, Dallas, San Francisco, Seattle} A cycle is a path that starts and ends at the same node: p = {Seattle, Salt Lake City, Dallas, San Francisco, Seattle} p = {Seattle, Salt Lake City, Seattle, San Francisco, Seattle} A simple cycle is a cycle that is also a simple path (in undirected graphs, no edge can be repeated)

9. Trees as Graphs • Every tree is a graph with some restrictions: • the tree is directed • there are no cycles (directed or undirected) • there is a directed path from the root to every node A B C D E F G H

10. main() mult() add() read() access() Directed Acyclic Graphs (DAGs) DAGs are directed graphs with no (directed) cycles. Aside: If program call-graph is a DAG, then all procedure calls can be in-lined {Tree}  {DAG}  {Graph}

11. Han Luke Leia Rep 1: Adjacency Matrix A |V| x |V| array in which an element (u,v) is true if and only if there is an edge from uto v Han Luke Leia Han Luke Leia Runtimes: Iterate over vertices? Iterate over edges? Iterate edges adj. to vertex? Existence of edge? Space requirements?

12. Han Luke Leia Rep 2: Adjacency List A |V|-ary list (array) in which each entry stores a list (linked list) of all adjacent vertices Han Luke Leia Runtimes: Iterate over vertices? Iterate over edges? Iterate edges adj. to vertex? Existence of edge? Space requirements?

13. Some Applications:Moving Around Washington What’s the shortest way to get from Seattle to Pullman? Edge labels: Distance

14. Some Applications:Moving Around Washington What’s the fastest way to get from Seattle to Pullman? Edge labels: Distance, speed limit

15. Some Applications:Reliability of Communication If Wenatchee’s phone exchange goes down,can Seattle still talk to Pullman?

16. Some Applications:Bus Routes in Downtown Seattle If we’re at 3rd and Pine, how can we get to 1st and University using Metro? How about 4th and Seneca?

17. This is a partial ordering, for sorting we had a total ordering CSE 403 CSE 321 CSE 322 CSE 421 CSE 326 CSE 142 CSE 143 CSE 341 CSE 451 CSE 370 CSE 467 CSE 378 Application: Topological Sort Given a directed graph, G = (V,E), output all the vertices in V such that no vertex is output before any other vertex with an edge to it. Minimize and DO a topo sort Is the output unique?

18. Topological Sort: Take One • Label each vertex with its in-degree (# of inbound edges) • While there are vertices remaining: • Choose a vertex v of in-degree zero; output v • Reduce the in-degree of all vertices adjacent to v • Remove v from the list of vertices Runtime:

19. void Graph::topsort(){ Vertex v, w; labelEachVertexWithItsIn-degree(); for (int counter=0; counter < NUM_VERTICES; counter++){ v = findNewVertexOfDegreeZero(); v.topologicalNum = counter; for each w adjacent to v w.indegree--; } } Time? Time? Time? What’s the bottleneck? O(depends)

20. Topological Sort: Take Two • Label each vertex with its in-degree • Initialize a queue Q to contain all in-degree zero vertices • While Q not empty • v = Q.dequeue; output v • Reduce the in-degree of all vertices adjacent to v • If new in-degree of any such vertex u is zeroQ.enqueue(u) Note: could use a stack, list, set, box, … instead of a queue Runtime:

21. void Graph::topsort(){ Queue q(NUM_VERTICES); int counter = 0; Vertex v, w; labelEachVertexWithItsIn-degree(); q.makeEmpty(); for each vertex v if (v.indegree == 0) q.enqueue(v); while (!q.isEmpty()){ v = q.dequeue(); v.topologicalNum = ++counter; for each w adjacent to v if (--w.indegree == 0) q.enqueue(w); } } intialize the queue get a vertex with indegree 0 insert new eligible vertices Runtime: O(|V| + |E|)

22. Graph Connectivity Undirected graphs are connected if there is a path between any two vertices Directed graphs are strongly connected if there is a path from any one vertex to any other Directed graphs are weakly connected if there is a path between any two vertices, ignoring direction A complete graph has an edge between every pair of vertices

23. Graph Traversals • Breadth-first search (and depth-first search) work for arbitrary (directed or undirected) graphs - not just mazes! • Must mark visited vertices so you do not go into an infinite loop! • Either can be used to determine connectivity: • Is there a path between two given vertices? • Is the graph (weakly) connected? • Which one: • Uses a queue? • Uses a stack? • Always finds the shortest path (for unweighted graphs)?

24. CSE 326: Data StructuresGraph Traversals James Fogarty Autumn 2007

25. Graph Connectivity • Undirected graphs are connected if there is a path between any two vertices • Directed graphs are strongly connected if there is a path from any one vertex to any other • Directed graphs are weakly connected if there is a path between any two vertices, ignoring direction • A complete graph has an edge between every pair of vertices

26. Graph Traversals • Breadth-first search (and depth-first search) work for arbitrary (directed or undirected) graphs - not just mazes! • Must mark visited vertices. Why? • So you do not go into an infinite loop! It’s not a tree. • Either can be used to determine connectivity: • Is there a path between two given vertices? • Is the graph (weakly/strongly) connected? • Which one: • Uses a queue? • Uses a stack? • Always finds the shortest path (for unweighted graphs)?

27. The Shortest Path Problem • Given a graph G, edge costs ci,j, and vertices s and t in G, find the shortest path from s to t. • For a path p = v0v1v2 … vk • unweighted length of path p = k (a.k.a. length) • weighted length of path p = i=0..k-1ci,i+1 (a.k.a cost) • Path length equals path cost when ? ci,j = 1 for every edge

28. Single Source Shortest Paths (SSSP) • Given a graph G, edge costs ci,j, and vertex s, find the shortest paths from s to all vertices in G. • Is this harder or easier than the previous problem? Potentially harder, but unfortunately nobetter algorithms known for the former! So we’ll solve this instead!

29. All Pairs Shortest Paths (APSP) • Given a graph G andedge costs ci,j, find the shortest paths between all pairs of vertices in G. • Is this harder or easier than SSSP? • Could we use SSSP as a subroutine to solve this? What would be the running time? Yes. Will take time |V| . T(SSSP) to run. However, better algorithms known. WON’T COVER IN THIS CLASS

30. Depth-First Graph Search Open – Stack Criteria – Pop • DFS( Start, Goal_test) • push(Start, Open); • repeat • if (empty(Open)) then return fail; • Node := pop(Open); • if (Goal_test(Node)) then return Node; • for each Child of node do • if (Child not already visited) then push(Child, Open); • Mark Node as visited; • end

31. Breadth-First Graph Search Open – Queue Criteria – Dequeue (FIFO) • BFS( Start, Goal_test) • enqueue(Start, Open); • repeat • if (empty(Open)) then return fail; • Node := dequeue(Open); • if (Goal_test(Node)) then return Node; • for each Child of node do • if (Child not already visited) then enqueue(Child, Open); • Mark Node as visited; • end

32. Comparison: DFS versus BFS • Depth-first search • Does not always find shortest paths • Must be careful to mark visited vertices, or you could go into an infinite loop if there is a cycle • Breadth-first search • Always finds shortest paths – optimal solutions • Marking visited nodes can improve efficiency, but even without doing so search is guaranteed to terminate • Is BFS always preferable?

33. DFS Space Requirements • Assume: • Longest path in graph is length d • Highest number of out-edges is k • DFS stack grows at most to size dk • For k=10, d=15, size is 150

34. BFS Space Requirements • Assume • Distance from start to a goal is d • Highest number of out edges is k BFS • Queue could grow to size kd • For k=10, d=15, size is 1,000,000,000,000,000

35. Conclusion • For large graphs, DFS is hugely more memory efficient, if we can limit the maximum path length to some fixed d. • If we knew the distance from the start to the goal in advance, we can just not add any children to stack after level d • But what if we don’t know d in advance?

36. Iterative-Deepening DFS (I) • Bounded_DFS(Start, Goal_test, Limit) • Start.dist = 0; • push(Start, Open); • repeat • if (empty(Open)) then return fail; • Node := pop(Open); • if (Goal_test(Node)) then return Node; • if (Node.dist Limit) then return fail; • for each Child of node do • if (Child not already i-visited) then • Child.dist := Node.dist + 1; • push(Child, Open); • Mark Node as i-visited; • end

37. Iterative-Deepening DFS (II) • IDFS_Search(Start, Goal_test) • i := 1; • repeat • answer := Bounded_DFS(Start, Goal_test, i); • if (answer != fail) then return answer; • i := i+1; • end

38. Analysis of IDFS • Work performed with limit < actual distance to G is wasted – but the wasted work is usually small compared to amount of work done during the last iteration Ignore low order terms! Same time complexity as BFS Same space complexity as (bounded) DFS

39. Saving the Path • Our pseudocode returns the goal node found, but not the path to it • How can we remember the path? • Add a field to each node, that points to the previous node along the path • Follow pointers from goal back to start to recover path

40. Example Seattle Salt Lake City San Francisco Dallas

41. Example (Unweighted Graph) Seattle Salt Lake City San Francisco Dallas

42. Example (Unweighted Graph) Seattle Salt Lake City San Francisco Dallas

43. Graph Search, Saving Path • Search( Start, Goal_test, Criteria) • insert(Start, Open); • repeat • if (empty(Open)) then return fail; • select Node from Open using Criteria; • if (Goal_test(Node)) then return Node; • for each Child of node do • if (Child not already visited) then • Child.previous := Node; • Insert( Child, Open ); • Mark Node as visited; • end

44. Weighted SSSP:The Quest For Food Home Ben & Jerry’s Cedars Neelam’s 40 U Village 285 Delfino’s 70 75 365 25 375 Coke Closet 350 The Ave 350 10 25 HUB ALLEN 35 35 Café Allegro 15 Schultzy’s Vending Machine in EE1 15,356 Parent’s Home Can we calculate shortest distance to all nodes from Allen Center?

45. Weighted SSSP:The Quest For Food Home Ben & Jerry’s Cedars Neelam’s 40 U Village 285 Delfino’s 70 75 365 25 375 Coke Closet 350 The Ave 5 10 25 HUB ALLEN 35 35 Café Allegro 15 Schultzy’s Vending Machine in EE1 15,356 Parent’s Home Can we calculate shortest distance to all nodes from Allen Center?

46. Edsger Wybe Dijkstra (1930-2002) • Invented concepts of structured programming, synchronization, weakest precondition, and "semaphores" for controlling computer processes. The Oxford English Dictionary cites his use of the words "vector" and "stack" in a computing context. • Believed programming should be taught without computers • 1972 Turing Award • “In their capacity as a tool, computers will be but a ripple on the surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind.”

47. General Graph Search Algorithm Open – some data structure (e.g., stack, queue, heap) Criteria – some method for removing an element from Open • Search( Start, Goal_test, Criteria) • insert(Start, Open); • repeat • if (empty(Open)) then return fail; • select Node from Open using Criteria; • if (Goal_test(Node)) then return Node; • for each Child of node do • if (Child not already visited) then Insert( Child, Open ); • Mark Node as visited; • end

48. Shortest Path for Weighted Graphs • Given a graph G = (V, E) with edge costs c(e), and a vertex s  V, find the shortest (lowest cost) path from s to every vertex in V • Assume: only positive edge costs

49. Dijkstra’s Algorithm for Single Source Shortest Path • Similar to breadth-first search, but uses a heap instead of a queue: • Always select (expand) the vertex that has a lowest-cost path to the start vertex • Correctly handles the case where the lowest-cost (shortest) path to a vertex is not the one with fewest edges

50. CSE 326: Data StructuresDijkstra’s Algorithm