RAIK 283: Data Structures & Algorithms

1. RAIK 283: Data Structures & Algorithms Decrease and Conquer I Dr. Ying Lu ylu@cse.unl.edu Design and Analysis of Algorithms – Chapter 4

2. RAIK 283: Data Structures & Algorithms • Giving credit where credit is due: • Most of the lecture notes are based on the slides from the Textbook’s companion website • http://www.aw-bc.com/info/levitin • Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University, Dr. Chuck Cusack, Hope College and İnançTahralı, Gebze Institute of Technology • I have modified many of their slides and added new slides. Design and Analysis of Algorithms – Chapter 4

3. Decrease and Conquer • Reduce a problem instance to a smaller instance of the same problem and extend solution • Solve the smaller instance • Extend solution of smaller instance to obtain solution to original problem • Also referred to as inductive, incremental approach or chip and conquer Design and Analysis of Algorithms – Chapter 4

4. Examples of Decrease and Conquer • Decrease by one: • Insertion sort • Graph algorithm: • Topological sorting • Algorithms for generating permutations, subsets • Decrease by a constant factor • Binary search • Fake-coin problems • multiplication à la russe • Josephus problem • Variable-size decrease • Euclid’s algorithm • Selection by partition Design and Analysis of Algorithms – Chapter 4

5. Insertion Sort • A Decrease-by-One algorithm: Insertion-Sort(A[0…n-1]) { Insertion-Sort(A[0…n-2]) Insert A[n-1] into the sorted list A[0…n-2] return the sorted list A[0…n-1] }

6. Directed Acyclic Graphs • A directed acyclic graph or DAG is a directed graph with no directed cycles: Design and Analysis of Algorithms – Chapter 4

7. Topological Sort of a DAG (I) • DAGs arise in many modeling problems, e.g.: • course prerequisite structure • food chains • A topological sort implies a partial ordering on the domain Design and Analysis of Algorithms – Chapter 4

8. Topological Sort of a DAG (II) • Topological sort of a DAG: • Linear ordering of all vertices in graph G such that for every edge (u, v)  G, the vertex, u, where the edge starts is listed before the vertex, v, where the edge ends. cs101 math101 cs525 cs401 cs445 cs990 cs203 cs310 Design and Analysis of Algorithms – Chapter 4

9. Topological Sort cs101 cs203 math101 cs310 cs401 cs445 cs525 cs990 Design and Analysis of Algorithms – Chapter 4

10. Topological Sorting Algorithm (I) 1. Source removal algorithm • Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges • How would you find a source (or determine that such a vertex does not exist) in a digraph • represented by adjacency matrix? • represented by adjacency linked list? Design and Analysis of Algorithms – Chapter 4

11. Topological Sorting Algorithm (I) 1. Source removal algorithm • Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges • How would you find a source (or determine that such a vertex does not exist) in a digraph • represented by adjacency matrix? • represented by adjacency linked list? • Θ(|V|2) using adjacency matrix • Θ(|V|+|E|) using adjacency linked lists Design and Analysis of Algorithms – Chapter 4

12. Topological Sorting Algorithm (I) 1. Source removal (a Decrease-by-One) algorithm Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges cs101 cs203 math101 cs310 cs401 cs445 cs525 cs990 Design and Analysis of Algorithms – Chapter 4 12

13. Topological Sorting Algorithm (II) 2. DFS-based algorithm: cs101 math101 cs525 cs401 cs445 cs990 cs203 cs310 Design and Analysis of Algorithms – Chapter 4

14. Topological Sorting Algorithm (II) 2. DFS-based algorithm: • DFS traversal noting order vertices are popped off stack • Reverse order solves topological sorting • Θ(V2) using adjacency matrix • Θ(V+E) using adjacency linked lists Design and Analysis of Algorithms – Chapter 4

15. In-Class Exercise • Page 143 Design and Analysis of Algorithms – Chapter 4

16. Sum Rule • If a task can be done either in one of n1 ways or in one of n2 ways, where none of the set of n1 ways is the same as any of the set of n2 ways, then there are n1+n2 ways to do the task • In general, • If A1, A2, …, Am are disjoint finite sets, then the number of elements in the union of these sets is as follows |A1⋃A2 ⋃… ⋃Am|=|A1|+|A2|+…+|Am| Design and Analysis of Algorithms – Chapter 4

17. Sum Rule • If A1, A2, …, Am are disjoint finite sets, then the number of elements in the union of these sets is as follows |A1⋃A2 ⋃… ⋃Am|=|A1|+|A2|+…+|Am| Design and Analysis of Algorithms – Chapter 4

18. Insertion Sort • A Decrease-by-One algorithm: Insertion-Sort(A[0…n-1]) { Insertion-Sort(A[0…n-2]) Insert A[n-1] into the sorted list A[0…n-2] return the sorted list A[0…n-1] }

19. Topological Sorting Algorithm Source removal (a Decrease-by-One) algorithm Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges cs101 cs203 math101 cs310 cs401 cs445 cs525 cs990 Design and Analysis of Algorithms – Chapter 4 19

20. Algorithms for Generating Combinatorial Objects Definition : Most important types of combinatorial objects permutations combinations subsets of a given set They typically arise in problems that require a consideration of different choices TSP, Knapsack To solve these problems need to generate combinatorial objects Design and Analysis of Algorithms - Chapter 4 20

21. Generating Permutations Assume the set whose elements need to be permuted is the set of integers from 1 to n They can be interpreted as indices of elements in an n-element set {ai, …, an} Design and Analysis of Algorithms - Chapter 4 21

22. Generating Permutations Assume the set whose elements need to be permuted is the set of integers from 1 to n They can be interpreted as indices of elements in an n-element set {ai, …, an} What would the decrease-by-one technique suggest for the problem of generating all n! permutations? Design and Analysis of Algorithms - Chapter 4 22

23. Generating Permutations Approach : The smaller-by-one problem is to generate all (n-1)! permutations Assuming that the smaller problem is solved We can get a solution to the larger one by inserting n in each of the n possible positions among elements of every permutation of n-1 elements Total number of all permutations will be n.(n-1) ! = n! Design and Analysis of Algorithms - Chapter 4 23

24. Generating Permutations We can insert n in the previously generated permutations left to right right to left Design and Analysis of Algorithms - Chapter 4 24

25. Generating Permutations minimal-change requirement Each permutation differs from the previous one by exchanging only two elements Benefical for algorithm’s speed EX: Advantage of this order in TSP: The length of the new tour can be calculated in constant time By using the length of the previous tour Tour Cost a-b-c-d-e-f-g-h-a 2+4+3+5+2+1+4+1=22 a-b-c-d-e-f-h-g-a 2+4+3+5+2+3+4+2=25 Design and Analysis of Algorithms - Chapter 4 25

26. Generating Permutations Approach : Assume all permutations of {1,2,…,(n-1)} are available Start with inserting n into 1,2,…,(n-1) by moving right to left Then swicth direction every time a new permutation of {1,2,…,(n-1)} is processed Design and Analysis of Algorithms - Chapter 2 26

27. Generating Permutations This order satisfiesminimal-change requirement This approach requires that all the permutations of {1,2,…,(n-1)} are calculated already Not easy to do!... (requires lots of space) Design and Analysis of Algorithms - Chapter 4 27

28. Generating Permutations Another way to get the same ordering : Associate a direction with each component k in a permutation Indicate such a direction by a small arrow 3 2 4 1 The component k is said to be mobile if its arrow points to a smaller number adjacent to it Design and Analysis of Algorithms - Chapter 4 28

29. Generating Permutations Another way to get the same ordering : Associate a direction with each component k in a permutation Indicate such a direction by a small arrow 3 2 4 1 The component k is said to be mobile if its arrow points to a smaller number adjacent to it 3 and 4 are mobile 2 and 1 are not The following algorithm uses this notion Design and Analysis of Algorithms - Chapter 4 29

30. Generating Permutations ALGORITHM Johnson Trotter (n) // Implements Johnson-Trotter algorithm for generating permutations // Input : A positive integer n // Output : A list of permutations of {1, … , n} Initialize the first permitation with 1 2 … n while there exists a mobile integer k do find the largest mobile integer k swap k and the adjacent integer its arrow points to reverse the direction of all integers that are larger than k Design and Analysis of Algorithms - Chapter 4 30

31. Generating Permutations An application of Johnson Trotter algorithm : 1 2 3 1 3 2 3 1 23 2 1 2 3 1 2 1 3 Design and Analysis of Algorithms - Chapter 4 31

32. In-class Exercise Page 148 exercise 4.3.2 Generate all permutations of {1,2,3,4} by a. the bottom-up minimal-change algorithm. b. the Johnson-Trotter algorithm. Design and Analysis of Algorithms – Chapter 4 32

33. Generating Permutations The Johnson-Trotter algorithm does not produce permutations in lexicographical order Example Johnson-Trotter algorithm: 123, 132, 312, 321, 231, 213 Lexicographical order: 123, 132, 213, 231, 312, 321 Use an example: produce permutations {1,2,3,4,5} in lexicographical order, to get an idea for the algorithm Design and Analysis of Algorithms – Chapter 4 33

34. Lexicographical Order Algorithm • Initial permutation: a1, a2, … an in increasing order • 2. Scan a current permutation from right to left looking for the first pair of consecutive elements ai and ai+1 such that ai < ai+1 • 3. Find the smallest digit in the tail that is larger than ai and put it in position I • 4. Put the rest n elements in increasing order in position i+1 to n Design and Analysis of Algorithms – Chapter 4 34