Recursion

1. Recursion CS 1037a – Topic 14

2. Overview • Basic concept of recursion • recursive vs. iterative • recursive function calls and Call Stack • Recursive algorithms • Towers of Hanoi • Recursive sorting (quick sort)

3. Related materials from Main and Savitch “Data Structures & other objects using C++” • Sec. 9.1: Recursive functions • Sec. 9.3: Reasoning about recursion • Sec. 13.2: Recursive Sorting • Divide and Conquer using recursion (p.635) • Quick-Sort and its complexity analysis (p.646-654)

4. Recursive Definitions • Recursion: defining something in terms of itself • A recursive definition: • uses the word or concept being defined in the definition itself • Includes a base case that is defined directly, without self-reference

5. Recursive Definitions • Example: • Iterativedefinition of a flockof sheep: 2 sheep together, or 3 sheep together, or 4 sheep together, … • Recursive definition: 2 sheep together, or a flock of sheep plus 1 sheep

6. Recursive Definitions • A recursive definition defines an entity in terms of one or more smaller versions of itself • Includes direct definition of a base case that defines the simplest version of the entity • Eventually, one of the smaller versions must be the base case

7. Recursive Functions • Recursive functions call themselves • Embedded calls are intended to solve smaller versions of the problem • Sequence of recursive calls ends when the base case is reached • If the base case is never reached, we have infinite recursion – NOT GOOD

8. Definition of Factorials • Non-recursive definition: n factorial (n!) is the product of the integers between 1 and n inclusive when n >=1; 0! = 1 • Recursive definition: • 0! = 1 (base case) • n! = n * (n-1)! for all n >0 (general case) 4 ! = 4*(3!) = 4*(3*(2!)) = 4*(3*(2*(1!))) = 4*(3*(2*(1*(0!)))) = 4*(3*(2*(1*1))) = 4*(3*(2*1)) = 4*(3*2) = 4*6 = 24 ^^^ base case is applied here

9. Factorial Function Recursive solution Iterative solution int fact( int n ) { // precondition: n >= 0 if ( n == 0 ) return 1; return n * fact( n-1 ); } int fact( int n) { int prod = 1; // 0! for (int k = 1; k <= n; k++) prod = prod * k; return prod; } Recursive implementation looks a lot like the definition

10. Factorial Function Examine the call stack for a main program call int k = fact( 4 ); Main program call returns to the OS; all others return to the multiplication in n * fact( n-1 ); 2 n ? ret 3 n ? ret 3 n ? ret 4 n ? ret 4 n ? ret 4 n ? ret ? k ? k ? k ? k

11. Factorial Function 0 n ? ret 0 n 1 ret 1 n ? ret 1 n ? ret 1 n ? ret 1 n 1 ret 2 n ? ret 2 n ? ret 2 n ? ret 2 n ? ret 3 n ? ret 3 n ? ret 3 n ? ret 3 n ? ret 4 n ? ret 4 n ? ret 4 n ? ret 4 n ? ret ? k ? k ? k ? k

12. Factorial Function 2 n 2 ret 3 n ? ret 3 n 6 ret 4 n ? ret 4 n ? ret 4 n 24 ret ? k ? k ? k 24 k

13. Why Use Recursion? • May be the clearest, most elegant approach to solving a problem • Recursive solution is at times easier to formulate than an iterative solution • A natural approach to use with recursive data structures such as binary trees (Topic 15)

14. Computing Fibonacci Numbers • Fibonacci numbers are the sequence 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, … • The usual definition is recursive: fib(1) = 1 fib(2) = 1 fib(n) = fib(n-1) + fib(n-2) for all n > 2

15. Fibonacci Implementation int fib( int n ) { // precondition: n >= 1 if ( (n==1) || (n==2) ) return 1; return fib(n-1) + fib(n-2); } In this instance, there are two base cases, and the general problem is solved in terms of two smaller versions of the problem

16. 8 fib(5) + fib(4) b 5 3 fib(4) + fib(3) fib(3) + fib(2) c l 3 2 2 1 fib(3) + fib(2) fib(2)+fib(1) fib(2)+fib(1) d i m 2 1 1 1 1 1 fib(2)+fib(1) e 1 1 Evaluating fib(6) Letters: Give order of calls Numbers: Return values a fib(6) k h o g n j f

17. Efficiency of Fibonacci Implementation • Recursive implementation appears to be inefficient: in computing fib(6), fib(1) is called 3 times, fib(2) is called 5 times, fib(3) is called 3 times, fib(4) is called 2 times • Recursive solution was easy to implement using the definition, but an iterative solution is more efficient

18. Iterative Fibonacci Implementation int fib( int n ) { int f1, f2, f3; if ( (n==1) || (n==2) ) return 1; f1 = f2 = 1; for (int k = 3; k <= n; k++ ) { f3 = f1 + f2; f1 = f2; f2 = f3; } // end of for loop return f3; }

19. Towers of Hanoi • We have three pegs, A, B and C, and n disks of decreasing size stacked on A • Challenge: Move the disks to peg C, always observing the following rules: • Move only one disk at a time • A disk may be moved from any peg to any other, as long as a larger disk is never place on top of a smaller one

20. Towers of Hanoi • Iterative solutions to this problem are possible, but are quite challenging to write • In the recursive solution: • the base case is very simple: move the only disk from peg A to peg C • General case must be formulated in terms of smaller version(s) of the problem

21. Towers of Hanoi • Observe: there is only one legal position that will allow the bottom (nth) disk to be moved: • nth disk is alone on peg A • The rest of the disks are stacked legally on a second peg, and the third peg is empty • To solve the problem we’ve been given, we want the empty peg to be peg C

22. Towers of Hanoi Initial situation when n = 5 B C A Only situation in which largest (nth) disk can move to C B C A

23. Towers of Hanoi • By examining how the nth disk can be moved, rather than by trying to figure out the 1st, 2nd, 3rd, etc moves, we can gain insight into the general case for the recursive solution: • First, solve n-1 disk problem, from A to B • Then move nth disk from A to C • Next, solve n-1 disk problem, from B to C

24. Towers of Hanoi • So, we can solve the n disk problem in terms of two different versions of the n-1 disk problem • The trick to building a recursive function is to work out what the parameters should be, so that the recursive calls it contains actually solve the smaller versions

25. Towers of Hanoi • Parameters must indicate: • Number of disks for which the problem is being solved (n) • Which peg we’re moving from (source) • Which peg we’re moving to (target) • Which peg is being used for auxiliary storage by the call (aux) • Disks in each peg will be “displayed” after each move

26. Towers of Hanoi void hanoi( int n, Stack<Rect> & source, Stack<Rect> & target, Stack<Rect> & aux ) { if ( n == 1 ) { target.push(source.pop()) ; /*1*/ //moving one “Rect” from source to target draw(); // some global function “displaying” (current) state of each stack } else { hanoi( n-1, source, aux, target); /*2*/ target.push(source.pop()) ;/*3*/ //moving one “Rect” from source to target draw(); // some global function “displaying” (current) state of each stack hanoi( n-1, aux, target, source); /*4*/ } } see demo: www.csd.uwo.ca/courses/CS1037a/demos.html

27. Towers of Hanoi example of using hanoi() in main() #include “Stack.h” #include “Rect.h”// disks will be represented by rectangles of different widths Stack<Rect> A, B, C;// global variables representing “pegs with disks” (stacks of disks) void draw(); // declaration of global function that “displays” rectangles in stacks A, B, C void hanoi( int n, Stack<Rect> & source, Stack<Rect> target, Stack<Rect> *aux ); // declaration of “hanoi” function main() { for (inti=4; i>=1; i--) A.push(Rect(1,i)); // rectangles of height 1 and width i hanoi(4,A,C,B); } COMMENT: call-by-reference for arguments source, target, and aux in hanoi() implies that all “moves” are done on the same global objects A, B, C see demo: www.csd.uwo.ca/courses/CS1037a/demos.html

28. Trace of hanoi with n=3 hanoi( 3,”A”,”C”,”B” ); /*original call*/ 10:move disk from A to C /*3*/ 1 19 9 11 hanoi( 2,”A”,”B”,”C” ); /*2*/ hanoi( 2,”B”,”C”,”A” ); /*4*/ 5:move disk from A to B /*3*/ 15:move disk from B to C /*3*/ 8 12 6 2 16 18 hanoi( 1,”C”,”B”,”A” ); /*4*/ 14 hanoi( 1,”A”,”C”,”B” ); /*4*/ 7:move disk from C to B /*1*/ 4 17:move disk from A to C /*1*/ hanoi( 1,”A”,”C”,”B” ); /*2*/ hanoi( 1,”B”,”A”,”C” ); /*2*/ 3:move disk from A to C /*1*/ 13:move disk from B to A /*1*/ Numbers show the order in which statements are executed

29. Visual Trace of hanoi with n=4 A B C A B C A B C A B C see demo: www.csd.uwo.ca/courses/CS1037a/demos.html

30. Visual Trace of hanoi with n=4 A B C A B C A B C A B C see demo: www.csd.uwo.ca/courses/CS1037a/demos.html

31. Visual Trace of hanoi with n=4 A B C A B C A B C A B C see demo: www.csd.uwo.ca/courses/CS1037a/demos.html

32. Visual Trace of hanoi with n=4 A B C A B C A B C A B C see demo: www.csd.uwo.ca/courses/CS1037a/demos.html

33. Analysis: Towers of Hanoi • For our analysis, we’ll use a recurrence relation to count the total number of “moves” as a function of the number of “stones” n • Base case: n is 1, and 1 move is made, so T(1) = 1 • General case: There are two recursive calls to solve the problem for n-1 disks, and a single move in between: T(n) = 1 + 2*T(n-1)

34. Analysis: Towers of Hanoi T(n) = 1 + 2*T(n-1) = 1 + 2*(1 + 2*T(n-2)) = 1 + 2 + 22*T(n-2) = 1 + 2 + 22*(1 + 2*T(n-3)) = 1 + 2 + 22 + 23*T(n-3) = 1 + 2 + 22 + 23* + … + 2k*T(n-k) Recursive substitution stops when the base case, T(1), is reached; that is, when n - k = 1, or k = n - 1 T(n) = 1 + 2 + 22 + 23* + … + 2n-1*T(n-(n-1)) = 1 + 2 + 22 + 23* + … + 2n-1*T(1) = 1 + 2 + 22 + 23* + … + 2n-1

35. Analysis: Towers of Hanoi T(n) = 20 + 21 + 22 + 23 + … + 2n-1 2*T(n) = (21 + 22 + 23 + … + 2n-1) + 2n = (T(n) - 20) + 2n = T(n) + 2n – 1 So, T(n) = 2n – 1 Our solution to the Towers of Hanoi problem requires 2n – 1 moves, and the algorithm is O(2n) which is exponential complexity. - It’s possible to show that the problem cannot be solved any faster => computationally “unsolvable”

36. Recursive Sortingat last,… practically useful recursion 

37. Quicksort Efficient practical algorithm for sorting a set of items using recursion • average case time complexity O(n*log2(n)) is better than average complexity of insertion- or selection-sort methodsO(n2) (see in topic 13)

38. Quicksort • Quicksort algorithm proceeds by randomly choosing a pivot element from among the values being sorted, and then partitioning the remaining elements into two groups: those greater than the pivot element, and those less than it • Same process is then applied to the two partitions

39. Quicksort • Pivot element lies between the contents of the two partitions when sorting is finished • Partitioning process eventually stops at empty partitions

40. Quicksort • Can be carried out in place in an array • Can be implemented iteratively • Lends itself to recursive implementation, since two smaller versions of the problem are used to solve the original • Base case occurs when a partition is empty

41. Quicksort Algorithm • Our version, not carried out in place, uses bags to sort contents of a queue • Assume that objects to be sorted are from a class that has the comparison operations (>, <, ==, >=, etc) defined on them • Remember that our bags and queues hold pointers to the data items

42. Quicksort Algorithm Example: moves items from bag binto sorted queue q If bag of items b is not empty // do nothing otherwise (base case) Get an item from b and call it pivot Create two new bags called bigger and smaller While b is not empty Get an item from b and call it current If current < pivot add current to smaller Else add current to bigger Recursively sort smaller Add pivot to the sorted queue q Recursively sort bigger

43. quickSort Algorithm 14 27 - order of recursive quickSort calls i 35 6 44 (i) - order of numbers in queue q 19 8 6 1 pivot: 14 (3) 10 19 35 6 smaller 8 8 bigger 27 27 44 2 7 5 35 pivot: 27 11 44 44 pivot: 8 (5) (2) pivot: 44 (7) bigger 6 6 smaller smaller 19 14 19 smaller 35 35 pivot: 6 pivot: 19 4 bigger (1) pivot: 35 9 3 (4) 8 (6) 13 12

44. Example ofquickSort Implementation • We’ll give oneexample of quickSort as a member function for class Queuethat enqueues all content from a given bag into the queue in a specified order • A comparison functorwill be an extra template parameter for this function template <class Item> template<class Order> void Queue<Item>::quickSort( Bag<Item> &b) call by reference

45. Example ofquickSort Implementation template <class Item> template<class Order> void Queue<Item>::quickSort( Bag<Item> &b) {// call by reference if ( b.isEmpty() ) return; // base case Bag<Item> smaller, bigger; Item pivot = b.getOne( ); while ( !(b.isEmpty( )) ) { Item current = b.getOne( ); if ( Order::compare(current, pivot) )smaller.add( current ); else bigger.add( current ); } quickSort( smaller); // call by reference enqueue( pivot ); quickSort( bigger ); // call by reference }

46. Example of callingquickSort inmain #include “StandardFunctors.h” #include “Queue.h” int main(void) { Queue<int> intQ; Bag<int> b; … // assume that some values are added into bagb here intQ.quickSort<IsLess>( b ); // call by reference // sortscontent of b into queue intQ in ascending order }

47. Complexity analysis ofour quickSort • We’ll analyze two cases for our version of quicksort: • In the first case, the pivot item is always the largest item in the bag • In the second case, the pivot item is the middle item • These turn out to be the worst and best cases respectively (but we won’t prove this fact)

48. Complexity analysis ofour quickSort • Worst case analysis: pivot element is always the largest item in the bag • Assume there are n items in the bag b ; we need to determine T(n)counting the number of calls to getOne() (check thatcalls to getOne dominate all other calls) • If n=0, no calls to getOne are made, so T(0) is 0 • If n > 0, the pivot element is removed from b (1 getOne operation), and the remaining n-1 elements are moved from b to smaller (n-1 getOne operations)

49. Complexity analysis ofour quickSort • First recursive call then sorts smaller, a bag holding n-1 items; this requires T(n-1)getOneoperations • Second recursive call sorts bigger, an empty bag; this requires T(0) operations • So, T(n) = 1 + (n-1) + T(n-1) +T(0) = = n + T(n-1)for n>0

50. Complexity analysis ofour quickSort T(n) = n + T(n-1) = n + (n-1) + T(n-2) = n + (n-1) + (n-2) + T(n-3) = … = n + (n-1) + (n-2) + … + (n-k) + …+ 2 + 1 + T(0) = n + (n-1) + (n-2) + … + (n-k) + …+ 2 + 1 + 0 = n(n+1)/2 T(n)= O(n2) Worst case complexity