Introduction to Computer Science 1

1. Introduction to Computer Science 1 Topic 6: Generative Recursion

2. Outline • Introduction to generative recursion • Sorting with generative recursion • Guidelines for the design of generative recursive procedures • Structural versus generative recursion • Traversing graphs

3. Generative Recursion • So far, we have used structural recursionto process structurally recursive data • We have decomposed the data to their immediate structural components • We have processed these components and combined the results • However: • Not all problems can be solved using structurally recursive functions • Even if there is a structurally recursive solution for a problem, it might not be optimal • Now we will consider a new programming style: Generative recursion

4. Generative Recursion • DIVIDE and CONQUERis an important principle underlying generative recursion • If the problem is trivial to solve, the solution for the trivial case is returned • Otherwise: • Divide the problem in new smaller problems (generate new sub-problems not necessarily derived from the structure of data) • Conquer the sub-problems by applying the same technique recursively • Combine the solutions for the sub-problems into a solution for the original problem • The design of generative recursive programs is more an ad-hoc activity as compared to the design of the structural recursive programs • Needs an insight – a "EUREKA!"

5. Modeling a rolling ball on a table Task Description: • Assume that we want to write a program that simulates a ball rolling on a table • The ball rolls at a constant speed until it drops off the edge of the table • We can model the table as a canvas/surface with a pre-defined length and width • The ball can be represented as a disc that moves across the canvas • The disc movement can be represented by repeating the following steps: • Draw the disc at the current position on the canvas • Wait for a certain pre-defined time • Erase the disc at the current position • Move to a new position

6. Ball Structure and Operations ;;TeachPack: draw.ss ;; structure: (make-ball number number number number) (define-struct ball (x y delta-x delta-y)) ;; draw-and-clear : a-ball -> true (define (draw-and-clear a-ball) (and (draw-solid-disk (make-posn (ball-x a-ball) (ball-y a-ball)) 5 'red) (sleep-for-a-while DELAY) (clear-solid-disk (make-posn (ball-x a-ball) (ball-y a-ball)) 5 'red)))

7. Ball Structure and Operations ;; move-ball : ball -> ball (define (move-ball a-ball) (make-ball (+ (ball-x a-ball) (ball-delta-x a-ball)) (+ (ball-y a-ball) (ball-delta-y a-ball)) (ball-delta-x a-ball) (ball-delta-y a-ball))) ;; Dimension of canvas (define WIDTH100) (define HEIGHT100) (define DELAY.1)

8. To move the ball multiple times we can write: This gets tedious after a while. We need a function that moves the ball until it is out of bounds. Ball Structure and Operations (define the-ball (make-ball10 20 -5 +17)) (and (draw-and-clear the-ball) (and (draw-and-clear (move-ball the-ball)) ...))

9. Rolling the Ball Determine whether a-ball is outside of the bounds: Template for moving the ball until it is out of bounds: ;; out-of-bounds? : a-ball -> boolean (define (out-of-bounds? a-ball) (not (and (<= 0 (ball-x a-ball) WIDTH) (<= 0 (ball-y a-ball) HEIGHT)))) The trivial case: return true ;; move-until-out : a-ball -> true (define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) ... ] [else ...])) true ?

10. Rolling the Ball After drawing and moving the ball, we apply move-until-out again, which means the function is recursive: (define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) true] [else (and (draw-and-clear a-ball) (move-until-out (move-ball a-ball)))])) We can now test the function as follows: The code creates a canvas of proper size and a ball that moves to the bottom left of the canvas. (startWIDTHHEIGHT) (move-until-out (make-ball1020-5+17)) (stop)

11. A New Type of Recursion The procedure move-until-out uses a new type of recursion: • conditions have nothing to do with the input data • recursive application in the body does not use part of the input • move-until-outgenerates an entirely new and different ball structure and uses it for the recursion We do not have a design recipe for this (define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) true] [else (and (draw-and-clear a-ball) (move-until-out (move-ball a-ball)))]))

12. Outline • Introduction to generative recursion • Sorting with generative recursion • Guidelines for the design of generative recursive procedures • Structural versus generative recursion • Traversing graphs

13. Sorting: Quicksort and Mergesort • We are once more concerned with sorting the elements of a list … • We have seen insertion sort: • a structural recursive procedure • Now we will see two other algorithms for sorting: Quicksort and Mergesort • Two classic examples of generative recursion • Based on the idea of divide and conquer

14. [Reminder: insertion sort] ;; insertion-sort: list-of-numbers  ->  list-of-numbers ;; creates a sorted list of numb. from numbers in alon (define (insertion-sort alon) (cond [(empty? alon) empty] [else (insert (first alon) (insertion-sort (rest alon)))])) unsorted sorted an an

15. Quicksort : The Idea • An arbitrary intermediate step: sorting an arbitrary sub-list: L0=(list elp…elr) • Divide: PartitionL0 in two (possibly empty) lists, L1=(list elp…elq-1) and L2=(list elq+1…elr), such that each element of L1 is smaller than elq, and the latter is in turn smaller than each element in L2 • Conquer: Apply the same procedure recursively to sort L1 and L2 • Combine: simply concatenate the sorted lists pivot element elq < elq > elq

16. Quicksort : The Idea • Two open questions: • How do we select the pivot element? • We always use the first element • When do we stop? That is: what is the trivial case of Quicksort? • The empty list is always sorted!

17. Sort (list 11 8 7 14): Pivot item is 11. Two sub-lists: (list 8 7) and (list 14) Sort (list 8 7) and (list 14) Concatenate sorted sub-lists: (list 7 8),11 and (list 14) 11 (list 14) (list 8 7) empty 14 empty 8 empty (list 7) empty 7 empty (list 14) (list 7) (list 7 8) (list 7 8 11 14) Quicksort @ Work

18. QuicksortSchematically pivot item

19. quick-sort distinguishes two cases: If the input is empty, it returns empty. Otherwise, it performs generative recursion. Each sub-list is sorted separately, using quick-sort The sorted versions of the two lists are then combined using append Quicksort Algorithm ;; quick-sort : (listof number) -> (listof number) (define (quick-sort alon) (cond [(empty? alon) empty] [else (append (quick-sort (below alon (first alon))) (list (first alon)) (quick-sort (above alon (first alon))) ) ]))

20. Auxiliary Functions of Quicksort • above filters out the items that are larger than threshold: • below filters out the items that are smaller than threshold: (define (above alon threshold) (filter1 > alon threshold))) (define (below alon threshold) (filter1 < alon threshold)))

21. Quicksort Evaluation Example (quick-sort (list 11 8 14 7)) = (append (quick-sort (list 8 7)) (list 11) (quick-sort (list 14))) = (append (append (quick-sort (list 7)) (list 8) (quick-sort empty)) (list 11) (quick-sort (list 14))) = (append (append (append (quick-sort empty) (list 7) (quick-sort empty)) (list 8) (quick-sort empty)) (list 11) (quick-sort (list 14))) = ...

22. Quicksort Evaluation Example = (append (append (append empty (list7) empty) (list 8) empty) (list 11) (quick-sort (list 14))) = (append (append (list 7) (list 8) empty) (list 11) (quick-sort (list 14))) = (append (list 78) (list 11) (quick-sort (list 14))) = ...

23. mergesort:The Idea Idea: (a) split the list in the middle, (b) apply the function to the sub-lists recursively, (c) merge the two sorted lists into a new sorted list

24. 8 5 4 2 9 6 2 3 3 4 5 6 8 9 Merging two ordered lists • Given two ordered lists, ls1 and ls2, • how do we merge them into a new ordered list?

25. Merging two ordered lists ;; merge: ;; listof number listof number -> listof number (define (merge ls1 ls2) (cond [(empty? ls1) ls2] [(empty? ls2) ls1] [(< (first ls1) (first ls2)) (cons (first ls1) (merge (rest ls1) ls2)) ] [else (cons (first ls2) (merge ls1 (rest ls2)))] ) )

26. mergesort:algorithm in Scheme (define (merge-sort alon) (local ((define (merge-step left right) (cond [(>= left right) alon] [else (local ( (define mid (floor (/ (+ left right) 2))) (define left-list (merge-sort (extract alon left mid))) (define right-list (merge-sort (extract alon (+ mid 1) right)))) (merge left-list right-list) ) ] ))) (merge-step 1 (length alon))))

27. mergesort algorithm in Scheme ;; extract : (listof X) number number -> (listof X) ;; to extract all elments of list alox from ;; index left to index right (define (extract alox left right) (cond [(empty? alox) empty] [(> left right) empty] [(> left 1) (extract (rest alox) (- left 1) (- right 1))] [else (cons (first alox) (extract alox (+ left 1) right))]))

28. Outline • Introduction to generative recursion • Sorting with generative recursion • Guidelines for the design of generative recursive procedures • Structural versus generative recursion • Traversing graphs

29. Guidelines for Designing Generative Recursive Procedures • Understand the nature of the procedure's data • Describe the process in terms of data, creating a new structure or partitioning a list of numbers • Distinguish between input data, • which can be processed trivially, • and those which cannot • The generation of problems is the key to algorithm design • The solutions of the generated problems must be combined

30. Six Stages of Structural Design • Data analysis and design: • analyze and define the data collections representing the problem • Contract, purpose, header • specify what the function does • explain in general terms how it works • Function examples: • illustrate how the algorithm proceeds for some given input • Template: • follow a general template • Definition: • answer the questions posed by the template • Test • Test the completed function • Eliminate the bugs

31. General Template for Generative Recursive Functions (define (generative-recursive-fun problem) (cond [(trivially-solvable? problem) (determine-solution problem)] [else (combine-solutions ... problem ... (generative-recursive-fun (generate-problem-1 problem)) ... (generative-recursive-fun (generate-problem-n problem)))]))

32. Procedure Definition • What is a trivially solvable problem, and the pertaining solution? • How do we generate new problems that are easier to solve than the initial problem? • Is there one new problem to generate, or are there more? • Is the solution for the given problem the same as for (one of) the new problems? • Or do we have to combine solutions to create a solution for the initial problem? • If so, do we require parts of the data constituting the initial problem?

33. Termination of Structurally Recursive Procedures • So far, each function has produced an output for a valid input  evaluation of structurally recursive procedures has always terminated. • Important characteristic of the recipe for structurally recursive procedures: • each step of natural recursion uses an immediate sub-component of the input, not the input itself • Because data is constructed in a hierarchical manner, the input shrinks at every stage • The function sooner or later consumes an atomic piece of data and terminates.

34. Termination of Generative Recursive Procedures • This characteristic is not true for generative recursive functions. • The internal recursions don't consume an immediate component of the input but some new piece of data, which is generated from the input. • This step may produce the initial input over and over again and thus prevent the evaluation from ever producing a result • We say that the program is trapped in an INFINITE LOOP.

35. Non-Terminating Procedures • What happens if we place the following three expressions at the bottom of the DrScheme's Definitions window and click execute? • Does the second expression ever produce a value so that the third expression is evaluated and the canvas disappears? (startWIDTHHEIGHT) (move-until-out (make-ball102000)) (stop)

36. Non-Terminating Procedures The slightest mistake in process definition may cause an infinite loop ;; below : (listof number) number -> (listof number) (define (below alon threshold) (filter1 <= alon threshold) instead of < Quicksort does not terminate with the new function (quick-sort (list 5)) = (append (quick-sort (below 5 (list 5))) (list 5) (quick-sort (above 5 (list 5)))) = (append (quick-sort (list 5)) (list 5) (quick-sort (above 5 (list 5))))

37. Termination Argument • The termination argument is an additional step in the design recipe of generative recursive procedures • Termination argument explains • why the process produces an output for every input • how the function implements this idea • when the process possibly does not terminate

38. Termination Argument for Quicksort At each step, quick-sortpartitions the list into two sublists using below and above. Each function produces a list that is smaller than the input (the second argument), even if the pivot element (the first argument) is an item on the list. Hence each recursive application ofquick-sort consumes a strictly shorter list than the given one. Eventually, quick-sort receives an empty list and returns empty.

39. New Termination Cases Termination argument may reveal additional termination cases. This knowledge can be added to the algorithm: E.g., (below N (list N)) and (above N (list N)) always produce empty (define (quick-sort alon) (cond [(empty? alon) empty] [(empty? (rest alon)) alon] [else (append (quick-sort (smaller-items alon (first alon))) (list (first alon)) (quick-sort (larger-items alon (first alon))) )]))

40. Guidelines for Algorithm Design

41. Outline • Introduction to generative recursion • Sorting with generative recursion • Guidelines for the design of generative recursive procedures • Structural versus generative recursion • Traversing graphs

42. trivially-solvable? empty? generate-problem rest Structural Recursion as a Special Case of Generative Recursion Template for generative recursion (define (generative-recursive-fun problem) (cond [(trivially-solvable? problem) (determine-solution problem)] [else (combine-solutions problem (generative-recursive-fun (generate-problem problem)))])) Template for list processing (define (generative-recursive-fun problem) (cond [(empty? problem) (determine-solution problem)] [else (combine-solutions problem (generative-recursive-fun (rest problem)))]))

43. Structural vs. Generative Recursion • Is there a difference between structural and generative recursion? • Structurally recursive functions seem to be just special cases of generative recursion • But this "it's all the same" attitude does not improve the understanding of the design process • Structurally and generative recursive functions are designed using different approaches and have different consequences.

44. Greatest Common Denominator (GCD) • Examples • 6 and 25 are both numbers with several denominators: • 6 is evenly divisible by 1, 2, 3, and 6; • 25 is evenly divisible by 1, 5, and 25. • The greatest common denominator of 25 and 6 is 1. • 18 and 24 have many common denominator : • 18 is evenly divisible by 1, 2, 3, 6, 9, and 18; • 24 is evenly divisible by 1, 2, 3, 4, 6, 8, 12, and 24. • The greatest common denominator is 6.

45. GCD Based on Structural Recursion Test for every number i = [min(n,m) .. 1] whether it divides both n and m evenly and return the first such number. ;; gcd-structural : N[>= 1] N[>= 1] -> N ;; structural recursion using data definition of N[>= 1] (define (gcd-structural n m) (local ((define (first-divisor i) (cond [(= i 1) 1] [(and (= (remainder n i) 0) (= (remainder m i) 0)) i] [else (first-divisor (- i 1))])] ) ) ) (first-divisor (min m n)))) inefficient for large numbers

46. Analysis of Structural Recursion • gcd-structural simply tests every number whether it divides both n and m evenly and returns the first such number • For small natural numbers, this process works just fine • However, for (gcd-structural 101135853 450146405) = 177 the procedure will compare 101135853 – 177 = 101135676 numbers! • Even reasonably fast computers spend several minutes on this task. • Enter the definition of gcd-structural into the Definitions window and evaluate the following expression… in the Interactions window … and go get a coffee  (time (gcd-structural 101135853 450146405))

47. GCD Euclidean Algorithm • The Euclidean algorithm to determine the greatest common denominator (GCD) of two integers is one of the oldest algorithms known • Appeared in Euclid's Elements around 300 BC • However, the algorithm probably was not discovered by Euclid and it may have been known up to 200 years earlier. Insight: For two natural numbersnandm, n > m, GCD(n, m) = GCD(m,remainder(n,m)) (gcd larger smaller) = (gcd smaller (remainder larger smaller)) Example: GCD(18, 24) = GCD(18,remainder(24/18)) = GCD(18,6) = GCD(6,0) = 6

48. GCD Generative Algorithm ;; gcd-generative : N[>= 1] N[>=1] -> N (define (gcd-generative n m) (local ((define (clever-gcd larger smaller) (cond [(= smaller 0) larger] [else (clever-gcd smaller (remainder larger smaller))]))) (clever-gcd (max m n) (min m n)))) clever-gcd is based on generative recursion: • The trivially solvable case, when smaller is 0 • The generative step calls clever-gcd with smaller and (remainder larger smaller) (gcd-generative 101135853 450146405) needs only 9 iterations!

49. GCD Euclidean Algorithm Let n = mq + r, then any number u which dividesbothn and m (n = su and m = tu), also dividesr r = n – qm = su – qtu = (s – qt)u Any number v which divides both m and r(m = s’v and r = t’v ), also divides n n = qm + r = qs´v + t´v = (s´q + t´)v • Therefore, every common denominator of n and m is also a common denominator of m and r. • gcd(n,m) = gcd(m,r) • It is enough if we continue the process with m and r • Since r is smaller in absolute value than m, we will reach r = 0 after a finite number of steps.

50. Which one to use? • Question: can we conclude that generative recursion is better than structural recursion? • Answer: no, not automatically. • Even a well-designed generative procedure is not always faster than structural recursion. • e.g., quick-sortwins over insertion sort only for large lists • Structural recursion is easier to design • Designing generative recursive procedures often requires deep mathematical insight • Structural recursion is easier to understand • It may be difficult to grasp the idea of the generative step.