Stuff

1. Stuff • Assn 5 is available and due Friday, 7pm. • Winding down! Possible Remaining topics: • Shell and Radix sorts (not on exam) • Storage of numbers in Java – the source and effects of roundoff error (possibly on exam?) • We still need to pick a pre-exam tutorial time. (Exam is April 27). CISC121 - Prof. McLeod

2. Last Time • Two more unrolling examples. • Mergesort algorithm and code. CISC121 - Prof. McLeod

3. Today • Complexity of Mergesort. • Quicksort. CISC121 - Prof. McLeod

4. Mergesort – “aMergeSort” Code • Code for sorting arrays: public static void aMergeSort (int[] A) { aMergeSort(A, 0, A.length-1); } // end aMergeSort public static void aMergeSort (int[] A, int first, int last) { if (first < last) { int mid = (first + last) / 2; aMergeSort(A, first, mid); aMergeSort(A, mid + 1, last); aMerge(A, first, last); } // end if } // end aMergeSort recursive CISC121 - Prof. McLeod

5. Mergesort – “aMerge” Code private static void aMerge (int[] A, int first, int last) { int mid = (first + last) / 2; int i1 = 0, i2 = first, i3 = mid + 1; int[] temp = new int[last - first + 1]; while (i2 <= mid && i3 <= last) { if (A[i2] < A[i3]) { temp[i1] = A[i2]; i2++; } else { temp[i1] = A[i3]; i3++; } i1++; } // end while CISC121 - Prof. McLeod

6. Mergesort – “aMerge” Code - Cont. while (i2 <= mid) { temp[i1] = A[i2]; i2++; i1++; } // end while while (i3 <= last) { temp[i1] = A[i3]; i3++; i1++; } // end while i1 = 0; i2 = first; while (i2 <= last) { A[i2] = temp[i1]; i1++; i2++; } // end while } // end aMerge CISC121 - Prof. McLeod

7. Complexity of Mergesort • Consider the aMergeSort code shown above: • Suppose that the entire method takes t(n) time, where n is A.length. We want to know the big O notation for t(n) (yes, “we” really do!). • There are no loops in aMergeSort, just some constant time operations, the two recursive calls and the call to aMerge. CISC121 - Prof. McLeod

8. Complexity of Mergesort - Cont. • What is the time function for aMerge? • There is some O(1) stuff and four loops that are O(n): • So, CISC121 - Prof. McLeod

9. Complexity of Mergesort - Cont. • So far, we have not made any mention of the state of the data. Does it make any difference if the data is in reverse order (worst case), random order (average case) or in order already (best case)? • Express t(n) in a recursive expression: CISC121 - Prof. McLeod

10. Complexity of Mergesort - Cont. • Assume that n is a power of 2: • (It is easy enough to show that the proof still holds when n is not a power of two - but I’m not going to do that here). CISC121 - Prof. McLeod

11. Complexity of Mergesort - Cont. • Substitute n/2 for n, to get t(n/2): CISC121 - Prof. McLeod

12. Complexity of Mergesort - Cont. • Do the next unrolling, which will be n/22: • So, after i unrolling’s: CISC121 - Prof. McLeod

13. Complexity of Mergesort - Cont. • This recursion stops when the anchor case, n  1 is encountered. This will occur when: • Substituting this back in the equation on the previous slide: CISC121 - Prof. McLeod

14. Complexity of Mergesort - Cont. • At the anchor case: • Now the equation can be simplified to yield the big O notation, which indicates that t(n) is O(nlog(n)). CISC121 - Prof. McLeod

15. Complexity of Mergesort - Cont. • This also can be seen just by considering the “tree” constructed by the recursive calls: n n/2 n/2 n/4 n/4 n/4 n/4 n/8 n/8 n/8 n/8 n/8 n/8 n/8 n/8 And so on… CISC121 - Prof. McLeod

16. Complexity of Mergesort - Cont. • The dividing stops when the size of the subarray is one, or when n/2i=1. As before this gives i=log(n), which is the “height” of the “tree”. • At every line, n operations must take place to merge the elements, so the “width” of the tree is n. • The total number of operations would be n times log(n). CISC121 - Prof. McLeod

17. Quicksort • Quicksort was invented by C.A.R. Hoare (1961). • Used the same reasoning of dividing a large array into smaller arrays that can be then sorted efficiently, as does the Shell sort algorithm. • Recursion allows the algorithm to be more easily coded. • The algorithm starts by dividing the array into two subarrays based on a key value called the “pivot” or “bound” value. • All values less than the pivot go into the lower subarray, and all others go into the upper subarray. • Then each of these subarrays is subjected to the same process using a recursive call. • This continues until the subarrays are of size 1. CISC121 - Prof. McLeod

18. Quicksort – Cont. • Pseudocode: • Quicksort(array): • If array.length > 1: • Choose pivot value. • For all elements in array: • If element < pivot then it goes in array1, otherwise it goes in array2. • Quicksort(array1). • Quicksort(array2). CISC121 - Prof. McLeod

19. Quicksort – Cont. • How to choose the pivot value? • The idea is to get the subarrays to have about the same size. • The simplest approach is to choose the first element of the array as the pivot. • Better yet, if the array is already sorted to some degree, it will be safer to choose a value from the middle of the array, as is done in the code shown on the next few slides. CISC121 - Prof. McLeod

20. Quicksort – Cont. • Code for “preparation” method: public static void quickSort(int[] A) { int max = 0; for (int i = 1; i < A.length; i++) if (A[i] > A[max]) max = i; swap(A, A.length-1, max); quickSort(A, 0, A.length-2); } // end quickSort(array) CISC121 - Prof. McLeod

21. Quicksort – Cont. • This method locates the maximum element in A and moves it to the last position in the array. • Prevents “lower” (in the next method) from running off the end of the array, in the remote chance that the pivot is chosen as the largest value in the array. • (It also calls the typical “swap” method.) CISC121 - Prof. McLeod

22. public static void quickSort (int[] A, int first, int last) { int lower = first + 1; int upper = last; swap(A, first, (first+last)/2); int pivot = A[first]; while (lower <= upper) { while (A[lower] < pivot) lower++; while (A[upper] > pivot) upper--; if (lower < upper) swap(A, lower++, upper--); else lower++; } swap(A, upper, first); if (first < upper - 1) quickSort(A, first, upper-1); if (upper + 1 < last) quickSort(A, upper+1, last); } // end quickSort(subarrays) CISC121 - Prof. McLeod

23. Example - sorting array: 8 5 4 7 6 1 6 3 8 12 10 • After “preparation”: 8 5 4 7 6 1 6 3 8 10 12 • After selection of pivot value and initial assignment of lower and upper: 6 5 4 7 8 1 6 3 8 10 12 • Upper and lower move in until a swap has to be made: 6 5 4 7 8 1 6 3 8 10 12 • Make the swap and move to next swap to be made: 6 5 4 3 8 1 6 7 8 10 12 Max value pivot lower upper pivot lower upper pivot lower upper CISC121 - Prof. McLeod

24. pivot • Make the swap, and then upper and lower pass each other: 6 5 4 3 6 1 8 7 8 10 12 • Put pivot value back in, and divide into two subarrays: 1 5 4 3 6 6 8 7 8 10 12 • (Note how old pivot and max value are already in correct positions and do not need to be included in next recursive call). upper lower CISC121 - Prof. McLeod

25. pivot pivot • Two recursive calls with subarrays: 4 5 1 3 6 6 7 8 8 10 12 • After upper and lower have moved through arrays and pivots have been replaced: • 3 4 5 6 6 7 8 8 10 12 • Last set of recursive calls with subarrays: 1 3 4 5 6 6 7 8 8 10 12 • Array is sorted: 1 3 4 5 6 6 7 8 8 10 12 CISC121 - Prof. McLeod

26. Quicksort – Cont. • Note that new arrays are not created, only the bounds of the subarrays in the array A are changed. The recursive calls will only contain different bounding values. And the subarrays get smaller for each call. • The anchor case is when the size of the subarray is one. CISC121 - Prof. McLeod

27. Quicksort - Cont. • The worst case is when a near-median value is not chosen – the pivot value is always a maximum or a minimum value. Now the algorithm is O(n2). • However, if the pivot values are always near the median value of the arrays, the algorithm is O(nlog(n)) – which is the best case. (See the derivation of this complexity for merge sort). • The average case also turns out to be O(nlog(n)). CISC121 - Prof. McLeod

28. Choice of Pivot Value • Several techniques can be used: • Choose first value • Choose middle value • Calculate “pseudo-median”. This is how quicksort is implemented in Arrays.sort(). • Why not calculate the actual median? • The technique you use will depend how random the data set is. CISC121 - Prof. McLeod

29. Quicksort - Cont. • So, the choice of the pivot value can be critical. Knowledge of the nature of the data to be sorted might suggest another algorithm to use in choosing the pivot value. • Experiment has shown that Quicksort is faster than any other efficient sort. • It has also been suggested that Quicksort can be made even faster by choosing to use a simple sort like insertion sort for subarray sizes less than about 30. CISC121 - Prof. McLeod

30. Comparison of Quicksort and Mergesort • Mergesort is slower than Quicksort primarily because it does not do any preliminary shuffling of the data when it divides it. • It also requires more memory because of the creation of temporary arrays (or lists) to complete the merge operation. CISC121 - Prof. McLeod

31. Comparison of Quicksort and Mergesort • However, merge sort is very useful when sorting data that does not have all be in memory at once (“in place”). For example, if the data must be saved on disk because it will not all fit in memory, then mergesort will make the minimum number of file reads and writes. • If you have to sort lists, it is very easy to code mergesort for them. CISC121 - Prof. McLeod

32. Comparison of Sorts • Sorting 50,000 random int’s (average case) on a PIII using Ready To Program. • Selection sort is not influenced by the state of the data, where bubble and insertion are. O(n2) O(n) simple O(n2) O(n) O(n2) O(n2) CISC121 - Prof. McLeod

33. Comparison of Sorts – Cont. Efficient O(n1.25) O(n1.25) O(nlog(n)) O(nlog(n)) O(nlog(n)) O(nlog(n)) O(nlog(n)) O(nlog(n)) O(n) Java CISC121 - Prof. McLeod