Unit IV : Searching and sorting Techniques

2. Unit IV : Searching and sorting Techniques Contents : Need of searching and sorting, Concept of internal and external sorting, sort stability. Searching methods: Linear and binary search algorithms their comparison and complexity analysis Sorting methods: Bubble, selection, insertion, merge, quick, bucket sort and their time and space complexity analysis • Objective : • To understand frequent operations for accessing and manipulating data in information systems. • To understand the comparative performance of various operations on varying spread of data.

3. Syllabus • Need of searching & sorting • Concept of internal & external sorting • Sort stability • Searching methods: Linear & Binary search, comparision & complexity analysis • Sorting Techniques: Bubble, Selection, Insertion, Quick, Merge, Bucket sort and their time & space complexity

4. Searching • Search: locate an item in a list of information • Two algorithms : • Linear search • Binary search

5. Linear Search • Also called the sequential search • Starting at the first element, this algorithm sequentially steps through an array examining each element until it locates the value it is searching for.

6. 12 7 14 22 5 18 40 List of Comparable Items: A = key Item: key = 5

7. 12 7 14 22 5 18 40 A = key key key key key key key = 18 0 1 2 3 4 5 key == a[5]? key == a[0]? key == a[1]? key == a[2]? key == a[3]? key == a[4]? Done!

8. int linearSearch(int [ ] a, int n, int key) { int i = 0; while(i < n)) { if (key == a[i]) return i i++; } return -1; }

9. key is at A[0] key is at A[size-1] OR not found key can appear in each of the n index positions with equal probability Number of comparisions: Best Case Analysis: 1 Worst Case Analysis: n Average Case Analysis: (n + 1)/2

10. Int binarySearch(int a[ ], int first, int last, int key) { int mid; if (first > last) return false; else { mid = (first + last)/2; if (key = = a[mid]) return true; else if (key < a[mid]) return binarySearch(a, first, mid -1, key); return binarySearch(a, mid + 1, last, key); } //end else return -1; } //end binarySearch

11. Analysis: After 1 comaparisons  Remaining file size = n / 2 = n / 21 After 2 comaparisons  Remaining file size = n / 4 = n / 22 After 3 comaparisons  Remaining file size = n / 8 = n / 23 …… After k comaparisons  Remaining file size = n / 2k The process terminates when no more partitions are possible i.e. remaining file size = 1 n / 2k = 1 k = log2n Thus, the time complexity is O(log2n). which is very efficient.

12. Sorting One fundamental problems of computer science is ordering a list of items. Many solutions exist to this problem, known as sorting algorithms. Some sorting algorithms are simple such as the bubble sort. Others, such as the quick sort are extremely complicated, but produce lightning-fast results.

13. Sorting Algorithms are divided into two categories: • Internal Sorts and • External sorts.

14. Sorting Internal Sort: Any sort algorithm which uses main memory exclusively during the sort. This assumes high-speed random access to all memory.

15. Sorting External Sort: Any sort algorithm which uses external memory, such as tape or disk, during the sort.

16. Internal Vs External sorting • Internal sorting is done in internal memory • the internal sorting can reside in main memory • internal sorting is independent of time to read/write a record • internal sorting takes input only which can be fit into its memory...i.e. it takes small input • Egs : Heap sort , Bubble sort, Selection sort, quick sort , shell sort, Insertion sort, Bucket sort • external sorting is done in external memory like hard disk or magnetic tape. • external use secondary memory • external sorting can take as much as large input • Egs : Mergesort,Tag Sorts ,Four Tape Sort Polyphase Sort ,ExternalRadixSort,External Merge.

17. Sort stability A sort algorithm is said to be “stable” if multiple items which compare as equal will stay in the same order they were in after a sort.

18. All the George names maintain their position, although Arthur is less than Peter Sorting Tom Greene Ann George Peter George Arthur George Joe Summers Ann George Peter George Arthur George Tom Greene Joe Summers Sorting on surnames Sort Stable Example:

19. In-place • In computer science, an in-place algorithm is an algorithm which transforms input using a data structure with a small, constant amount of extra storage space. • The input is usually overwritten by the output as the algorithm executes. • An algorithm which is not in-place is sometimes called not-in-place or out-of-place. • An algorithm is sometimes informally called in-place as long as it overwrites its input with its output.

20. Bubble sort • Compare each element (except the last one) with its neighbor to the right • If they are out of order, swap them • This puts the largest element at the very end • The last element is now in the correct and final place • Compare each element (except the last two) with its neighbor to the right • If they are out of order, swap them • This puts the second largest element next to last • The last two elements are now in their correct and final places • Compare each element (except the last three) with its neighbor to the right • Continue as above until you have no unsorted elements on the left

21. 2 2 2 7 2 2 2 2 2 2 2 2 2 4 7 4 7 2 7 4 5 7 5 7 5 7 5 8 5 8 5 5 4 7 5 8 4 5 5 7 4 4 8 5 5 5 7 4 7 4 7 7 8 8 8 8 4 8 4 4 4 8 8 8 8 2 5 4 7 8 Example of bubble sort Pass 1 Pass 2 Pass 3 Pass 4 (done)

22. Bubble sort C code Void bubblesort(int a[],int n) { int I,j,temp,f=1; for(i=1;i<n && f==1 ;i++) { f=0 for(j=0;j<n-i;j++) { if(a[j+1]<a[j]) { swap(a[j],a[j+1]) f=1 } } }

23. 7 2 2 2 2 4 2 4 7 4 8 8 8 5 5 5 5 8 5 7 8 7 4 7 4 Selection sort • The selection sort might swap an array element with itself--this is harmless, and not worth checking for • Analysis: • The outer loop executes n-1 times • The inner loop executes about n/2 times on average (from n to 2 times) • Work done in the inner loop is constant (swap two array elements) • Time required is roughly (n-1)*(n/2) • You should recognize this asO(n2)

24. Selection Sort C code void selectionsort(int arr[], int n) { int i,j,k,temp; for(i=0;i<n-1;i++) { k=i; for(j=i+1;j<n;j++) { if(arr[j]<arr[k]) k=j; } if(k!=i) { temp=arr[i]; arr[i]=arr[k]; arr[k]=temp; } } }

25. Insertion Sort • Idea: like sorting a hand of playing cards • Start with an empty left hand and the cards facing down on the table. • Remove one card at a time from the table, and insert it into the correct position in the left hand • compare it with each of the cards already in the hand, from right to left • The cards held in the left hand are sorted • these cards were originally the top cards of the pile on the table

26. 24 10 6 Insertion Sort To insert 12, we need to make room for it by moving first 36 and then 24. 36 12

27. 24 10 6 Insertion Sort 36 12

28. 24 36 Insertion Sort 10 6 12

29. Insertion Sort input array 5 2 4 6 1 3 at each iteration, the array is divided in two sub-arrays: left sub-array right sub-array unsorted sorted

30. Insertion Sort

31. Insert Action: i=1 temp 8 20 8 5 10 7 i = 1, first iteration 8 20 20 5 10 7 --- 8 20 5 10 7

32. Insert Action: i=2 temp 5 8 20 5 10 7 i = 2, second iteration 5 8 20 20 10 7 5 8 8 20 10 7 --- 5 8 20 10 7

33. Insert Action: i=3 temp 10 5 8 20 10 7 i = 3, third iteration 10 5 8 20 20 7 --- 5 8 10 20 7

34. Insert Action: i=4 temp 7 5 8 10 20 7 i = 4, forth iteration 7 5 8 10 20 20 7 5 8 10 10 20 7 5 8 8 10 20 --- 5 7 8 10 20

35. Insertion sort C code • void insertionsort(int a[],int n) • { • int i,k,temp; • for(k=1;k<n;k++) • { • temp=a[k]; • for(i=k-1;i>=0&&temp<x[i];i--) • a[i+1]=a[i]; • a[i+1]=temp; • } • }

36. Bucket sort Idea: BucketSort on each digit, least significant to most significant (lsd to msd)

37. Bucket sort Example (1st pass) Bucket sort by 1’s digit After 1st pass Input data 721 3 123 537 67 478 38 9 478 537 9 0 1 2 3 4 5 6 7 8 9 721 721 3 123 537 67 478 38 9 3 38 123 67

38. Bucket Sort Example (2nd pass) Bucket sort by 10’s digit After 1st pass After 2nd pass 3 9 721 123 537 38 67 478 721 3 123 537 67 478 38 9 0 1 2 3 4 5 6 7 8 9 03 09 721 123 537 38 67 478

39. Bucket Sort Example (3rd pass) Bucket sort by 100’s digit After 2nd pass After 3rd pass 3 9 721 123 537 38 67 478 3 9 38 67 123 478 537 721 0 1 2 3 4 5 6 7 8 9 003 009 038 067 123 478 537 721 Invariant: after k passes the low order k digits are sorted.

40. Quicksort Recursive algorithm consists of four steps: • If there is one or less element in the array to be sorted, return immediately. 2. Pick an element in the array to serve as a "pivot" point. (Usually the left-most element in the array is used.)

41. Quicksort • Split the array into two parts - one with elements larger than the pivot and the other with elements smaller than the pivot. 4. Recursively repeat the algorithm for both halves of the original array.

42. 26 22 12 29 19 35 33 0 0 1 1 2 2 3 3 4 4 5 5 6 6 19 22 12 26 29 35 33 29 12 35 19 33 22 0 1 4 5 6 2 33 35 5 6 Initial list ‘A': QS(A,0,6) • Select 26 as pivot • Place Pivot at A[3] QS(A,0,2) QS(A,4,6) Select 19 as pivot Place Pivot at A[1] Select 29 as pivot Place Pivot at A[4] QS(A,5,6) Select 35 as pivot Place Pivot at A[6] QS (A,2,2) QS(A,0,0) Done Done Done QS(A,5,5) Done

43. 26 22 12 29 19 35 33 j i 26 22 12 19 29 35 33 26 22 12 29 19 35 33 Swap i j Pivot j i 26 22 12 19 29 35 33 26 22 12 29 19 35 33 26 22 12 19 29 35 33 19 22 12 26 29 35 33 j i Pivot j i i j Pivot j 26 22 12 29 19 35 33 Right Subarray Left Subarray i j Pivot QS( A,0,6 ) Pivot Pivot Pivot Swap 26 22 12 29 19 35 33 Pivot j i

44. Quick sort Pros: Extremely fast. Cons: Very complex algorithm,massively recursive.

45. Merge Sort • MergeSort is a divide and conquer method of sorting

46. MergeSort Algorithm • MergeSort is a recursive sorting procedure that uses at most O(n lg(n)) comparisons. • To sort an array of n elements, we perform the following steps in sequence: • If n < 2 then the array is already sorted. • Otherwise, n > 1, and we perform the following three steps in sequence: • Sort thelefthalf of the the array using MergeSort. • Sort therighthalfof the the array using MergeSort. • Merge the sorted left and right halves.

47. An example Initial: 25 57 48 37 12 92 86 33 Split: 25 57 48 37 12 92 86 33 Split: 25 57 48 37 12 92 86 33 Split: 25 57 48 37 12 92 86 33 Merge: 25 57 37 48 12 92 33 86 Merge: 25 37 48 57 12 33 86 92 Merge: 12 25 33 37 48 57 86 92

48. MergeSort

49. Sorting comparison