Agenda

1. Agenda Introduction to Binary Trees Implementing Binary Trees Searching Binary Search Trees Tree Traversal …1. Breadth-First ….2. Depth-First Insertion Deletion by Copying Balanced Trees Heaps / Heap Sort

2. Lecture Outline • Heap Definition • Heapifying an Array • Heap Sort • Priority Queues

3. Readings • Definitions of heap: • Section 6.1, 6.2, 6.3, Heaps • Section 7.5, Heap Sort • Links: • Code http://www.mathcs.duq.edu/drozdek/DSinCpp/sorts.h • Animation http://www.cs.usfca.edu/~galles/visualization/HeapSort.htmlhttp://www.cs.usfca.edu/~galles/visualization/Heap.html

4. What is a “heap”? • Book Sections • A large area of memory from which the programmer can allocate blocks as needed, and deallocate them (or allow them to be garbage collected) when no longer needed • A balanced, left-justified binary tree in which no node has a value greater than the value in its parent • These two definitions have little in common • Heapsort uses the second definition

5. d-2 d-1 d Balanced Balanced Not balanced Balanced binary trees • Recall: • The depth of a node is its distance from the root • The depth of a tree is the depth of the deepest node • A binary tree of depth d is balanced if all the nodes at depths 0 through d-2 have two children

6. Left-justified Not left-justified Left-justified binary trees • A balanced binary tree is left-justified if: • all the leaves are at the same depth, or • all the leaves at depth d+1 are to the left of all the nodes at depth d

7. 1. Heap • A max heap is a binary tree with the following properties: • The value stored in each node is not less than the value stored in each of each children. • All levels are full, with the possible exception of the bottom level whose nodes are all in the leftmost positions. • A min heap is similar with every node is not great than its children. • Heap propertymeans meeting the first condition above.

9. Implementing Heaps A heap can be represented by an array. Nodes can ordered in the array cells from top to bottom from left to right. Node i will have its children at 2 * i+ 1 and 2 * i+ 2. heap[i]heap[2*i + 1]for 0i<(n-1)/2 heap[i]heap[2*i + 2]for 0i<(n-2)/2 The array is not ordered but the descendents along any path are ordered.

10. Heaps as Arrays Heap as array [20 18 16 13 10 15 3 12 11] 20 16 18 13

11. 3. Heapifying an Array • Sometimes we have an array and we need to convert it in-place to a heap. • Williams’ top-down approach. • Floyd’s bottom-up approach.

12. 2 8 8 8 2 2 6 8 8 10 2 6 2 6 8 6 1 1 1 Williams’ Algo 4 Heapifying an Array 2 ______ |_2_|__ _________ |_8_|_2_|__ ____________ |_8_|_2_|_6 |__ 1 2 3 5 10 2 ___________________ |_8_|_10_|_6 |_1 |_2 |__ ___________________ |_8_|_2_|_6 |_1 |_10 |__ 4 _______________ |_8_|_2_|_6 |_1 |__ ___________________ |_10_|_8_|_6 |_1 |_2 |__

13. Williams’ Algo 4 Heapifying an Array

14. Heapify • Heapify picks the largest child key and compare it to the parent key. If parent key is larger than heapify quits, otherwise it swaps the parent key with the largest child key. So that the parent is now becomes larger than its children. Heapify(A, i) {        l  left(i)        r  right(i)        if l <= heapsize[A] and A[l] > A[i]            then largest l            else largest i        if r <= heapsize[A] and A[r] > A[largest]            then largest  r        if largest != i            then swap A[i]  A[largest]Heapify(A, largest) }

15. Build Heap • We can use the procedure 'Heapify' in a bottom-up fashion to convert an array A[1 . . n] into a heap. Since the elements in the subarray A[n/2 +1 . . n] are all leaves, the procedure BUILD_HEAP goes through the remaining nodes of the tree and runs 'Heapify' on each one. The bottom-up order of processing node guarantees that the subtree rooted at children are heap before 'Heapify' is run at their parent. Buildheap(A) {heapsize[A] length[A]        for i|length[A]/2 //down to 1            do Heapify(A, i) } Chapter 6: Binary Trees

16. Floyd’s Algo 4 Heapifying an Array • It does better than William’s • It works bottom-up • Small heaps are build and then merged Floyed Algo (data[]) for (i = index of last non-leaf; i >= 0 ; i--) restore heap property for tree whose root is data[i] by calling moveDown (data, i, n-1)

17. 1 12 15 12 12 6 11 11 11 11 3 3 15 1 1 1 6 12 Floyd’s Algo 4 Heapifying an Array _________________________________ |_2_|_8_|_6 |_1 |_10 |_ 15 | 3 | 12 |_11 | _________________________________ |_2_|_8_|_6 |_12|_10 |_ 15 | 3 | 1 |_11 | _________________________________ |_2_|_8_|_6 |_12 |_10 |_ 15 | 3 | 1 |_11 | _________________________________ |_2_|_8_|_15 |_12|_10 |_ 6 | 3 | 1 |_11 |

18. 8 12 11 15 15 12 3 11 8 3 6 1 6 1 Floyd’s Algo 4 Heapifying an Array 10 10 _________________________________ |_2_|_ 8 |_15|_12 |_10 |_ 6 | 3 | 1 |_11 | _________________________________ |_2_|_12_|_15 |_11 |_ 10 |_ 6 | 3 | 1 |_8|

19. 2 12 12 15 11 15 6 11 8 3 8 3 2 1 6 1 Floyd’s Algo 4 Heapifying an Array 10 10 _______________________________ |_2_|_12_|_15 |_11 |_ 10 |_ 6 | 3 | 1 |_8| _______________________________ |_15_|_12_|_6 |_11 |_ 10 |_ 2 | 3 | 1 |_8|

20. Analysis of Floyd’s Algorithm • Assume a complete binary tree n = 2k - 1 • Move Down is called (n-1)/2, once for each non-leaf • In worst case, Move Down moves the next node from next level to last one (to be a leaf). • From second to last level to last (leaves) requires (n+1)/4 moves. • From third to last to leaves requires 2 moves x (n+1)/8 nodes. • For the next level, it is 3 x (n+1)/16 moves.

21. Analysis of Floyd’s Algorithm • This is (n+1) ∑ (1/2) (k / 2k) for k = 1 to (log (n+1))-1 • But ∑ (k / 2k) for k = 1 to ∞ converges to 2 • Hence as n goes to ∞, this amount will be (n+1)  O(n)

22. 4. Heap Sort • Merge sort time is O(n log n) but still requires, temporarily, n extra storage items • Heapsort does not require any additional storage • Quick sort is O(n log n) in the average case but is still O(n2) in worst case (which is rare in practice) • Heap sort is + O (n log n) in worst case + In-place • Constant is large and is slower than Qsort in practice. • Not a stable algorithm • Not suitable for external sort / parallelization

23. Why study Heapsort? • It is a well-known, traditional sorting algorithm that you are expected to know • Heapsort is always O(n log n) • Quicksort is usually O(n log n) but in the worst case slows to O(n2) • Quicksort is generally faster, but Heapsort is better in time-critical applications • Heapsort is a really cool algorithm!

24. A node has the heap property if the value in the node is as large as or larger than the values in its children All leaf nodes automatically have the heap property A binary tree is a heap if all nodes in it have the heap property 12 12 12 8 3 8 12 8 14 Blue node has heap property Blue node has heap property Blue node does not have heap property The (Max)heap property Chapter 10: Sorting

25. Given a node that does not have the heap property, you can give it the heap property by exchanging its value with the value of the larger child This is sometimes called sifting up Notice that the child may have lost the heap property 14 12 8 12 8 14 Blue node has heap property Blue node does not have heap property siftUp

26. Add a new node here Add a new node here Constructing a heap I (William) • A tree consisting of a single node is automatically a heap • We construct a heap by adding nodes one at a time: • Add the node just to the right of the rightmost node in the deepest level • If the deepest level is full, start a new level • Examples:

27. Constructing a heap II • Each time we add a node, we may destroy the heap property of its parent node • To fix this, we sift up • But each time we sift up, the value of the topmost node in the sift may increase, and this may destroy the heap property of its parent node • We repeat the sifting up process, moving up in the tree, until either • We reach nodes whose values don’t need to be swapped (because the parent is still larger than both children), or • We reach the root

28. 8 10 10 10 8 8 5 10 10 12 8 5 12 5 10 5 12 8 8 8 1 2 3 4

29. The node containing 8 is not affected because its parent gets larger, not smaller The node containing 5 is not affected because its parent gets larger, not smaller The node containing 8 is still not affected because, although its parent got smaller, its parent is still greater than it was originally 10 10 14 14 12 12 12 14 10 5 5 5 8 8 8 Other children are not affected

30. Here’s a sample binary tree after it has been heapified Notice that heapified does not mean sorted Heapifying does not change the shape of the binary tree; this binary tree is balanced and left-justified because it started out that way 25 22 17 19 22 14 15 18 14 21 3 9 11 A sample heap

31. Notice that the largest number is now in the root Suppose we discard the root: How can we fix the binary tree so it is once again balanced and left-justified? Solution: remove the rightmost leaf at the deepest level and use it for the new root 11 22 17 19 22 14 15 18 14 21 3 9 11 Removing the root

32. Our tree is balanced and left-justified, but no longer a heap However, only the root lacks the heap property We can siftUp() the root After doing this, one and only one of its children may have lost the heap property 11 22 17 19 22 14 15 18 14 21 3 9 The reHeap method I

33. Now the left child of the root (still the number 11) lacks the heap property We can siftUp() this node After doing this, one and only one of its children may have lost the heap property 22 11 17 19 22 14 15 18 14 21 3 9 The reHeap method II

34. Now the right child of the left child of the root (still the number 11) lacks the heap property: We can siftUp() this node After doing this, one and only one of its children may have lost the heap property —but it doesn’t, because it’s a leaf 22 22 17 19 11 14 15 18 14 21 3 9 The reHeap method III

35. Our tree is once again a heap, because every node in it has the heap property Once again, the largest (or a largest) value is in the root We can repeat this process until the tree becomes empty This produces a sequence of values in order largest to smallest 22 22 17 19 21 14 15 18 14 11 3 9 The reHeap method IV

36. Sorting • What do heaps have to do with sorting an array? • Here’s the neat part: • Because the binary tree is balanced and left justified, it can be represented as an array • All our operations on binary trees can be represented as operations on arrays • To sort: • heapify the array; • while the array isn’t empty { • remove and replace the root; • reheap the new root node;}

37. 25 22 17 19 22 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 18 14 21 3 9 11 25 22 17 19 22 14 15 18 14 21 3 9 11 Mapping into an array • Notice: • The left child of indexiis at index2*i+1 • The right child of index i is at index2*i+2 • Example: the children of node 3 (19) are 7 (18) and 8(14) Chapter 10: Sorting

38. The “root” is the first element in the array The “rightmost node at the deepest level” is the last element Swap them... ...And pretend that the last element in the array no longer exists—that is, the “last index” is 11 (9) 0 1 2 3 4 5 6 7 8 9 10 11 12 0 1 2 3 4 5 6 7 8 9 10 11 12 25 11 22 22 17 17 19 19 22 22 14 14 15 15 18 18 14 14 21 21 3 3 9 9 25 11 Removing and replacing the root

39. Reheap the root node (index 0, containing 11)... ...And again, remove and replace the root node Remember, though, that the “last” array index is changed Repeat until the last becomes first, and the array is sorted! 0 1 2 3 4 5 6 7 8 9 10 11 12 0 1 2 3 4 5 6 7 8 9 10 11 12 0 1 2 3 4 5 6 7 8 9 10 11 12 22 9 11 22 22 22 17 17 17 19 19 19 22 22 21 14 14 14 15 15 15 18 18 18 14 14 14 11 21 21 3 3 3 22 9 9 25 25 25 Reheap and repeat

40. C++ Implementation // Surprisingly Simple template<class T> void heapSort(T data[], const int n) { int i; for (i = n/2 - 1; i >= 0; --i)// Floyed moveDown (data,i,n-1); for (i = n-1; i >= 1; --i) { swap(data[0],data[i]); // Sort 1 item moveDown(data,0,i-1); // Reheap } }

41. Algorithm Analysis template<class T> void heapSort(T data[], const int n) { int i; for (i = n/2 - 1; i >= 0; --i) moveDown (data,i,n-1); for (i = n-1; i >= 1; --i) { swap(data[0],data[i]); moveDown(data,0,i-1); } } O(n) n – 1 swaps log 1 + log 2 + …..

42. Algorithm Analysis • Heapifying the array with Floyd’s algorithm is O(n). • Removing the root of the heap to put it in its place occurs n – 1 times, which is also O(n).

43. Algorithm Analysis • To reheap the array after each element, in the worst case, we will need to do moves down: log 1 + log 2 + ……..+ log (n-1) times • This is ∑ log i, for i = 1 to n-1 • Which is log n! ≤ log nn • Hence, reheaping is O(n log n) • Hence Heap Sort O (n + n + n log n) • Hence Heap Sort O (n log n) • This is the worst case performance.

44. Comparing Heap, Quick & Merge Sort • What is the adventage of Heap Sort over Merge Sort, if both are O(nlogn)? • What is the adventage of Heap Sort over Quick Sort, if both are in-place algorithms and are O(nlogn)in average case? • What are the disadventages of Heap Sort compared to the other two sorts?

45. Priority Q Potential Implementations O(1)/O(N)worst-array full, should say WHY, might reject on full instead. O(N) – to find value O(1) O(N) – to find value O(log N) to find loc w. Bin search, but O(N) to move vals O(1) to find val, but O(N) to move vals, (or O(1) if in reverse order) O(N) to find loc, O(1) to do the insert O(1) O(N) O(N) Plus – good memory usage O(log N)close to O(1) 1.67 levels on average O(log N) Binary Heap