CS 261 – Data Structures

1. CS 261 – Data Structures CS261 Data Structures Priority Queue ADT & Heaps

2. Goals • Introduce the Priority Queue ADT • Heap Data Structure Concepts

3. Priority Queue ADT • Not really a FIFO queue – misnomer!! • Associates a “priority” with each element in the collection: • First element has the highest priority (typically, lowest value) • Applications of priority queues: • To do list with priorities • Active processes in an OS

4. Priority Queue ADT: Interface • Next element returned has highest priority void add(newValue + priority); TYPE getMin(); void removeMin();

5. Priority Queue ADT: Implementation Heap: has 2 completely different meanings • Classic data structure used to implement priority queues • Memory space used for dynamic allocation We will study the data structure (not dynamic memory allocation)

6. Priority Queue ADT: Implementation Binary Heap data structure: a complete binary tree in which every node’s value is less than or equal to the values of its children (min heap) Review: a complete binary tree is a tree in which • Every node has at most two children (binary) • The tree is entirely filled except for the bottom level which is filled from left to right (complete) • Longest path is ceiling(log n) for n nodes

7. Min-Heap: Example Root = Smallest element 2 3 5 9 10 7 8 14 12 11 16 Next open spot Last filled position (not necessarily the last element added)

8. Maintaining the Heap: Addition 2 Add element: 4 3 5 9 10 7 8 New element in next open spot. 4 14 12 11 16 Place new element in next available position, then fix it by “percolating up”

9. Maintaining the Heap: Addition (cont.) 2 3 5 2 9 10 4 8 3 4 14 12 11 16 7 After first iteration (swapped with 7) 9 10 5 8 14 12 11 16 7 After second iteration (swapped with 5) New value not less than parent  Done Percolating up: while new value is less than parent, swap value with parent

10. Maintaining the Heap: Removal • Since each node’s value is less than or equal to the values of its children, the root is always the smallest element • Thus, the operations getMin and removeMin access and remove the root node, respectively • Heap removal (removeMin): What do we replace the root node with? Hint: How do we maintain the completeness of the tree?

11. Maintaining the Heap: Removal Heap removal (removeMin): • Replace root with the element in the last filled position • Fix heap by “percolating down”

12. Maintaining the Heap: Removal Root = Smallest element • removeMin: • Move element in last • filled posinto root • 2. Percolate down 2 3 5 9 10 7 8 14 12 11 16 Last filled position

13. Maintaining the Heap: Removal (cont.) 16 3 5 3 9 10 7 8 16 5 14 12 11 Root value removed (16 copied to root and last node removed) 9 10 7 8 14 12 11 Percolating down: while greater than smallest child swap with smallest child After first iteration (swapped with 3)

14. Maintaining the Heap: Removal (cont.) 3 9 5 3 16 10 7 8 8 9 5 14 12 11 After second iteration (moved 9 up) 12 10 7 8 14 16 11 Percolating down: while greater than smallest child swap with smallest child After third iteration (moved 12 up) Reached leaf node  Stop percolating

15. Maintaining the Heap: Removal (cont.) Root = New smallest element 3 9 5 12 10 7 8 14 16 11 New last filled position

16. Practice Insert the following numbers into a min-heap in the order given: 54, 13, 32, 42, 52, 12, 6, 28, 73, 36

18. Practice Remove the minimum value from the min-heap 5 6 50 7 8 60 70

19. 5 6 50 7 8 60 70

20. Your Turn • Complete Worksheet: Heaps Practice

21. CS 261 – Data Structures CS261 Data Structures Heap Implementation

22. Goals • Heap Representation • Heap Priority Queue ADT Implementation

23. Dynamic Array Representation Complete binary tree has structure that is efficiently represented with an array (or dynamic array) • Children of node i are stored at 2i + 1 and 2i + 2 • Parent of node i is at floor((i - 1) / 2) a Root b c 0 a 1 b 2 c 3 d 4 e 5 f 6 7 d e f Why is this a bad idea if tree is not complete?

24. Dynamic Array Implementation (cont.) Big gaps where the level is not filled! a If the tree is not complete (it is thin, unbalanced, etc.),the DynArr implementation will be full of holes b c d e f 0 a 1 b 2 c 3 4 d 5 6 e 7 8 9 10 11 12 13 f 14 15

25. Heap Implementation: add • Where does the new value get placed to maintain completeness? • How do we guarantee the heap order property? • How do we compute a parent index? • When do we ‘stop’ • Complete Worksheet #33 – heapAdd( )

26. Write : addHeap void addHeap (structdyArray * heap, TYPE newValue) { }

27. Heap Implementation: removeMin void removeMinHeap(DynArr *heap){ intlast; last = sizeDynArr(heap) – 1; putDynArr(heap, 0, getDynArr(heap, last)); /* Copy the last element to the first */ removeAtDynArr(heap, last); /* Remove last element. */ _adjustHeap(heap, last , 0);/* Rebuild heap */ } First element (data[0]) Last element (last) Percolates down from Index 0 to last (not including last…which is one beyond the end now!) 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8 Last element (last) 3 9 5 5 7 14 8 12 9 11 10 16 11 0 7 1 3 2 4 4 10 6 8

28. Heap Implementation: removeMin 2 3 4 9 10 5 8 7 14 12 11 16 last 3 9 5 5 7 14 8 12 9 11 10 16 11 7 0 2 1 3 2 4 4 10 6 8

29. Heap Implementation: removeMin (cont.) 7 last = sizeDynArr(heap) – 1; putDynArr(heap, 0, getDynArr(heap, last)); /* Copy the last element to the first */ removeAtDynArr(heap, last); _adjustHeap(heap, last, 0); 3 4 9 10 5 8 New root 14 12 11 16 Last element (last) 3 9 5 5 7 14 8 12 9 11 10 16 11 0 7 1 3 2 4 4 10 6 8

30. Heap Implementation: _adjustHeap 7 _adjustHeap(heap, upTo , start); _adjustHeap(heap, last , 0); 3 4 9 10 5 8 Smallest child (min = 3) 14 12 11 16 last 3 9 5 5 7 14 8 12 9 11 10 16 11 0 7 1 3 2 4 4 10 6 8

31. Heap Implementation: _adjustHeap 3 _adjustHeap(heap, last , 1); 7 4 9 10 5 8 14 12 11 16 last 3 9 5 5 7 14 8 12 9 11 10 16 11 0 3 1 7 2 4 4 10 6 8 smallest child

32. Heap Implementation: _adjustHeap 3 current is less than smallest child so _adjustHeap exits and removeMin exits 7 4 9 10 5 8 14 12 11 16 last 3 9 5 5 7 14 8 12 9 11 10 16 11 0 3 1 7 2 4 4 10 6 8 smallest child

33. Write: _adjustHeap void _adjustHeap (structdyArray * heap, int max, intpos) { }

34. Recursive _adjustHeap void _adjustHeap(structDynArr *heap, int max, intpos) { intleftIdx = pos * 2 + 1; intrghtIdx = pos * 2 + 2; if (rghtIdx < max) { /* Have two children? */ /* Get index of smallest child (_minIdx). */ /* Compare smallest child to pos. */ /* If necessary, swap and call _adjustHeap(max, minIdx). */ } else if (leftIdx < max) { /* Have only one child. */ /* Compare child to parent. */ /* If necessary, swap and call _adjustHeap(max, leftIdx). */ } /* Else no children, we are at bottom  done. */ }

35. Useful Routines void swap(structDynArr *arr, inti, int j) { /* Swap elements at indices i and j. */ TYPE tmp = arr->data[i]; arr->data[i] = arr->data[j]; arr->data[j] = tmp; } intminIdx(structDynArr *arr, inti, int j) { /* Return index of smallest element value. */ if (compare(arr->data[i], arr->data[j]) == -1) return i; return j; }

36. Priority Queues: Performance Evaluation(ave) So, which is the best implementation of a priority queue? (What if wrote a getMin() for AVLTree ?)

37. Priority Queues: Performance Evaluation • Recall that a priority queue’s main purpose is rapidly accessing and removing the smallest element! • Consider a case where you will insert (and ultimately remove) n elements: • ReverseSortedVector and SortedList: Insertions: n * n = n2 Removals: n * 1 = n Total time: n2 + n = O(n2) • Heap: Insertions: n * log n Removals: n * log n Total time: n * log n + n * log n = 2n log n = O(n log n) How do they compare in terms of space requirements?

38. Your Turn • Complete Worksheet #33 - _adjustHeap( .. )

39. CS 261 – Data Structures CS261 Data Structures BuildHeap and Heapsort

40. Goals • Build a heap from an array of arbitrary values • HeapSort algorithm • Analysis of HeapSort

41. BuildHeap • How do we build a heap from an arbitrary array of values???

42. BuildHeap: Is this a proper heap? 3 8 5 6 7 9 8 11 9 10 10 2 0 12 1 7 2 5 4 3 6 4 12 7 5 8 3 6 4 9 11 10 2 Are any of the subtreesguaranteed to be proper heaps?

43. BuildHeap: Leaves are proper heaps 12 7 5 8 3 6 4 First non-leaf 9 11 10 2 3 8 5 6 7 9 8 11 9 10 10 2 0 12 1 7 2 5 4 3 6 4 11 Size = 11 Size/2 – 1 = 4

44. BuildHeap • How can we use this information to build a heap from a random array? • _adjustHeap: takes a binary tree, rooted at a node, where all subtrees of that node are proper heaps and percolates down from that node to ensure that it is a proper heap • void _adjustHeap(structDynArr *heap,intmax, intpos) Adjust up to (not inclusive) Adjust from

45. BuildHeap • Find the first non-leaf node,i, (going from bottom to top, right to left) • adjust heap from it to max • Decrement i and repeat until you process the root

46. BuildHeap: Leaves are proper heaps 12 7 5 8 3 6 4 First non-leaf 9 11 10 2 3 8 5 6 7 9 8 11 9 10 10 2 0 12 1 7 2 5 4 3 6 4 11 Size = 11 Size/2 – 1 = 4

47. HeapSort • BuildHeap and _adjustHeap are the keys to an efficient , in-place, sorting algorithm • in-place means that we don’t require any extra storage for the algorithm • Any ideas???

48. HeapSort • BuildHeap – turn arbitrary array into a heap • Swap first and last elements • Adjust Heap (from 0 to the last…not inclusive!) • Repeat 2-3 but decrement last each time through

49. HeapSort Simulation: BuildHeap 3 4 5 7 0 9 1 3 2 8 4 5 9 Size= 6 i=Size/2 – 1 = 2 3 8 4 5 7 i=2 _adjustHeap(heap,6,2)

50. HeapSort Simulation: BuildHeap 3 4 5 8 0 9 1 3 2 7 4 5 9 Size= 6 i=1 3 7 4 5 8 i=1 _adjustHeap(heap,6,1)