Chapter 12&13: Binary Search Trees (BSTs)

1. Chapter 12&13: Binary Search Trees (BSTs) Overview: Definition of dynamic sets Operations on dynamic sets Definition of BST Examples of balanced and unbalanced BSTs support dynamic-set operations T(n) proportional to height of tree Extensions, like Red-Black trees, ensure balance

2. Basic Definitions dynamic sets = sets that change due to manipulations performed by algorithms. dictionary = dynamic set that supports insertion, deletion, and membership testing. (priority queue is a dictionary) elements of dynamic set = objects with fields that can be examined and manipulated key = a field of an object that identifies it For 2 keys a and b either a<b, a=b, or a>b) allows relationships to be defined between set members. (minimum, next largest, etc.) satellite data = fields that are not used in set manipulations queries = operations on sets that only extract information

3. Operations on dynamic sets Search(S,k) = query that returns a pointer x if there exist an element of S such that key[x] = k. Otherwise returns NIL Insert(S,x) = add element with pointer x to S Delete(S,x) = removes element with pointer x from S Minimum(S) = query that returns a pointer to the element of S with the smallest key Maximum(S) = query that returns a pointer to the element of S with the largest key Successor(S,x) = query that returns a pointer to the element with a key that is the next larger than the key of element with pointer x Predecessor(S,x) = query that returns a pointer to the element with a key that is the next smaller than the key of element with pointer x

4. Binary Search Trees Binary search trees (BSTs) are data structures that support dictionary operations (Search, Insert and Delete) as well as Minimum, Maximum, Successor, and Predecessor Running time is proportional to h = height of tree Balanced binary tree with n nodes has h ~ lg n. Extreme case of unbalanced tree is chain of nodes  h = n Performance of a BST can be as good as O(lg n) or as bad as O(n) Red-Black BSTs are one of many “balanced” search-tree schemes that guarantee O(lg n) time for dynamic-set operations in the worst case

5. Binary-Search-Tree Property BST is a linked data structure Each node represents an object with a key (displayed) and satellite data (not shown) Node also contains fields left, right and p that point to left child, right child and parent, respectively (Contain NIL if a child or parent is missing) Root is only node with parent missing

6. Examples of BSTs 5 3 2 Keys of BST are sorted to satisfy the BST property: For any node x, keys in left subtree of x are no larger than key[x] and keys in right subtree of x are no smaller than key[x] 5

7. Searching for the node x that contains key k Recursive-tree-search(x,k) if x = NIL or k = key[x] return x (boundary condition) if k < key[x] then return Recursive-Tree-Search(left[x],k) else return Recursive-Tree-Search(right[x],k) end if Recursive calls direct search until boundary condition is met Iterative-Search-Tree(x,k) while x  NIL and k  key[x] doif k < key[x] then x  left[x] else x  right[x] return x

8. Find node x with the smallest key Tree-Minimum(x) while left[x]  NIL do x  left[x] return x Print keys in subtree at x in sorted order Inorder-Tree-Walk(x) if x  NIL then Inorder-Tree-Walk(left[x]) print key[x] Inorder-Tree-Walk(right[x]) Recursive calls direct algorithm to keys with smaller values

9. Find the successor of node x in an inorder-tree-walk (if keys are distinct, this is the node with the smallest key > key[x]) Tree-Successor(x) if right[x]  NIL then return Tree-Minimum(right[x]) (If the right[x] is not empty, then successor of x is its left-most node) y  p[x] while y  NIL and x = right[y] do x  y y  p[y] return y (Walk up the tree to find the lowest ancestor that whose left child is also an ancestor of x. Ancestors of x are all nodes above x in the path that leads to the root) A similar procedure finds the predecessor of x: node with the largest key < key[x]

10. Walk up the tree to find the lowest ancestor that whose left child is also an ancestor of x. Successor node with key=15 is node with key=17 (min of right subtree) Successor node with key=6 is node with key=7 (min of right subtree) Successor node with key=4 is node with key=6 Successor node with key=13 is node with key=15 Predecessor of node with key=6 is node with key=4 (max in left subtree)

11. Insertion and Deletion When a nodes is added and deleted (as in operation of a priority queue), the binary search tree property must be maintained In algorithm Tree-Insert(T,z), T denotes the tree to which node z is add The key[z] determines where z should be placed in the tree z will be inserted as a leaf to some node y = p(z)  left[z] = right [z] = NIL After finding y, update the fields of y depending on whether z is left or right child

12. Tree-Insert(T,z) x  root[T] (x will trace a path down tree to the place where z is inserted) y  NIL (y is maintained as the parent of x) while x  NIL (fails when place to insert z has been found) do y  x (candidate for p(z)) if key[z] < key[x] (get new assignment of x) then x  left[x] else x  right[x] (now we have found the parent of z) p[z]  y if y = NIL (T was empty when z was inserted) then root[T]  z else if key[z] < key[y] then left[y]  z else right[y]  z

13. Example: Insert a node with key = 13 13

14. Tree-Delete(T, z) has 3 cases case 1: z has no children modify p(z) by replacing z by NIL case 2: z has one child splice out z by making a link between p(z) and it’s child case 3: z has two children splice out z’s successor, y. copy key[y] into z

15. Case 1: delete 13 No children Case 2: delete 16 1 child Case 3: delete 5 2 children

16. Red-Black Trees Red-Black Tree is a BST with one extra field per node that denotes whether it is “red” or “black” By constraining the way nodes are colored we can ensure that no path from the root to a leaf is more than twice as long as any other (i.e. balanced tree) All leaves are black and do not hold keys Leaves are also called external nodes May be drawn as single node labeled NIL black-height of node x = number of black nodes in any path between node x and a leaf. The leaf, which is black, is included in the count The node x, which may be red, is not included in the count

17. R-B BST with black heights shown Same R-B tree with “sentinel” node All leaves and parent of root

18. Properties of red-black trees: 1. Every node is either red or black 2. Root is black 3. nil[T] is black 4. If node is red, its children are black 5. All paths from any node to nil[T] have the same number of black nodes. bh same as before h(x) = number of edges in longest path to leaf h(T) = h(root)

19. Lemma 13.1: A red-black tree with n internal nodes (not counting leaves) has a height of at most 2lg(n+1) Implications of Lemma 13.1: Red-black trees can be used to implement dynamic-set operations Search, Minimum, Maximum, Successor, and Predecessor with running time O(lg n) Tree-Insert and Tree-Delete will also run on red-black trees in O(lg n); however, they may not preserve the red-black tree properties

20. Proof by induction that subtree at x has at least 2bh(x) – 1 nodes Base: x = nil[T], height = 0, bh(x) = 0, 2bh(x) - 1 = 0 Node x with height > 0 has two children, only one of which can be red Red child has bh(x). Black child has bh(x) –1 Assume the subtress at these children have at least 2bh(child) -1 nodes • tree rooted at x has at least 2(2bh(x) -1 – 1) + 1 = 2bh(x) - 1 nodes Calculation is for 2 black children (add one for root) If one of children is red have more nodes Use to prove Lemma 13.1

21. Lemma 13.1: A red-black tree with n internal nodes (not counting leaves) has a height of at most 2lg(n+1) By property (4), at least half of the nodes on the path between the root and nil[T] (excluding the root itself) are black. (i.e. bh(root) is at least h/2) Number of nodes in tree, n, at least 2bh(root) – 1 (previous slide) n > 2h/2 – 1 n+1 > 2h/2 lg(n +1) > h/2 h < 2 lg(n+1)

22. To restore red-black tree properties after Insert or Delete, we must change color of some nodes and adjust pointer structure Rotation (only update algorithm we will discuss): Local modification that changes the root of a subtree while preserving the BST property. Left-Rotate(T,x) replaces the root of subtree at x by its right child. x becomes y’s left child so x stays to the left of y y’s left child becomes x’s right child so it stays right of x Right-Rotate(T,y) replaces the root of subtree at y by its left child.

23. Example of left rotations in BST x y y x

24. CptS 450 Spring 2014 [All problems are from Cormen et al, 3rd Edition] Homework Assignment 9: due 4/16/14 1. ex 12.2-5 p 293 proof by contradiction 2. ex 12.3-3 p 299 write pseudo-code and analyze 3. ex 13.1-5 p 312 analyze in terms of h(x) and bh(x)