Lecture 3: Randomized Algorithm and Divide and Conquer II:

1. Lecture 3:Randomized Algorithm and Divide and Conquer II: Shang-Hua Teng

2. Randomized Algorithm • Random Number Generator (RNG) • RANDOM(a,b) returns an integer from {a,a+1,…, b-1,b}, with each such integer being equally likely. • Special case (random coin flip): RANDOM(0,1) chooses 0 and 1 with probability 1/2 each. • A Randomized Algorithm uses a random number generator • its behavior is determined by not only by its input but also the values chosen by RNG

3. Why Randomized Algorithms? • Efficiency • Simplicity • Reduction of the impact of bad cases!

4. The Selection Problem • Input:  Array A[1...n] of the elements in the an arbitrary order, and an index k • Output:  the kth smallest element in A[1..n]. • If k = 1, we are asking for the smallest element • If k = n, we are asking for the largest element • If k = n/2, we are asking for the median

5. Quick-Selection • Choose a random element, split the array, eliminate the fraction that does not contain the desired element and recursively apply the procedure. • Quick-Selection(A,1,n,k) • More generally, Quick-Selection(A,p,r,k) • A procedure that selects the kth smallest element of elements in sub-array A[p..r]

6. Quick-Selection(A,p,r,k) • Quick-Selection(A,p,r,k) • ifp =r and k = 1, do return A(p) • ifp< rthen • t = Partition(A,p,r,q) • if t = k do return A[t] • else if t > k doreturn Quick-Selection(A,p,t-1,k) • else if t < k do return Quick-Selection(A,t+1,r,k-t)

7. Partition(A,p,r,q)Suppose there are t-1 elements in A that is smaller than s = A[q]. Then return t and reorder A so thatA[p..p+t-1] < A[t] = s <= A[t+1..r] • Partition(A,p,r,q) • forj ptor-1 • do if • then • return i+1 SWAP(A[q],A[r])

8. Expected Time Complexity

9. Time Complexity • Let G(n) = E(T(n)), we have

10. Time Complexity • Conjecture: G(n) = O(n). • Use the substitution method • Assuming for all m < n, • Need to show

11. Time Complexity(if c>=24)

12. Quick-Sort • Quick-Sort(A,1,n) • Divide • t = Partition(A,1,n,q) • Conquer • Quick-Sort(A,1,t-1) • Quick-Sort(A,t+1,n)

13. Quick-Sort(A,p,r) • Quick-Sort(A,p,r) • ifp< rthen • t = Partition(A,p,r,q) • Quick-Sort(A,p,t-1) • Quick-Sort(A,t+1,r) • Theorem: Worst Case: • Theorem: Expected Case: O(nlg n)

14. Partition(A,p,r,q)Suppose there are t-1 elements in A that is smaller than s = A[q]. Then return t and reorder A so thatA[p..p+t-1] < A[t] = s <= A[t+1..r] • Partition(A,p,r,q) • forj ptor-1 • do if • then • return i+1 SWAP(A[q],A[r])

15. Complexity of Quick-Sort • The Element chosen by RANDOM is called thepivot element • Each number can be a pivot element at most once • So totally, at most n calls to Partition procedure • So the total steps is bounded by a constant factor of the number of comparisons in Partition.

16. Compute the totalnumber of comparison in calls to Partition • When does the algorithm compare two elements? • When does not the algorithm compare two elements? • Suppose (Z[1],Z[2],…,Z[n]) are the sorted array of elements in A • that is, Z[k] is the kth smallest element of A.

17. Compute the totalnumber of comparison in calls to Partition • The reason to use Z rather than A directly is that is it hard to locate elements in A during Quick-Sort because elements are moving around. • But it is easier to identify Z[k] because they are in a sorted order. • We call this type of the analysis scheme: the backward analysis.

18. Compute the totalnumber of comparisons in calls to Partition • Under what condition does Quick-Sort compare Z[i] and Z[j]? • What is the probability of comparison? • First: Z[i] and Z[j] are compared at most once!!! • Let Eij be the random event that Z[i] is compared to Z[j]. • Let Xij be the indicator random variable of Eij. • Xij = I{Z[i] is compared to Z[j]}

19. Compute the totalnumber of comparisons in calls to Partition • So the total number of comparisons is • We are interested in

20. Compute the totalnumber of comparisons in calls to Partition • By linearity of expectation, we have • So what is Pr(Z[i] is compared to Z[j])?

21. Compute the totalnumber of comparisons in calls to Partition • So what isPr(Z[i] is compared to Z[j])? • What is the condition that • Z[i] is compared to Z[j]? • What is the condition that • Z[i] is not compared to Z[j]? • Answer: no element is chosen fromZ[i+1] … Z[j-1] before Z[i] or Z[j] is chosen as a pivot in Quick-Sort • therefore ...

22. Compute the totalnumber of comparisons in calls to Partition • Therefore

23. Compute the totalnumber of comparisons in calls to Partition • By linearity of expectation, we have

24. Original Quick-Sort(Tony Hoare) • Partition with the first element • Average-Case Complexity: • Assume inputs come from uniform permutations. • Our analysis of the Expected time analysis of Random Quick-Sort extends directly. • Notice the difference of randomized algorithm and average-case complexity of a deterministic algorithm