Comprehensive Analysis of Algorithms Quick Select and Quick Sort

1. CS 330: AlgorithmsQuick Select Gene Itkis

2. QuickSelect: random divide & concur QSort(A[], F, L): • if F>L then return; • k  Partition(A[],F,L); • QSort(A[], F, k-1); • QSort(A[], k+1, L); ----------------------- • Worst case: • T(n)=O(n)+ T(n-1)  T(n)=O(n2) • Best Case: • T(n)=O(n)+ 2T(n/2)  T(n)=O(n lg n) • Average: ??? QSel(A[], F, L, r): • if F>L then return; • k  Partition(A[],F,L); • if k=r thenreturn A[k]; • if k>r then returnQSel(A[],F,k-1,r); • if k<r then returnQSel(A[],k+1,L,r-k); ----------------------- • Worst case: • T(n)=O(n)+ T(n-1)  T(n)=O(n2) • Best Case: • T(n)=O(n)+ T(n/2)  (or even on 1st call) T(n)=O(n) • Average: ??? Gene Itkis

3. QSort Analysis • Average performance = Expected Number of Compares • Notation: • zi = i-th smallest element of A[1..n] • Ci,j=Cj,i = the cost of comparing zi and zj • Pi,j = the probability of QSort comparing zi and zj • E[#compares]= all i,j>i Ci,j Pi,j =all i,j>i Pi,j • zi and zj are not compared if for some k: i<k<j, zk is chosen as a pivot before either zi or zj •  Pi,j=2/(j-i+1) Gene Itkis

4. QSel Analysis(same as QSort!) • Average performance = Expected # of Compares • Notation: • zi = i-th smallest element of A[1..n] • Ci,j=Cj,i = the cost of comparing zi and zj • Pi,j = the probability of QSel comparing zi and zj • E[#compares]= all i,j>i Ci,j Pi,j =all i,j>i Pi,j • zi and zj are not compared if for some k: i<k<j, zk is chosen as a pivot before either zi or zj • NOT as in QSort: zi and zj are alsonot compared if for some k: r<k<j, zk is chosen as a pivot before either zr or zj • Recall: zr is the element QSel is looking for Gene Itkis

5. QSel Analysis ? zi<zj • Thus, Pi,j = 2/(max{|i-j|, |i-r|, |j-r|} +1) • Let • j= r+D (-r<D<n-r) • i= r+D’ (|D’|<|D|) • Then Pi,j < 2/D zj zi zk zr D D’ For QSort: Pi,j < 2/d; d=|j-i| Gene Itkis

6. QSort Analysis • Thus • E[#compares]=all i,j>iPi,j= all i,j>i 2/(j-i+1)< all i,j>i 2/(j-i) • Let j=i+d, where 0<d  n-i. Then • E[#compares]< all i,j>i 2/(j-i)= all i,d2/d= = i=1..n d=1..n-i 2/d < i=1..nd=1..n 2/d  2i=1..nln n  1.4 n lg n Gene Itkis

7. QSel Analysis E[#compares] = all i,j>iPi,j =all i,d2/d < i=1..nd=1..n2/d  2i=1..nln n  1.4n lg n • Thus • E[#compares]=all i,j>i Pi,j < all D,D’ (|D’|<|D|) 2/D≤ -r<D<n-r D’= 1-|D|…|D|-12/D< -r<D<n-r2D2/D= -r<D<n-r 4= 4 n 2ndmight be “over-counting”: e.g. r=5, D=10, then D’=-4…9, not -9…9 Gene Itkis

8. Conclusion • Can find r-th smallest element in linear time: O(n) • E.g. median (useful for hw and tests ;-) • No need to sort – works faster without sorting • Murphy is not always right • Sometimes best case is more common than worst: • QSort • Worst case: O(n2); Best and Average: O(n lg n) • QSel • Worst case: O(n2); Best and Average: O(n) • Randomized algorithms are COOL & USEFUL! Gene Itkis

9. Hiring Problem • Another example of the above principles • Problem • N candidates come for a job (at large intervals) • You are the big boss • You want the best, but do not want to wait • Your hiring strategy – greedy: • Get the best you’ve seen so far (i.e. better than the current choice) • Assume: easy to compare candidates/workers • Each “fire current & hire new” procedure costs $1K • How much you will spend eventually? Gene Itkis

10. Hiring Problem • Best case: • Best candidate comes first • Cost: $1k • Probability: • 1/N • Worst case: • Best candidate comes last • Not enough – e.g. if 2nd best comes first, cost=$2k • Worst case: every candidate is hired => • Candidates come in increasing quality order • Cost: $Nk • Probability: • 1/(N!) • Average: ??? Gene Itkis

11. Expected Number of Hires • QSort/Qsel – expected # of compareshere: expected # of hires E(#hires) • Let Pi= probability that i-th candidate is hired • E(#hires) = all i $1k×Pi = $1k×all i Pi= • Pi=Prob[i-th hired] = Prob[i-th is best of 1..i] • Pi = 1/i • E(#hires) = $1k×all iPi = = $1k×i=1..N1/i   $1k× ln n  $0.7k lg n Gene Itkis