Probabilistic Analysis and Randomized Algorithm

1. Probabilistic Analysis and Randomized Algorithm

2. Average-Case Analysis • In practice, many algorithms perform better than their worse case • The average case is analyzed by • Construct a probabilistic model of their input • Determine the probabilities and running times (or costs) of alternate executions • Calculate expected running time (or cost) COT5407

3. Example 1 COT5407

4. Probabilistic Model • Assume A has n distinct numbers. (What is the effect of duplicates?) • Assume each permutation of the numbers is equally likely (How can we guarantee this? through randomization) • How many permutations are there? • What is the probability of the best case? • What is the probability of the worst case? COT5407

5. Example 1: Analysis COT5407

6. Randomized Algorithm • We might not know the distribution of inputs, or we might not be able to model it computationally • Instead we use randomization within the algorithm in order to impose a distribution on the inputs • An algorithm is randomized if its behavior is determined in part by values produced by a random-number generator • How to compute a shuffle? COT5407

7. Goal • Input: Given n items to shuffle (cards, …) • Output: Return some list of exactly those n items; all n! lists should be equally likely. • Not the same as saying “each card is equally likely at each position!” Why not? • Possible methods? • Swap a pair of randomly chosen cards? • Choose keys and sort? • Swap each card with a randomly chosen card?

8. Choose key and sort • Book suggests: Assign each card a key number from [1..K]. Sort keys to permute cards • What is the probability that… • the second card gets the same key as the first? 1/K • the third gets the first or second, assuming that the first and second have different keys? 2/K • That we have some duplicate key among n cards? • 1/K + 2/K + … + n/K = n(n+1)/(2K) • Choose K = n^3 and the probability is < 1/n • Expected time: T(n) = O(n lg n) + T(n)/n = O(n lg n).

9. Random Shuffle? • Goal:uniform random permutation of an array. • RANDOM(n) – returns an integer 1 r  n with each of the n values of r being equally likely. • In iteration i, choose A[i] randomly from A[1..?]. • A[i] is never altered after iteration i. • Running Time:O(n) Shuffle(A) nlength[A] fori ndownto 2 do swap A[i] ↔ A[RANDOM(?)] n? i? (i-1)?

10. Finding the correct shuffle Shuffle(A) nlength[A] fori ndownto 2 do swap A[i] ↔ A[RANDOM(?)] • (i-1) forces changein each element. • n has nn-1 possibleoutcomes, but sincen! does not divide nn-1, some must occur more frequently than others. • i works … we should prove it.

11. Proving the shuffle correct Shuffle(A) nlength[A] fori ndownto 2 do swap A[i] ↔ A[RANDOM(i)] • Consider the random numbers chosen by a run of the algorithm:RANDOM(n), RANDOM(n-1), …, RANDOM(2), RANDOM(1) • Choices are independent: n·(n-1) ···2·1 = n! choices • We have chosen one uniformly at random. • Claim: Each choice produces to a unique permutation • By running algorithm, choices determine the permutation

12. Random Shuffle • Goal:uniform random permutation of an array. • RANDOM(n) – returns an integer 1 r  n with each of the n values of r being equally likely. • In iteration i, choose A[i] randomly from A[1..i]. • A[i] is never altered after iteration i. • Running Time:O(n) Shuffle(A) nlength[A] fori ndownto 1 do swap A[i] ↔ A[RANDOM(i)]