Randomized Algorithms CS648

1. Randomized AlgorithmsCS648 Lecture 2 Randomized Algorithm for Approximate Median Elementary Probability theory

2. This lecture was delivered at slow pace and its flavor was that of a tutorial. Reason:To show that designing and analyzing a randomized algorithm demands right insight and just elementary probability. Randomized Monte Carlo Algorithm forapproximate median

3. A simple probability exercise There is a coin which gives HEADS with probability ¼ and TAILS with probability ¾. The coin is tossed times. What is the probability that we get at least HEADS ? [Stirling’s approximation for Factorial: ]

4. Probability of getting “at least HEADSin tosses” Probability of getting at least heads: Using Stirling’s approximation Since, so … Inverse exponential in .

5. Approximate median Definition: Given an array A[] storing n numbers and ϵ > 0, compute an element whose rank is in the range [(1- ϵ)n/2, (1+ ϵ)n/2]. Best Deterministic Algorithm: • “Median of Medians” algorithm for finding exact median • Running time: O(n) • No faster algorithm possible for approximate median Can you give a short proof ?

6. ½ - Approximate medianA Randomized Algorithm Rand-Approx-Median(A) • Let k  c log n; • S  ∅; • For i=1 to k • x  an element selected randomly uniformly from A; • S  S U {x}; • Sort S. • Report the median of S. Running time: O(log n loglogn)

7. Analyzing the error probability of Rand-approx-median n/4 3n/4 Elements of A arranged in Increasing order of values Right Quarter Left Quarter When does the algorithm err ? To answer this question, try to characterize what will be a bad sample S ?

8. Analyzing the error probability of Rand-approx-median n/4 3n/4 Observation: Algorithm makes an error only if k/2 or more elements sampled from the Right Quarter(or Left Quarter). Median of S Elements of A arranged in Increasing order of values Left Quarter Right Quarter

9. Analyzing the error probability of Rand-approx-median n/4 3n/4 Elements of A arranged in Increasing order of values Pr[ An element selected randomly from A is from Right quarter] = ?? Pr[ Out of k elements sampled from A, at least k/2 are from Right quarter] = ?? for Right Quarter Left Quarter ¼ Exactly the same as the coin tossing exercise we did !

10. Main result we discussed Theorem: The Rand-approx-median algorithm fails to report ½ -approximate median from array A[1..] with probability at most. Homework: Design a randomized Monte Carlo algorithm for computing ϵ-approximate median of array A[1..] with running time O(log nloglogn) and error probability for any given constants ϵ and . [Do this homework sincerely without any friend’s help.]

11. Elementary probability theory(It is so simple that you underestimate its elegance and power)

12. Elementary probability theory(Relevant for CS648) • We shall mainly deal with discrete probability theory in this course. • We shall take the set theoretic approach to explain probability theory. Consider any random experiment : • Tossing a coin 5 times. • Throwing a dice 2 times. • Selecting a number randomly uniformly from [1..n]. How to capture the following facts in the theory of probability ? • Outcome will always be from a specified set. • Likelihood of each possible outcome is non-negative. • We may be interested in a collection of outcomes.

13. Probability Space Definition: Probability space associated with a random experiment is an ordered pair (Ω,P), where • Ωis the set of all possible outcomes of the random experiment • P : Ω R such that • P(ω) ≥ 0for each ωϵΩ Elements of Ω are called elementary events or sample points. Ω

14. Event in a Probability Space Definition:An event Ain a probability space (Ω,P) is a subset of Ω. The probability of event Ais defined as For sake of compact notation, we extend P for events as described above. A Ω

15. Exercises A randomized algorithm can also be viewed as a random experiment. • What is the sample space associated with Randomized Quick sort ? • What is the sample space associated with Rand-approx-median algorithm ?

16. An Important Advice In the following slides, we shall state well known equations (highlighted in yellow boxes) from probability theory. • You should internalize them fully. • We shall use them crucially in this course. • Make sincere attempts to solve exercises that follow.

17. Union of two Events Given two events A and B defined over a probability space (,P), what is P(AUB) ? Try to prove it by showing the following: Each ωϵAUB contributes exactly P(ω) in the right hand side. A Ω B P(AUB)= P(A) + P(B) P(A∩B)

18. Union of three Events Given three events A₁,A₂, A₃, defined over a probability space (,P), what is P(A₁ U A₂ U A₃) ? Try to prove this equation as well by showing the following: Each ωϵA₁ U A₂UA₃ contributes exactly P(ω) in the right hand side. A Ω B C P(A₁ U A₂UA₃) = P(A₁) + P(A₂) + P( A₃) P(A₁∩A₂) P(A₂∩A₃) P(A₁∩A₃) + P(A₁∩A₂∩A₃)

19. Exercises • For events ,…,defined over a probability space (,P), prove that P()= … ) • There are letters envelopes. For each letter, there is a unique envelope in which it should be placed. A careless postman places the letters randomly into envelopes (one letter in each envelope). What is the probability that no letter is placed correctly (into the envelope meant for it) ?

20. Conditional Probability Happening of some event influences the likelihood of happening of other events. This notion is formally captured by conditional probability as follows. Probability of event A conditioned on event B, compactly represented asP[A|B], means the following. Given that event B has happened, what is the probability that event A has also happened ? You might have seen and used the following equation for conditional probability. Can you give suitable reason to justify the validity of the above equation ? In particular, give justification for ] in numerator and ] in denominator in this equation. P[A|B] =

21. Exercises • A man possesses five coins, two of which are double-headed, one is double-tailed, and two are normal. He shuts his eyes, picks a coin at random, and tosses it. What is the probability that the lower face of the coin is a head ? He opens his eyes and sees that the coin is showing heads; what it the probability that the lower face is a head ? He shuts his eyes again, and tosses the coin again. What is the probability that the lower face is a head ? He opens his eyes and sees that the coin is showing heads; what is the probability that the lower face is a head ? He discards this coin, picks another at random, and tosses it. What is the probability that it shows heads ?

22. Partition of sample space and an “important Equation” A set of events ,…,defined over a probability space (,P) is said to induce a partition of if • = • =∅for all Given an event B, how can we express P(B) in terms of a given partition ? B Ω • P(B)= )

23. Exercises • There are sticks each of different heights. There are vacant slots arranged along a line and numbered from 1 to as we move from left to right. The sticks are placed into the slots according to a uniformly random permutation. A stick placed at th slot is said to be a dominating stick if its height is largest among all sticks placed in slots 1 to . Find the probability that th slot contains a dominating stick.

24. Independent Events Two events Aand B defined over a probability space (,P) are said to be independent if happening of one of them has no influence on the probability of the another event. Mathematically, it means that P(A|B)= P(A) and P(B|A)=P(B) The following equation also compactly captures independence of two events. Question: Can two independent events ever be disjoint ? P(A ∩ B)= P(A) · P(B)

25. Exercises • Two fair dice are rolled. Show that the event that their sum is 7 is independent of the score shown by the first die. • Let (,P) be a probability space where = {1,2,…,p} for a given prime number p, and each elementary event has probability 1/p. Show that if two events Aand B defined over (,P) are independent, then at least one of Aand Bis either ∅ or .