1 / 32

C HAPTER 2 Basic Concepts of Probability Theory

C HAPTER 2 Basic Concepts of Probability Theory. Prof. Sang-Jo Yoo sjyoo@inha.ac.kr http://multinet.inha.ac.kr. Random Experiments. Random Experiments An experiment in which the outcome varies in an unpredictable fashion when the experiment is repeated under the same conditions

brick
Télécharger la présentation

C HAPTER 2 Basic Concepts of Probability Theory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CHAPTER 2 Basic Concepts of Probability Theory Prof. Sang-Jo Yoo sjyoo@inha.ac.kr http://multinet.inha.ac.kr

  2. Random Experiments • Random Experiments • An experiment in which the outcome varies in an unpredictable fashion when the experiment is repeated under the same conditions • The specification of a random experiment must include an unambiguous statement of exactly what is measured or observed. • Example 2.1 • E1: Select a ball from an urn containing balls numbered 1 to 50 • E2: Select a ball from an urn containing balls numbered 1 to 4. Balls 1 & 2 are black and balls 3 & 4 are white. Note number and color. • E3: Toss a coin three times and note the sequence of heads and tails • E4: Toss a coin three times and note the number of heads • E7: Pick a number at random between zero and one • E12: Pick two numbers at random between zero and one • E13: Pick a number X at random between zero and one, then pick a number Y at random between zero and X

  3. Sample Space • Sample space of a random experiment • The set of all possible outcomes • Outcomes(or sample points) are mutually exclusive in the sense that they cannot occur simultaneously. • When we perform a random experiment, one and only one outcome occurs. • A sample space can be finite, countably infinite or uncountably infinite. • Example 2.2 • E1: Select a ball from an urn containing balls numbered 1 to 50 • E2: Select a ball from an urn containing balls numbered 1 to 4. Balls 1 & 2 are black and balls 3 & 4 are white. Note the number and color. • E3: Toss a coin three times and note the sequence of heads and tails • E4: Toss a coin three times and note the number of heads • E7: Pick a number at random between zero and one • E12: Pick two numbers at random between zero and one • E13: Pick a number X at random between zero and one, then pick a number Y at random between zero and X

  4. Sample Space • Toss a dice until a ‘six’ appears and count the number of times the dice was tossed. S = {1, 2, 3, …}; S is discrete and countably infinite (one-to-one correspondence with positive integers) • Pick a number X at random between zero and one, then pick a number Y at random between zero and X. S = {(x, y): 0 ≤ y ≤ x ≤ 1}; S is a continuous sample space. 4

  5. Events • Events • Outcomes that satisfies certain conditions • A subset of S • Certain event consists of all outcomes, always occurs • Null event contains no outcomes, never occurs • Event class is the events of interests: set of sets • Example 2.3 • E1: “An even-numbered ball is selected,” A1= ? • E2: “The ball is white and even-numbered,” A2= ? • E3: “The three tosses give the same outcome,” A3= ? • E4: “The number of heads equals the number of tails,” A4= ? • E7: “The number selected is non-negative,” A5= ? • E12: “The two numbers differ by less than 1/10,” A6= ?

  6. Review of Set Theory • Terminologies • U: universal set consists of all possible objects • Objects are called the elements or points of the set • Subset of B : A  B • Empty set:  • Equal: A=B • Union: AB={x: xA or x B} • Intersection: AB={x: xA and x B} • Disjoint (mutually exclusive): if AB=  • Complement: Ac={x: xA} • Relative complement (difference): A-B={x: xA and xB} • Notations

  7. Set Operations

  8. Properties of Set Operations • Commutative properties • Associative properties • Distributive properties • DeMorgan’s Rules

  9. Properties of Set Operations • De Morgan’s rules(A ∩ B)c = Ac ∪ Bc and (A ∪ B)c = Ac ∩ Bc • Proof of the second rule: • Suppose x ∈ (A ∪ B)c⇔ x is not contained in any ofthe events A and B⇔ x is contained in Ac and Bc⇔ x ∈ Ac ∩ Bc. • Proof of the first rule: • Based on the second rule, take A → Ac and B → Bc, we then have(Ac ∪ Bc)c = A ∩ B. • Taking complement on both sides, we obtain the first rule. 9

  10. The Axioms of Probability • Axioms 1 : • Axioms 2 : • Axioms 3 : • Axioms 3’ :

  11. Properties of Probability • Corollary 1 : • Corollary 2 : • Corollary 3 : • Corollary 4 : • Corollary 5 : • Corollary 6 : • Corollary 7 :

  12. Discrete Sample Space • Example 2.9 • An urn contains identical balls numbered 0.1,…,9. Randomly select a ball from the urn • A = “number of ball selected is odd,” • B = “number of ball selected is a multiple of 2,” • C = “number of ball selected is less than 5,”

  13. Continuous Sample Space • Continuous sample space • Outcome are numbers. • The events consist of intervals of the real line or rectangular regions in the plane. • Example 2.12: Pick a number between zero and one • Sample space S = ? • The probability that the outcome falls in a subinterval of A = the length of the subinterval • A = the event when the outcome is at least 0.3 away from the center of the unit interval. P [A] = ?

  14. Conditional Probability • Conditional probability • To determine whether two events A and B are related in the sense that knowledge about the occurrence of B alters the likelihood of occurrence of A.

  15. Example: Conditional Probability • Example 2.24 • A ball is selected from an urn containing two black balls (1, 2) and two white balls (3, 4). The sample space is S ={(1, b), (2, b), (3, w), (4, w)}. Assuming that the outcome are equally likely, find P [A|B], P [A|C], where A, B, and C are the following events: A = {(1, b), (2, b), (3, w), (4, w)}, “black ball selected” B = {(1, b), (2, b), (3, w), (4, w)}, “even-numbered ball selected’” C = {(1, b), (2, b), (3, w), (4, w)}, “number of ball is greater than 2” • Example Tossing 2 dice • Suppose the first die is ‘3’; given this information, what is the probability that the sum of the 2 dice equals 8? • There are 6 possible outcomes, given the first dice is ‘3’:(3, 1), (3, 2), (3, 3), (3, 4), (3, 5), and (3, 6). • The outcomes are equally probable. Hence, the probability that the sum is 8, given the first dice is ‘3’, is 1/6 . “” • Conditional Probability Note

  16. Theorem On Total Probability • Let B1 , B2 , …, Bn be mutually exclusive events whose union equals the sample space S. Any event can be represented as the union of mutually exclusive events:

  17. Example: Total Probability • Example 2.28 • A manufacturing process produces a mix of “good” memory chips and “bad” memory chips. The lifetime of good chips follows the exponential law with a rate of α. The life time of bad chips also follows the exponential law, but the rate of failure is 1000α. Suppose that the fraction of good chips is 1-p and of bad chips p. Find the probability that a randomly selected chip is still functioning after t seconds. • C = the event “chip still functioning after t seconds” • G = the event “chip is good” • B = the event “chip is bad”

  18. Example: Total Probability • Example • In a trial, the judge is 65% sure that Susan has committed a crime. Person F (friend) and Person E (enemy) are two witnesses who know whether Susan is innocent or guilty. • Person F is Susan’s friend and will lie with probability 0.25 if Susan is guilty. He will tell the true if Susan is innocent. • Person E is Susan’s enemy and will lie with probability 0.30 if Susan is innocent. Person E will tell the truth if Susan is guilty. • What is the probability that Person F and Person E will give conflicting testimony? • Solution • Let I and G be the two mutually exclusive events that Susan is innocent and guilty, respectively. Let C be the event that the two witnesses will give conflicting testimony. Find P[C] based on P[C|I] and P[C|G]. • By the law of total probabilities 18

  19. Bayes’ Rule • Let B1 , B2 , …, Bn be a partition of a sample space S. • Suppose that event A occurs. What is the probability of event Bj ?

  20. Example: Bayes’ Rule • Example 2.30 • A manufacturing process produces a mix of “good” memory chips and “bad” memory chips. Suppose that in order to “weed out” the bad chips, every chip is tested for t seconds prior to leaving the factory. • Find the value of t for which 99% of chips sent out to customers are good. • C = the event “chip still functioning after t seconds” • G = the event “chip is good” • B = the event “chip is bad” • We want to find the value of t for which • Example 2.26 & 2.29 (Note)

  21. Example: Bayes’ Rule • Example • A judge is 65% sure that a suspect has committed a crime. During the course of the trial, a witness convinces the judge that there is an 85% chance that the criminal is left-handed. If 23% of the population is left-handed and the suspect is also left-handed. With this new information,how certain should the judge be of the guilt of the suspect? • SolutionG = event that the suspect is guiltyI = event that the suspect is innocentL = event that the suspect is left-handed • Since {G, I} forms a partition of the sample space, by Bayes’ Theorem: 21

  22. Independence of Events • Event A is independent of B if knowledge of the occurrence of B does not alter the probability of A • Three events A, B, and C are independent if

  23. Properties of independence and mutual exclusiveness (1) If A ∩ B = φ, then P[A|B]= 0. A can never occur if B has occurred. (2) If A ≠ φ, B ≠ φ and A and B are independent, then A and B are not mutually exclusive. This is becauseP[A ∩ B]= P[A]P[B] ≠ 0. Remarks we have seen that if A and B are mutually exclusive, then A and B cannot be independent. (3) If A and B are independent and A ∩ B = φ, then either A = φ or B = φ or both. This is because0 = P[A ∩ B]= P[A] P[B]so that either P[A]= 0, P[B] = 0 or both are equal. 23

  24. Example: Independent Events • Example 2.31 • A ball is selected from an urn containing two black balls (1, 2) and two white balls (3, 4). The sample space is S = {(1, b), (2, b), (3, w), (4, w)}. Let events A, B, and C are defined as follows: A = {(1, b), (2, b)}, “black ball selected” B = {(2, b), (4, w)}, “even-numbered ball selected’” C = {(3, w), (4, w)}, “number of ball is greater than 2” Are events A and B independent? Are events A and C independent?  In general if two events have nonzero probability and are mutually exclusive, then they cannot be independent.

  25. Example: Independent Events Example Two numbers x and y are selected at random between zero and one. Let events A, B and C be defined by However, 25

  26. Sequential Experiment • Sequential of independent experiment • Suppose that a random experiment consists of performing experiments E1, E2, …, En. • Outcome of the experiment • Sample space of the experiment • Let A1 , A2 , …, An be events such that Ak concerns only the outcome of the kth sub-experiment. If the sub-experiments are independent, then

  27. Example: Sequential Events • Example 2.36 • Suppose that 10 numbers are selected at random from the interval [0, 1]. Find the probability that the first 5 numbers are less than ¼ and the last 5 numbers are greater than ½. Let x1, x2, …, x10 be the sequence of 10 numbers.

  28. Binomial Probability Law • Bernoulli trial • Performance an experiment once and nothing whether a particular event A occurs. • Outcome = “success” if A occurs, outcome = “fail” otherwise • Binomial probability law • Probability of k successes in n independent repetitions of a Bernoulli trial

  29. Example: Binomial Probability Law • Example 2.37 • Suppose that a coin is tossed three times. Assume that the tosses are independent and the probability of heads is p P [{HHH}] = P [{HHT}] = P [{HTH}] = P [{THH}] = P [{TTH}] = P [{THT}] = P [{HTT}] = P [{TTT}] = • Let k be the number of heads in three trials P [k=0] = P [k=1] = P [k=2] = P[ k=3] =

  30. Example: Binomial Probability Law • Example 2.39 Voice Communication System • Let k be the number of active speakers in a group of eight independent speakers. Suppose that a speaker is active with probability 1/3. • Find the probability that the number of active speakers is greater than six. • Example 2.40: Error Correction Coding • A communication system transmits binary information over a channel that introduces random bit errors with probability =10-3. the transmitter transmits each information bit three times and a decoder takes a majority vote of the received bits to decide on what the transmitted bit was. • Find the probability that the receiver will make an incorrect decision.

  31. Multinomial Probability Law • Let B1,B2, …, BM be a partition of the sample space S of some random experiment and P[Bj]=Pj. • The events are mutually exclusive • P1+P2+…+PM=1 • Suppose the n independent repetitions of the experiment are performed. • Let kj be the number of time event Bj occurs, then the vector (k1,k2,…,kM) specifies the number of times each of the event Bj occurs • Multinomial probability law • M=2: Binomial probability law

  32. Geometric Probability Law • A sequential experiment in which we repeat independent Bernoulli trials until the occurrence of the first success. • Let the outcome of this experiment be m. • Geometric probability law • The probability p(m), that m trials are required.

More Related