Introduction to probability

1. Introduction to probability

2. Probability • Probability – the chance that an uncertain event will occur (always between 0 and 1) Symbols: P(event A) = “the probability that event A will occur” P(red card) = “the probability of a red card” P(~event A) = “the probability of NOT getting event A” [complement] P(~red card) = “the probability of NOT getting a red card” P(A & B) = “the probability that both A and B happen” [joint probability] P(red card & ace) = “the probability of getting a red ace”

3. Assessing Probability 1. Theoretical probability—based on theory (a priori understanding of a phenomena) e.g.: theoretical probability of rolling a 2 on a standard die is 1/6 theoretical probability of choosing an ace from a standard deck is 4/52 theoretical probability of getting heads on a regular coin is 1/2 2. Empirical probability—based on empirical data e.g.: you toss an irregular die (probabilities unknown) 100 times and find that you get a two 25 times; empirical probability of rolling a two is 1/4 empirical probability of a major Earthquake (>=6.7 on the Richter scale) in Bay Area by 2032 is .62 (based on historical data) empirical probability of a lifetime smoker developing lung cancer is 15 percent (based on empirical data)

4. Probability example • Sample space: the set of all possible outcomes. For example, in genetics, if both the mother and father carry one copy of a recessive disease-causing mutation (d), there are three possible outcomes (the sample space): • child is not a carrier (DD) • child is a carrier (Dd) • child has the disease (dd). • Probabilities: the likelihood of each of the possible outcomes (always 0 P 1.0). • P(genotype=DD)=.25 • P(genotype=Dd)=.50 • P(genotype=dd)=.25. Note: mutually exclusive, exhaustive probabilities sum to 1.

5. Child’s outcome Father’s allele Mother’s allele P(DD)=.5*.5=.25 P(♂D=.5) P(♀D=.5) P(♂d=.5) P(Dd)=.5*.5=.25 P(♂D=.5) P(dD)=.5*.5=.25 P(♀d=.5) P(dd)=.5*.5=.25 ______________ 1.0 P(♂d=.5) Using a probability tree Mendel example: What’s the chance of having a heterozygote child (Dd) if both parents are heterozygous (Dd)? Rule of thumb: in probability, “and” means multiply, “or” means add

6. Conditional Probability: Read as “the probability that the father passes a D allele given that the mother passes a d allele.” Independence Intuitively, we understand that the mother’s and father’s outcomes are independent. The mother’s and father’s alleles are segregating independently. Intuitively, independence means that the probability of A happening does not change depending on whether B happens and vice versa. It’s easy to see on the probability tree: the father’s probabilities are the same regardless of which maternal branch you’re on: P(♂D/♀D)=.5 and P(♂D/♀d)=.5

7. Joint Probability: The probability of two events happening simultaneously. Marginal probability: This is the probability that an event happens at all, ignoring all other outcomes. Independence Formal definition: A and B are independent if and only if P(A&B)=P(A)*P(B) Formally, P(DD)=.25=P(D♂)*P(D♀)

8. Conditional probability Child’s outcome Marginal probability: mother Mother’s allele Joint probability P(DD)=.5*.5=.25 P(♀D=.5) P(Dd)=.5*.5=.25 P(dD)=.5*.5=.25 P(♀d=.5) P(dd)=.5*.5=.25 ______________ 1.0 Sum to get Marginal probability for the father On the tree Father’s allele P(♂D/ ♀D )=.5 P(♂d=.5) P(♂D=.5) P(♂d=.5)

9. Probabilities for cards • What is the probability of picking 2 aces from a full deck?

10. P(Ace=3/51) P(Ace=4/52) P(~Ace=48/51) P(Ace=4/51) P(~Ace=48/52) P(~Ace=47/51) On the tree Outcome Draw 2 Draw 1 ______________ 1.0

11. Outcome ______________ 1.0 On the tree Are the events (draw ace on the first and draw ace on the second) independent? Intuitively Probabilities change depending on the outcome of the first draw. Mathematically P(Ace1)=.0045+.0723 =.0768 (marginal)

12. Outcome ______________ 1.0 On the tree Are the events (draw ace on the first and draw ace on the second) independent? Intuitively Probabilities change depending on the outcome of the first draw. Mathematically P(Ace1)=.0045+.0723 =.0768 (marginal) P(Ace2) = .0045 + .0723 = .0768 (marginal)

13. Outcome ______________ 1.0 On the tree • Are the events (draw ace on the first and draw ace on the second) independent? Intuitively Probabilities change depending on the outcome of the first draw. Mathematically • P(Ace1)=.0045+.0723 =.0768 (marginal) • P(Ace2) = .0045 + .0723 = .0768 (marginal) • P(Ace1)* P(Ace2)=.0768*.0768 = .0059 • P(ace,ace)=.0045 (joint) • NOT independent!

14. Practice problem If HIV has a prevalence of 3% in San Francisco, and a particular HIV test has a false positive rate of .001 and a false negative rate of .01, what is the probability that a random person selected off the street will test positive?

15. Joint probability of being + and testing + Conditional probability: the probability of testing + given that a person is + Marginal probability of carrying the virus. P(test +)=.99 P(+)=.03 P(test - )= .01 P(test +) = .001 P(-)=.97 P(test -) = .999 Marginal probability of testing positive Answer P (+, test +)=.0297 P(+, test -)=.003 P(-, test +)=.00097 P(-, test -) = .96903 ______________ 1.0 P(test +)=.0297+.00097=.03067 P(+&test+)P(+)*P(test+) .0297 .03*.03067 (=.00092)  Dependent!

16. One of these has to be true (mutually exclusive, collectively exhaustive). They sum to 1.0. Law of total probability

17. Review Question 1 • What is the probability of rolling a total of 8 on two dice? • 1/36 • 8/36 • 1/6 • 5/36 • 4/6

18. Review Question 1 • What is the probability of rolling a total of 8 on two dice? • 1/36 • 8/36 • 1/6 • 5/36 • 4/6 36 ways to roll two die (6x6). 5 ways to get an 8: 5-3, 3-5, 4-4, 2-6, 6-2

19. Review Question 2 • What is the probability of rolling two sixes in a row? • 1/36 • 8/36 • 1/6 • 5/36 • 4/6

20. Review Question 2 • What is the probability of rolling two sixes in a row? • 1/36 • 8/36 • 1/6 • 5/36 • 4/6 P(6&6) = P(6)*P(6) by independence So P(6&6) = 1/6*1/6 = 1/36

21. Review Question 3 • The probability of getting a straight flush from a fair deck of 52 cards is an example of: • An empirical probability • A theoretical probability • None of the above • Both of the above

22. Review Question 3 • The probability of getting a straight flush from a fair deck of 52 cards is an example of: • An empirical probability • A theoretical probability • None of the above • Both of the above

23. Review Question 4 • Which of the following is an example of statistically independent events? • P(A)=.2; P(B)=.5; P(A&B) = .07 • P(A)=.2; P(B)=.5; P(A&B) = .10 • P(A)=.2; P(B)=.5; P(A&B) = .70 • P(A)=.2; P(B)=.5; P(A&B) = 0

24. Review Question 4 • Which of the following is an example of statistically independent events? • P(A)=.2; P(B)=.5; P(A&B) = .07 • P(A)=.2; P(B)=.5; P(A&B) = .10 • P(A)=.2; P(B)=.5; P(A&B) = .70 • P(A)=.2; P(B)=.5; P(A&B) = 0

25. Review Question 5 The “law of total probability” gives you: • A conditional probability • A marginal probability • A joint probability • All of the above • None of the above

26. Review Question 5 The “law of total probability” gives you: • A conditional probability • A marginal probability • A joint probability • All of the above • None of the above

27. Bayes’ rule

28. Bayes’ Rule derivation (optional) • Definition: Let A and B be two events with P(B)  0. The conditional probability of A given B is: The idea: if we are given that the event B occurred, the relevant sample space is reduced to B—eg, the P(B)=1 because we know B is true—and the probability of A is simply the proportion of B that’s also A (joint probability).

29. 40% chance of failing both 50% chance of failing the midterm 50% chance of failing the class Example: A A&B B Sample space

30. Once I’ve failed the midterm… A&B B

31. Once I’ve failed the midterm… Sample space is reduced to this 50%. A&B B What’s the probability of A now that we’re restricted to B? It’s the A&B proportion of B:

32. Bayes’ Rule derivation (optional) can be re-arranged to: and, since also:

33. From the “Law of Total Probability” Bayes’ Rule: OR

34. Bayes’ Rule: • Why do we care?? • Why is Bayes’ Rule useful?? • It turns out that sometimes it is very useful to be able to “flip” conditional probabilities. That is, we may know the probability of A given B, but the probability of B given A may not be obvious. An example will help…

35. Practice Problem • If HIV has a prevalence of 3% in San Francisco, and a particular HIV test has a false positive rate of .001 and a false negative rate of .01, what is the probability that a random person who tests positive is actually infected (also known as “positive predictive value”)?

36. P (+, test +)=.0297 P(test +)=.99 P(+)=.03 P(test - = .01) P(+, test -)=.003 P(test +) = .001 P(-, test +)=.00097 P(-)=.97 P(-, test -) = .96903 P(test -) = .999 ______________ 1.0 Answer: using probability tree A positive test places one on either of the two “test +” branches. But only the top branch also fulfills the event “true infection.” Therefore, the probability of being infected is the probability of being on the top branch given that you are on one of the two circled branches above.

37. Equivalently, using Bayes’ rule

38. In-class exercise (in groups of 2-3) An insurance company believes that drivers can be divided into two classes—those that are of high risk and those that are of low risk. Their statistics show that a high-risk driver will have an accident at some time within a year with probability .4, but this probability is only .1 for low risk drivers. • Assuming that 20% of the drivers are high-risk, what is the probability that a new policy holder will have an accident within a year of purchasing a policy? • If a new policy holder has an accident within a year of purchasing a policy, what is the probability that he is a high-risk type driver?

39. Answer to (a) Assuming that 20% of the drivers are of high-risk, what is the probability that a new policy holder will have an accident within a year of purchasing a policy? Use law of total probability: P(accident)= P(accident/high risk)*P(high risk) + P(accident/low risk)*P(low risk) = .40(.20) + .10(.80) = .08 + .08 = .16

40. P(accident, high risk)=.08 P(accident/HR)=.4 P(high risk)=.20 P( no acc/HR)=.6 P(no accident, high risk)=.12) P(accident/LR)=.1 P(accident, low risk)=.08 P(low risk)=.80 P( no accident/LR)=.9 P(no accident, low risk)=.72 ______________ 1.0 Answer to (b) If a new policy holder has an accident within a year of purchasing a policy, what is the probability that he is a high-risk type driver? P(high-risk/accident)= P(accident/high risk)*P(high risk)/P(accident) =.40(.20)/.16 = 50% Or use tree: P(high risk/accident)=.08/.16=50%

41. The Risk Ratio and the Odds Ratio as conditional probability In epidemiology, the association between a risk factor or protective factor (exposure) and a disease may be evaluated by the “risk ratio” (RR) or the “odds ratio” (OR). Both are measures of “relative risk”—the general concept of comparing disease risks in exposed vs. unexposed individuals.

42. Odds vs. Risk=probability 1:1 3:1 1:9 1:99 Note: An odds is always higher than its corresponding probability, unless the probability is 100%.

43. Exposed Disease-free cohort Not Exposed Cohort Studies Disease Disease-free Target population Disease Disease-free TIME

44. Exposure (E) No Exposure (~E) Disease (D) a b No Disease (~D) c d a+c b+d risk to the exposed risk to the unexposed The Risk Ratio, or Relative Risk (RR)

45. Normal BP Congestive Heart Failure High Systolic BP No CHF 400 400 1500 3000 1100 2600 Hypothetical Data

46. The odds ratio… • This risk ratio seems like the perfect measure of relative risk. Why not stop here? Why introduce the more complicated odds ratio?? • We cannot calculate a risk ratio from a case-control study. Case-control studies are a popular study design in epidemiology, because they are useful for studying rare diseases. • In a case-control study, we sample conditional on disease status, so we cannot calculate risk of disease.

47. Case-Control Studies Disease (Cases) Exposed in past Not exposed Target population Exposed No Disease (Controls) Not Exposed

48. Hep C + Hep C - Cases: Liver cancer 90 10 Controls 30 70 The Odds Ratio (OR) 100 100 What are P(D/E) and P(D/~E) here? We can’t tell, because, by design, we have fixed the proportion of liver cancer cases in this sample at 50% simply by selecting half controls and half cases. All these data give us is: P(E/D) and P(E/~D).

49. Hep C + Hep C - Cases: Liver cancer 90 10 Controls 30 70 Unfortunately, our sampling scheme precludes calculation of the marginals: P(E) and P(D), but turns out we don’t need these if we use an odds ratio because the marginals cancel out! The Odds Ratio (OR) 100 100 by Bayes’ Rule… Luckily, P(E/D) [=the quantity you have] 

50. Odds of disease in the exposed Odds of exposure in the cases Odds of disease in the unexposed Odds of exposure in the controls The Odds Ratio (OR) This expression is mathematically equivalent to: Backward from what we want… The direction of interest!