Lecture 4A : Probability Theory Review

1. Advanced Artificial Intelligence Lecture 4A: Probability Theory Review

2. Outline • Axioms of Probability • Product and chain rules • Bayes Theorem • Random variables • PDFs and CDFs • Expected value and variance

3. Introduction • Sample space - set of all possible outcomes of a random experiment • Dice roll: {1, 2, 3, 4, 5, 6} • Coin toss: {Tails, Heads} • Event space - subsets of elements in a sample space • Dice roll: {1, 2, 3} or {2, 4, 6} • Coin toss: {Tails}

4. Axioms of Probability • for all

5. examples • Coin flip • P(H) • P(T) • P(H,H,H) • P(x1=x2=x3=x4) • P({x1,x2,x3,x4} contains more than 3 heads)

6. Set operations • Let and • and • Properties: • If then

7. Conditional Probability • A and B are independent if • A and B are conditionally independent given C if

8. Conditional Probability • Joint probability: • Product rule: • Chain rule of probability:

9. examples • Coin flip • P(x1=H)=1/2 • P(x2=H|x1=H)=0.9 • P(x2=T|x1=T)=0.8 • P(x2=H)=?

10. Conditional Probability • probability of A given B • Example: • probability that school is closed • probability that it snows • probability of school closing if it snows • probability of snowing if school is closed

11. Conditional Probability • probability of A given B • Example: • probability that school is closed • probability that it snows

12. Quiz • P(D1=sunny)=0.9 • P(D2=sunny|D1=sunny)=0.8 • P(D2=rainy|D1=sunny)=? • P(D2=sunny|D1=rainy)=0.6 • P(D2=rainy|D1=rainy)=? • P(D2=sunny)=? • P(D3=sunny)=?

13. Joint Probability • Multiple events: cancer, test result

14. Joint Probability • The problem with joint distributions It takes 2D-1 numbers to specify them!

15. Conditional Probability • Describes the cancer test: • Put this together with: Prior probability

16. Conditional Probability • We have: • We can now calculate joint probabilities

17. Conditional Probability • “Diagnostic” question: How likely do is cancer given a positive test?

18. Bayes Theorem • We can relate P(A|B) and P(B|A) through Bayes’ rule:

19. Bayes Theorem • We can relate P(A|B) and P(B|A) through Bayes’ rule: Posterior Probability Prior Probability Likelihood Normalizing Constant

20. Bayes Theorem • We can relate P(A|B) and P(B|A) through Bayes’ rule: • can be eliminated with law of total probability: =

21. Random Variables • Random variable X is a function s.t. • Examples: • Russian roulette: X = 1 if gun fires and X = 0 otherwise and • X = # of heads in 10 coin tosses

22. Cumulative Distribution Functions • Defined as such that • Properties:

23. Probability Density Functions • For discrete random variables, defined as: • Relates to CDFs: • =

24. Probability Density Functions • For continuous random variables, defined as: • Relates to CDFs:

25. Probability Density Functions • Example: • Let X be the angular position of freely spinning pointer on a circle

26. Probability Density Functions • Example: • Let X be the angular position of freely spinning pointer on a circle f(X) 1/2 X 2

27. Probability Density Functions • Example: • Let X be the angular position of freely spinning pointer on a circle f(X) 1/2 X 2

28. Probability Density Functions • Example: • Let X be the angular position of freely spinning pointer on a circle • for f(x) 1/2 x 2 F(x) 1 x 2

29. Probability Density Functions • Example: • Let X be the angular position of freely spinning pointer on a circle • for • = f(x) 1/2 x 2 F(x) 1 x 2

30. Expectation • Given a discrete r.v. X and a function • For a continuous r.v. X and a function

31. Expectation • Expectation of a r.v. X is known as the first moment, or the mean value, of that variable -> • Properties: • for any constant • for any constant • +

32. Variance • For a random variable X, define: • Another common form: • Properties: • for any constant • for any constant

33. Gaussian Distributions • Let , then where represents the mean and the variance • Occurs naturally in many phenomena • Noise/error • Central limit theorem