Probability and statistics

1. Probability and statistics Dr. K.W. Chow Mechanical Engineering

2. Contents • Review of basic concepts: - permutations - combinations - random variables - conditional probability • Binomial distribution

3. Contents • Poisson distribution • Normal distribution • Hypothesis testing

4. Basics • Principle of counting: There are mndifferent combinations of marriage (i.e. for each lady, there are n possible marriage combinations, thus mn) m women n men A B

5. Basics • Permutation (order important ): Form a 3-digit number from (1, 2,…9) • Combination (order unimportant ): Mary marries John = John marries Mary

6. Permutations • Permutations of n things taken r at a time (assuming no repetitions): • For the first slot / vacancy, there are n choices. • For the second slot / vacancy, there are (n –1) choices. • Thus there are n(n– 1)…(n–r + 1) = n!/(n–r)! ways.

7. Combinations • Combinations of n things taken r at a time (assuming order unimportant): • Permutations: n(n– 1)…(n–r + 1) = n!/(n–r )! ways. • Every r! combinations are equivalent to a single way. • Hence number of combinations: • n!/((n–r)! r ! )

8. Conditional Probability • The probability that an event B occurs, given that another event A has happened. • Definition: • Note that when B and A are independent, then

9. Random variables • (Intuitive) Random variables are quantities whose values are random and to which a probability distribution is assigned. • Either discrete or continuous.

10. Random variables • Example of random variables: Outcome of rolling a fair die

11. Random variables • All possible outcomes belong to the set: • Outcome is ‘random’. • Probabilities of every outcome are the same, i.e. the outcomes follow the uniform distribution. • Hence the outcomes are random variables.

12. Random variables • (Rigorous definition) Random variable is a MAPPING from elements of the sample space to a set of real numbers (or an interval on the real line). • e.g. for a fair die – mapping from {1, 2,3,4,5,6} to 1/6.

13. Probability density function • In physics, mass of an object is the integral of density over the volume of that object: • Probability density function (pdf) f(x)is defined such that the probability of a random variable X occurring between a and b is equal to the integral of f between a and b.

14. Probability density function Defining properties: • Probability density function is non-negative. • The integral over the whole sample space (e.g. the whole real axis) must be unity.

15. Probability density function • The probability is not defined at single point, it does not make sense to say what is the chance of x = 1.23 for a continuous random variable, as that chance is zero (infinitely many points).

16. Probability density function • For discrete random variables, the probability at a point is equal to the probability density function evaluated at that point: • Probability between two points (inclusive):

17. Cumulative density function • Cumulative density function (cdf) F is related to pdf by: • Note: the lower limit is the smallest value that ηcan take, not necessarily

18. Cumulative density function • For discrete random variables: • cdf’s for discrete random variables are discontinuous

19. Cumulative density function cdf of a discrete random variable cdf of a continuous random variable

20. Expectation and variance of random variables • Expectation (or mean): Integral or sum of the probability of an outcome multiplied by that outcome. • For continuous variables, the “probability” of X falling in the interval (x, x+dx) is:

21. Expectation and variance of random variables • The expectation is: • The integral is taken over the whole sample space. • Not all distributions have expectation, since the integral may not exist, e.g. the Cauchy distribution.

22. Expectation and variance of random variables • For discrete variables, the “probability” of an outcome is: • The expectation is:

23. Expectation and variance of random variables • Expectation represents the average amount one "expects" as the outcome of the random trial when identical experiments are repeated many times.

24. Expectation and variance of random variables • Example: Expectation of rolling a fair die: • Note that this “expected value” is never achieved !!

25. Expectation and variance of random variables • Standard deviation : a measure of how a distribution is spread out relative to the mean. • Definition:

26. Expectation and variance of random variables • Variance is defined as the square of standard deviation:

27. Binomial distribution • Bernoulli experiment: outcome is either success or fail. • The number of successes in n independent Bernoulli experiments are governed by the Binomial distribution. • This is a distribution with discrete random variables.

28. Binomial distribution • Suppose we perform an experiment 4 times. What is the chance of getting three successes? (Chance for success = p, chance for failure = q, p + q = 1).

29. Binomial distribution • Scenario: • p, p, p, q • p, p, q, p • p, q, p, p • q, p, p, p • There are 4C3 ways of placing the failure case.

30. Binomial distribution • Thus the chance is 4 p3 q. • For a simpler case – getting 2 heads in throwing a fair coin 3 times: • H, H, T; • H, T, H; • T, H, H.

31. Binomial distribution • Example: chance of getting exactly 2 heads when a fair coin is tossed 3 times is:

32. Binomial distribution • The probability density function for r successes in a fixed number (n ) trials is: • (r = 0, 1, 2…n) where r is the number of successes, and p is the probability of success of each trial.

33. Binomial distribution • Expectation: • Variance:

34. Binomial distribution • Methods to derive the formula E(X) = np for the binomial distribution: (1) Direct argument: Gain of p at each trial. Hence total gain of np in n trials. (2) Direct summation of series. (3) Differentiate the series expansion of the ‘binomial theorem’.

35. Binomial distribution The probability density function

36. Binomial distribution The cumulative density function

37. Poisson distribution • Poisson distribution is a special limiting case of the binomial distribution by taking: while keeping the product npfinite. • The probability density function is:

38. Poisson distribution • Expectation of the Poisson distribution: • Variance of the Poisson distribution:

39. The Poisson distribution • Physical meaning: a large number of trials (n going to infinity), and the probability of the event occurring by itself is pretty small (p approaching zero). • BUT (!!) the combined effect is finite (np being finite).

40. The Poisson distribution • Examples: • (a) The number of incorrectly dialed telephone calls if you have to dial a huge number of calls. • (b) Number of misprints in a book. • (c) Number of accidents on a highway in a given period of time.

41. Poisson distribution The probability density function (usually shows a single maximum).

42. Poisson distribution The cumulative density function (must start from zero and end up in one)

43. Normal distribution • The normal distribution for a continuous random variable is a bell-shaped curve with a maximum at the mean value. It is a special limit of the binomial distribution when the number of data points is large (i.e. ‘n going to infinity’ but without special conditions on p).

44. Normal distribution • As such the normal distribution is applicable to many physical problems and phenomena. • The ‘Central Limit Theorem’ in the theory of probability asserts the usefulness of the normal distribution.

45. Normal distribution • The probability density function: where

46. Normal distribution • The curve is symmetric about The probability density function

47. Normal distribution • For small standard deviation, the curve is tall, sharply peaked and narrow. • For large standard deviation, the curve is short and widely spread out. • (As the area under the curve must sum up to one to be a probability density function).

48. Normal distribution The cumulative density function

49. Normal distribution • Cumulative density function or probability of a normally distributed random variable falling within the interval (a, b): • Values of the above integral can be found from standard tables.

50. Simple tutorial examples for the normal distribution It is obviously not possible to tabulate the normal distribution pdf for all values of mean and standard deviation. In practice, we reduce, by simple ‘scaling arguments’, every normal distribution problem to one with mean zero and standard deviation. (Notation: N(μ, σ2))