Some Basic Probability Concepts

1. Some Basic Probability Concepts

2. Some Basic Probability Concepts Experiments, Outcomes and Random Variables • An experiment is the process by which an observation is made. • Sample Space: ‘set of all possible well distinguished outcomes of an experiment’ and is usually denoted by the letter ‘S’. • For example, Tossing a coin: S= {H, T}, Tossing a die: S = {1,2,3,4,56} • Sample Point: ‘each outcome in a sample space’ • Event: ‘Subset of the sample space’ • A random variable is ‘a real valued function defined on the sample space’. • A random variable is a variable whose value is unknown until it is observed. The value of a random variable results from an experiment; it is not perfectly predictable. Some Basic Probability Concepts

3. Some Basic Probability Concepts • A discrete random variable can take only a finite number of values that can be counted by using the positive integers. • A continuous random variable can take any real value (not just whole numbers) in an interval on the real number line • A continuous random variable can take any real value (not just whole numbers) in an interval on the real number line. Some Basic Probability Concepts

4. The Probability Distribution of a Random Variable • The term Probability is used to give a quantitative measure to the uncertainty associated with outcomes of a random experiment. • Probability: The Classical Definition • In a random experiment, if there are ‘n’ equally likely and mutually exclusive outcomes, of which ‘f’ are favorable to an event ‘A’, then the probability of occurrence of event A, denoted by P(A), is given by the ratio, f/n. • The frequency approach: ‘the limit of relative frequency as the number of observations approached infinity’. Some Basic Probability Concepts

5. Some Basic Postulates • Postulate 1: The probability of an event is a nonnegative real number; that is, 0  P (Ai)  1 for each subset Si of S; • Postulate 2: P(S) = 1 • Postulate 3: If S1, S2, S3,…Sn are mutually exclusive events defined on the sample space S, then P(S1U S2U S3… U Sn) = P(S1) + P(S2) P(S3)+…+P(Sn) • An Illustration: • Suppose we have information about the population in Comilla . We are interested in two characteristics only, Sex (M or F) and economic status (Poor or Non poor). The two characteristics are not mutually exclusive. • S = { (M & P), (F & P), (M & NP), (F &NP)} Some Basic Probability Concepts

6. If the population is finite, then the distribution is Some Basic Probability Concepts

7. In terms of probabilities, the distribution would look like Some Basic Probability Concepts

8. The probabilities pertaining to intersection of sets are called joint probabilities. For instance, P (Male ∩ Poor) is the probability that a person selected at random in Comilla will be both male and poor, i.e., has two joint characteristics. • The probabilities that appear in the last row and in the last column of the table are called marginal probabilities. P (M) gives the probability of drawing a male regardless of his economic status. • It may be noted that marginal probabilities are equal to the corresponding joint probabilities. Some Basic Probability Concepts

9. What is the probability that a person of given sex is poor, or that a person of given economic status is a male (female)? Such probabilities are called conditional probabilities. For instance, P(Poor/Male) means that we have a male and we want to find out the probability that he is poor, which is given by Some Basic Probability Concepts

10. When the values of a discrete random variable are listed with their chances of occurring, the resulting table of outcomes is called a probability function. • For a discrete random variable X the value of the probability function f(x) is the probability that the random variable X takes the value x, f(x) =P(X=x). • Therefore, 0 f(xi) 1 and, if X takes n values x1, .., xn, then. • For the continuous random variable Y the probability density function f(y) can be represented by an equation, which can be described graphically by a curve. For continuous random variables the area under the probability density function corresponds to probability. Some Basic Probability Concepts

11. Probability function & Its Advantages • Consider the experiment of tossing two six-sided dice. Define the random variable as the sum total of dots observed. Its values range from 2, 3.. to 12. The sample space will consist of all possible permutations of the two sets of numbers from 1 to 6. In sum, there will be 36 permutations. Some Basic Probability Concepts

12. The resulting probability distribution will be as follows: Some Basic Probability Concepts

13. Expected Values Involving a Single Random Variable • The Rules of Summation • If X takes n values x1, ..., xn then their sum is • If a is a constant, then • If a is a constant then Some Basic Probability Concepts

14. If X and Y are two variables, then • If X and Y are two variables, then • The arithmetic mean (average) of n values of X is • Also, Some Basic Probability Concepts

15. We often use an abbreviated form of the summation notation. For example, if f(x) is a function of the values of X, • Several summation signs can be used in one expression. Suppose the variable Y takes n values and X takes m values, and let f(x, y) =x+y. Then the doublesummation of this function is Some Basic Probability Concepts

16. To evaluate such expressions work from the innermost sum outward. First set i=1 and sum over all values of j, and so on. • To illustrate, let m = 2 and n = 3. Then Some Basic Probability Concepts

17. The order of summation does not matter, so Some Basic Probability Concepts

18. The Mean of a Random Variable • The expected value of a random variable X is the average value of the random variable in an infinite number of repetitions of the experiment (repeated samples); it is denoted E[X]. • If X is a discrete random variable which can take the values x1, x2,…,xn with probability density values f(x1), f(x2),…, f(xn), the expected value of X is Some Basic Probability Concepts

19. Expectation of a Function of a Random Variable • If X is a discrete random variable and g(X) is a function of it, then • However, in general, if X is a discrete random variable and g(X) = g1(X) + g2(X), where g1(X) and g2(X) are functions of X, then Some Basic Probability Concepts

20. The expected value of a sum of functions of random variables, or the expected value of a sum of random variables, is always the sum of the expected values. • If c is a constant, • If c is a constant and X is a random variable, then • If a and c are constants then Some Basic Probability Concepts

21. The Variance of a Random Variable • Let a and c be constants, and let Z = a + cX. Then Z is a random variable and its variance is Some Basic Probability Concepts

22. A Recap • Probability: Basic Concepts • Classical & Frequency approaches • Some Basic Postulates • Some Examples • Probability function & its advantages • Mathematical expectation Some Basic Probability Concepts

23. Using Joint Probability Functions • Marginal Probability Functions • If X and Y are two discrete random variables then • Conditional Probability Functions Some Basic Probability Concepts

24. Independent Random Variables • If X and Y are independent random variables, then for each and every pair of values of x and y. The converse is also true. • If X1, …, Xn are statistically independent the joint probability function can be factored and written as Some Basic Probability Concepts

25. If X and Y are independent random variables, then the conditional probability function of X given that Y=y is for each and every pair of values x and y. The converse is also true. Some Basic Probability Concepts

26. The Expected Value of a Function of Several Random Variables: Covariance and Correlation • If X and Y are random variables, then their covariance is • If X and Y are discrete random variables, f(x,y) is their joint probability function, and g(X,Y) is a function of them,then Some Basic Probability Concepts

27. If X and Y are discrete random variables and f(x,y) is their joint probability function, then • If X and Y are random variables then their correlation is Some Basic Probability Concepts

28. The Mean of a Weighted Sum of Random Variables • If X and Y are random variables, then Some Basic Probability Concepts

29. The Variance of a Weighted Sum of Random Variables • If X, Y, and Z are random variables and a, b, and c are constants, then • If X, Y, and Z are independent, or uncorrelated, random variables, then the covariance terms are zero and: Some Basic Probability Concepts

30. If X, Y, and Z are independent, or uncorrelated, random variables, and if a = b = c = 1, then Some Basic Probability Concepts

31. Theoretical Derivation of Sampling Distribution of Estimators & Test Statistics: • Binomial Distribution: • Comilla Story: Picking a BPL Person • Let p be the proportion of BPL population in Comilla and q be the proportion of APL population. • Let n denote the sample size. • Let be the proportion of BPL in the sample. • Let X denote the number of poor in the sample. • If the person picked up happens to be poor, the experiment is a success and its probability is p. Otherwise, it is a failure with a probability given by q, that is, (1-p). Some Basic Probability Concepts

32. Let us define the sampling distributions of X and for samples of various sizes. Since = (X/n) or X = n , by the different results that we have learnt so far, we can determine the distribution of , if we know that of X and vice versa. Some Basic Probability Concepts

33. Sampling Distribution for n = 1 Some Basic Probability Concepts

34. Mean and variance of X: • Mean and variance of : E( ) – E(X/n) = E(X) = p Var( )=Var(X/n)=Var(X)=pq Some Basic Probability Concepts

35. Sampling Distribution for n = 2 Some Basic Probability Concepts

36. Mean and variance of X: • Mean and variance of : Some Basic Probability Concepts

37. Sampling Distribution for n = 3 Some Basic Probability Concepts

38. Mean and variance of X: • Mean and variance of Some Basic Probability Concepts

39. In general, we have: Some Basic Probability Concepts

40. That is, the probability of getting x poor people in a sample size of ‘n’ is • Properties: Some Basic Probability Concepts

41. E( ) = p, that is, unbiased estimator. • ,that is, the distribution gets concentrated as sample size increases. This property together with (i) implies is a consistent estimator. The dispersion of the sampling distribution decreases in inverse proportion to the square root of sample size. That is, if sample size increases k times, then the std. deviation of the sampling distribution decreases times. Some Basic Probability Concepts

42. The sampling distribution of is most dispersed when the population parameter p is equal to ½ and is least dispersed when p is 0 or 1. Some Basic Probability Concepts

43. The asymmetry (skewness) of the sampling distribution of decreases in inverse proportion to the square root of sample size (since ))))))))))))))))). • It is least skewed when p = ½ and is most skewed when p is 0 or 1. Some Basic Probability Concepts

44. The Normal Distribution • Properties: • The distribution is continuous and symmetric around its mean μ. This implies: (i) mean = median = mode; and (ii) the mean divides the area under the normal curve into exact halves. • The range of the distribution extends from -∞ to + ∞. In other words, the distribution is unbounded. Some Basic Probability Concepts

45. The curve attains maximum height at x = μ; the points of inflection occur at x = μσ(which means the standard deviation measures the distance from the center of the distribution to a point of inflection). • Normal distribution is fully specified by two parameters, mean (μ) and variance (σ2). If we know these two parameters, we know all there is to know about it. • If X, Y,…, Z are normally and independently distributed random variables and a,b,…,c are constants, then the linear combination aX+bY+…+cZ is also normally distributed. Some Basic Probability Concepts

46. How to calculate probabilities for a normal random variable? • From tabulated results • Different normal distributions lead to different probabilities due to differences in mean and variance. For the same reason, if we know the area under one specific normal curve, the area under any other normal curve can be computed by accounting for the differences in mean and variance. • One specific distribution for which areas have been tabulated is a normal distribution with mean μ = 0 and variance σ2 = 1, called the standard normal distribution (also called unit normal distribution). • Given that (i) X is normally distributed with mean μ and variance σ2; and (ii) the areas under the standard normal curve, how to determine the probability that x lies in some interval, say, (x1 and x2) ? Some Basic Probability Concepts

47. Let Z denote a normally distributed variable with mean zero and variance equal to unity. That is, • P(x1 < x < x2) = probability that X will lie between x1 and x2(x1 < x2); and P(z1< z <z2) = probability that Z will lie between z1 and z2 (z1 < z2) . • Since X is normally distributed, a linear function of X will also be normal. Some Basic Probability Concepts

48. Let it be denoted by aX + b, where a and b are constants. • Choose a and b such that (aX+b) is a standard normal variable. That is, Some Basic Probability Concepts

49. Solving for a and b , we get • Thus, we have aX+b = = Z • In other words, any variable with mean μ and variance σ2 can be transformed into a standard normal variable by expressing it as a deviation from its mean and dividing by σ. Some Basic Probability Concepts

50. Consider P(x1 < x < x2) where x1 < x2. • From = Z, we get X = Z+. Hence, we can write x1 = z1 +  and x2 = z2 +  • Now, P(x1 < x < x2) = P(z1 +  < Z +  <z2 +  ) = P(z1 < Z < z2) • where z1 = and z2 = Some Basic Probability Concepts