Mean, Variance, Moments and Characteristic Functions

1. Mean, Variance, Moments and Characteristic Functions

2. Mean and Variance • p.d.f of a r.v represents complete information about it. • quite often it is desirable to characterize the r.v in terms of its average behavior. • We introduce two parameters • Mean • Variance • represent the overall properties of the r.v and its p.d.f. 2

3. Mean • Mean or the Expected Value of a r.v X is defined as • If X is a discrete-type r.v, • Mean represents the average value of the r.v in a very large number of trials. This is the midpoint of the interval (a,b). Example: Uniform Distribution 3

4. Mean Example: Exponential Distribution 4

5. Mean Example: Poisson Distribution 5

6. Mean Example: Binomial Distribution 6

7. Mean Example: Normal Distribution 7

8. Functions of r.vs • Given • suppose defines a new r.v with p.d.f • Y has a mean • To determine we don’t need to determine • Recall that for any y, where represent the multiple solutions of the equation • This can be rewritten as where the terms form nonoverlapping intervals. 8

9. Functions of r.vs – continued • Hence • and hence as y covers the entire y-axis, • the corresponding Δx’s are nonoverlapping, and they cover the entire x-axis. • As • Which in the discrete case, reduces to: • So, is not required to evaluate for 9

10. Functions of r.vs – continued • Determine the mean of where X is a Poisson r.v. Example is known as the kth moment of r.v X 10

11. (b) (a) Another Measure • Mean alone will not be able to truly represent the p.d.f of any r.v. • Consider two Gaussian r.vs and • Despite having the same mean , their p.d.fs are quite different as one is more concentrated around the mean. • We need an additional parameter to measure this spread around the mean. 11

12. Variance and Standard Deviation • For a r.v X with mean (positive or negative)represents the deviation of the r.v from its mean. • So, consider the quantity and its average value represents the average mean square deviation of X around its mean. • Define • Using with we have: • The standard deviation represents the root mean square spread of the r.v X around its mean. variance of the r.v X standard deviation of X 12

13. Variance and Standard Deviation This can be used to compute Example: Poisson Distribution 13

14. Variance and Standard Deviation • To simplify this, we can make use • For a normal p.d.f. This gives • Differentiating both sides of this equation with respect to we get • or Example: Normal Distribution 14

15. Moments • in general are known as the moments of the r.v X, and are known as the central moments of X. • Clearly, and • It is easy to relate and • generalized moments of X about a, • absolute moments of X. 15

16. Variance and Standard Deviation • It can be shown that Example: Normal Distribution 16

17. Characteristic Function • The characteristic function of a r.v X is defined as • and for all • For discrete r.vs: 17

18. Example Example: Poisson Distribution 18

19. Example Example: Binomial Distribution 19

20. Characteristic Function and Moments • Taking the first derivative of this equation with respect to , and letting it to be equal to zero, we get • Similarly, the second derivative gives 20

21. Characteristic Function and Moments – continued • repeating this procedure k times, we obtain the kth moment of X to be • We can use these results to compute the mean, variance and other higher order moments of any r.v X. 21

22. Example These results agree with previous ones, but the efforts involved in the calculations are very minimal. Example: Poisson Distribution So, 22

23. Example Example: Mean and Variance of Binomial Distribution So, 23

24. Example The characteristic function of a Gaussian r.v itself has the “Gaussian” bell shape. Example: Normal(Guassian) Distribution 24

25. (b) (a) Example - continued • Note the reverse roles of in and For we have and therefore: 25

26. Example Example: Cauchy Distribution Which diverges to infinity. Similarly: With So the double sided integral for mean is undefined. It concludes that the mean and variance of a Cauchy r.v are undefined. 26

27. Chebychev Inequality • Consider an interval of width 2ε symmetrically centered around its mean μ. • What is the probability that X falls outside this interval? we can start with the definition of So the desired probability is: Chebychev inequality 27

28. Chebychev Inequality • So, the knowledge of is not necessary. We only need • With we obtain • Thus with we get the probability of X being outside the 3σ interval around its mean to be 0.111 for any r.v. • Obviously this cannot be a tight bound as it includes all r.vs. • For example, in the case of a Gaussian r.v, from the Table, • Chebychev inequality always underestimates the exact probability. 28

29. Moment Identities • Suppose X is a discrete random variable that takes only nonnegative integer values. i.e., • Then • Similarly, • Which gives: These equations are at times quite useful in simplifying calculations. 29

30. Example: Birthday Pairing • In a group of n people, what is A) The probability that two or more persons will have the same birthday? B) The probability that someone in the group will have the birthday that matches yours? Solution • C = “no two persons have the same birthday” For n=23, 30

31. Example – continued • D = “a person not matching your birthday” • let Xrepresent the minimum number of people in a group for a birthday pair to occur. • The probability that “the first n people selected from that group have different birthdays” is given by 31

32. Example – continued • But the event the “the first n people selected havedifferent birthdays” is the same as the event “ X > n.” • Hence • Using moment identities, this gives the mean value of X to be 32

33. Example – continued • Similarly, • Thus, and • Since the standard deviation is quite high compared to the mean value, the actual number of people required could be anywhere from 25 to 40. 33

34. Moment Identities for Continuous r.vs ifXis a nonnegative random variable with density function fX(x)and distribution functionFX(X), then where Similarly 34