1 / 39

Lectures prepared by: Elchanan Mossel elena Shvets

This lecture explores the concepts of variance and standard deviation in probability distributions and discusses their computational formulas and properties. It also examines the Chebyshev's inequality and the variance of indicators, as well as the variance of a sum of independent random variables. The lecture concludes with the square root law, the weak law of large numbers, and the normal approximation.

bbush
Télécharger la présentation

Lectures prepared by: Elchanan Mossel elena Shvets

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lectures prepared by:Elchanan Mosselelena Shvets

  2. Histo 1 X = 2*Bin(300,1/2) – 300 E[X] = 0

  3. Histo 2 Y = 2*Bin(30,1/2) – 30 E[Y] = 0

  4. Histo 3 Z = 4*Bin(10,1/4) – 10 E[Z] = 0

  5. Histo 4 W = 0 E[W] = 0

  6. A natural question: • Is there a good parameter that allow to distinguish between these distributions? • Is there a way to measure the spread?

  7. Variance and Standard Deviation • The variance of X, denoted by Var(X) is the mean squared deviation of X from its expected value m = E(X): • Var(X) = E[(X-m)2]. • The standard deviation of X, denoted by SD(X) is the square root of the variance of X:

  8. Computational Formula for Variance Claim: Proof: E[ (X-m)2] = E[X2 – 2m X + m2] E[ (X-m)2] = E[X2] – 2m E[X] + m2 E[ (X-m)2] = E[X2] – 2m2+ m2 E[ (X-m)2] = E[X2] – E[X]2

  9. Variance and SD For a general distribution Chebyshev inequality states that for every random variable X, X is expected to be close to E(X) give or take a few SD(X). Chebyshev Inequality: For every random variable X and for all k > 0: P(|X – E(X)| ¸ k SD(X)) · 1/k2.

  10. Properties of Variance and SD • Claim: Var(X) ¸ 0. • Pf: Var(X) = å (x-m)2 P(X=x) ¸ 0 • Claim: Var(X) = 0 iff P[X=] = 1.

  11. Chebyshev’s Inequality • P(|X – E(X)| ¸ k SD(X)) · 1/k2 • proof: • Let m = E(X) and s = SD(X). • Observe that |X–m| ¸ k s, |X–m|2¸ k2s2. • The RV |X–m|2 is non-negative, so we can use Markov’s inequality: • P(|X–m|2¸ k2s2) · E [|X–m|2 ] / k2s2 • P(|X–m|2¸ k2s2)·s2 / k2s2 = 1/k2.

  12. Variance of Indicators Suppose IA is an indicator of an event A with probability p. Observe that IA2 = IA. Ac A IA=1=IA2 IA=0=IA2 E(IA2) = E(IA) = P(A) = p, so: Var(IA) = E(IA2) – E(IA)2 = p – p2 = p(1-p).

  13. Variance of a Sum of Independent Random Variables Claim: if X1, X2, …, Xn are independent then: Var(X1+X2+…+Xn) = Var(X1)+Var(X2)+…+Var(Xn). Pf: Suffices to prove for 2 random variables. E[( X+Y – E(X+Y) )2 ] = E[( X-E(X) + Y–E(Y) )2] = E[( X-E(X))2 + 2 E[(X-E(X)) (Y-E(Y))] + E(Y–E(Y) )2]= Var(X) + Var(Y) + 2 E[(X-E(X))] E[(Y-E(Y))](mult.rule) = Var(X) + Var(Y) + 0

  14. Variance and Mean under scaling and shifts • Claim: SD(aX + b) = |a| SD(X) • Proof: • Var[aX+b] = E[(aX+b – am –b)2] = • = E[a2(X-m)2] = a2s2 • Corollary: If a random variable X has • E(X) = m and SD(X) = s > 0, then • X*=(X-m)/s has • E(X*) =0 and SD(X*)=1.

  15. Square Root Law Let X1, X2, … , Xn be independent random variables with the same distribution as X, and let Sn be their sum: Sn = åi=1n Xi, and their average, then:

  16. Weak Law of large numbers Thm: Let X1, X2, … be a sequence of independent random variables with the same distribution. Let m denote the common expected value m = E(Xi). Then for every e > 0:

  17. Weak Law of large numbers Proof: Let m = E(Xi) and s = SD(Xi). Then from the square root law we have: Now Chebyshev inequality gives us: For a fixed e right hand side tends to 0 as n tends to 1.

  18. The Normal Approximation • Let Sn = X1 + … + Xn be the sum of independent random variables with the same distribution. • Then for large n, the distribution of Sn is approximately normal with mean E(Sn) = n  and SD(Sn) =  n1/2, • where  = E(Xi) and  = SD(Xi). • In other words:

  19. Sums of iid random variables • Suppose Xi represents the number obtained on the i’th roll of a die. • Then Xi has a uniform distribution on the set • {1,2,3,4,5,6}.

  20. Distribution of X1

  21. Sum of two dice • We can obtain the distribution of S2 = X1 +X2 by the convolution formula: • P(S2 = k) = åi=1k-1 P(X1=i) P(X2=k-i| X1=i), • by independence • = åi=1k-1 P(X1=i) P(X2=k-i).

  22. Distribution of S2

  23. Sum of four dice • We can obtain the distribution of • S4 = X1 + X2 + X3 + X4 = S2 + S’2 again by the convolution formula: • P(S4 = k) = åi=1k-1 P(S2=i) P(S’2=k-i| S2=i), • by independence of S2 and S’2 • = åi=1k-1 P(S2=i) P(S’2=k-i).

  24. Distribution of S4

  25. Distribution of S8

  26. Distribution of S16

  27. Distribution of S32

  28. Distribution of X1

  29. Distribution of S2

  30. Distribution of S4

  31. Distribution of S8

  32. Distribution of S16

  33. Distribution of S32

  34. Distribution of X1

  35. Distribution of S2

  36. Distribution of S4

  37. Distribution of S8

  38. Distribution of S16

  39. Distribution of S32

More Related