STATISTICAL INFERENCE PART II POINT ESTIMATION

1. STATISTICAL INFERENCEPART IIPOINT ESTIMATION

2. SUFFICIENT STATISTICS • X, f(x;),  • X1, X2,…,Xn be a sample rvs • Y=U(X1, X2,…,Xn ) is a statistic. • A sufficient statistic, Y is a statistic which contains all the information for the estimation of .

3. SUFFICIENT STATISTICS • Given the value of Y, the sample contains no further information for the estimation of . • Y is a sufficient statistic (ss) for  if the conditional distribution h(x1,x2,…,xn|y) does not depend on  for every given Y=y. • A ss for is not unique. • If Y is a ss for , then a 1-1 transformation of Y, say Y1=fn(Y) is also a ss for .

4. SUFFICIENT STATISTICS • The conditional distribution of sample rvs given the value of y of Y, is defined as • If Y is a ss for , then Not depend on  for every given y. ss for  may include y or constant. • Also, the conditional range of Xi given y not depend on .

5. SUFFICIENT STATISTICS EXAMPLE: X~Ber(p). For a r.s. of size n, show that is a ss for p.

6. SUFFICIENT STATISTICS • Neyman’s Factorization Theorem:Y is a ss for  iff Not depend on  for every given y (also in the conditional range of xi.) The likelihood function Does not contain any other xi where k1and k2 are non-negative functions and k2 does not depend on  or y.

7. EXAMPLES 1. X~Ber(p). For a r.s. of size n, find a ss for p if exists.

8. EXAMPLES 2. X~Beta(θ,2). For a r.s. of size n, find a ss for θ.

9. SUFFICIENT STATISTICS • A ss may not exist. • Jointly ss Y1,Y2,…,Yk may be needed. Example: Example 10.2.5 in Bain and Engelhardt (page 342 in 2nd edition), X(1) and X(n) are jointly ss for  • If the MLE of  exists and unique and if a ss for  exists, then MLE is a function of a ss for .

10. EXAMPLE X~N(,2). For a r.s. of size n, find jss for  and 2.

11. MINIMAL SUFFICIENT STATISTICS • If is a ss for θ, then, is also a SS for θ. But, the first one does a better job in data reduction. A minimalss achieves the greatest possible reduction.

12. MINIMAL SUFFICIENT STATISTICS • A ss T(X) is called minimal ss if, for any other ss T’(X),T(x) is a function of T’(x). • THEOREM: Let f(x;) be the pmf or pdf of a sampleX1, X2,…,Xn. Suppose there exist a function T(x) such that, for two sample points x1,x2,…,xn and y1,y2,…,yn, the ratio is constant as a function of  iff T(x)=T(y). Then, T(X) is a minimal sufficient statistic for .

13. EXAMPLE • X~N(,2) where 2 is known. For a r.s. of size n, find minimal ss for . Note: A minimal ss is also not unique. Any 1-to-1 function is also a minimal ss.

14. RAO-BLACKWELL THEOREM • Let X1, X2,…,Xn have joint pdf or pmf f(x1,x2,…,xn;) and let S=(S1,S2,…,Sk) be a vector of jss for . If T is an UE of () and (T)=E(TS), then • (T) is an UE of() . • (T) is a fn of S, so it is also jss for . • Var((T) ) Var(T) for all . • (T) is a uniformly better unbiased estimator of () .

15. RAO-BLACKWELL THEOREM • Notes: • (T)=E(TS) is at least as good as T. • For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs.

16. Example • Hogg & Craig, Exercise 10.10 • X1,X2~Exp(θ) • Find joint p.d.f. of ss Y1=X1+X2 for θ and Y2=X2. • Show that Y2 is UE of θ with variance θ². • Find φ(y1)=E(Y2|Y1) and variance of φ(Y1).

17. ANCILLARY STATISTIC • A statistic S(X) whose distribution does not depend on the parameter  is called an ancillary statistic. • An ancillary statistic contains no information about .

18. Example • Example 6.1.8 in Casella & Berger, page 257: Let Xi~Unif(θ,θ+1) for i=1,2,…,n Then, range R=X(n)-X(1) is an ancillary statistic because its pdf does not depend on θ.

19. COMPLETENESS • Let {f(x; ), } be a family of pdfs (or pmfs) and U(x) be an arbitrary function of x not depending on . If requires that the function itself equal to 0 for all possible values of x; then we say that this family is a complete family of pdfs (or pmfs).

20. EXAMPLES 1. Show that the family {Bin(n=2,); 0<<1} is complete.

21. EXAMPLES 2. X~Uniform(,). Show that the family {f(x;), >0}is not complete.

22. (n-1)S2/ 2 ~ By Basu theorem, and S2are independent. BASU THEOREM • If T(X) is a complete and minimal sufficient statistic, then T(X) is independent of every ancillary statistic. • Example:X~N(,2). S2 Ancillary statistic for 

23. COMPLETE AND SUFFICIENT STATISTICS (css) • Y is a complete and sufficient statistic (css) for  if Y is a ss for  and the family is complete. The pdf of Y. 1) Y is a ss for . 2) u(Y) is an arbitrary function of Y. E(u(Y))=0 for all  implies that u(y)=0 for all possible Y=y.

24. THE MINIMUM VARIANCE UNBIASED ESTIMATOR • Rao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then (T)=E(TS)is • an UE of , i.e.,E[(T)]=E[E(TS)]= and • the MVUE of .

25. LEHMANN-SCHEFFE THEOREM • Let Y be a cssfor . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of . • Y css for . • T(y)=fn(y) and E[T(Y)]=. • T(Y) is the UMVUE of . • So, it is the best estimator of .

26. THE MINIMUM VARIANCE UNBIASED ESTIMATOR • Let Y be a cssfor . Since Y is complete, there could be only a unique function of Y which is an UE of . • Let U1(Y) and U2(Y) be two function of Y. Since they are UE’s, E(U1(Y)U2(Y))=0 imply W(Y)=U1(Y)U2(Y)=0 for all possible values of Y. Therefore, U1(Y)=U2(Y) for all Y.

27. Example • Let X1,X2,…,Xn ~Poi(μ). Find UMVUE of μ. • Solution steps: • Show that is css for μ. • Find a statistics (such as S*) that is UE of μ and a function of S. • Then, S* is UMVUE of μ by Lehmann-Scheffe Thm.

28. Note • The estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique.