PS 601 Notes – Part II Statistical Tests

1. PS 601 Notes – Part II Statistical Tests • Notes Version - March 8, 2005

2. Statistical tests • We can use the properties of probability density functions to make probability statements about the likelihood of events occurring. • The standard normal curve provides us with a scale or benchmark for the likelihood of being at (or above or below) any point on the scale

3. Standard normal values • Note for instance that if we look at the value 1.5 under the standard normal table, we find the value .4332. • This means that the probability of having a standard normal value greater than 1.5 is .5 - .4332 = .0668

4. In Applied Terms • If IQ has a mean of 100, and a standard deviation of 20, what is the probability that any given individuals IQ will be greater than or equal to 130. • Standardize the score of 130 • Look up 1.5 in the standard normal table

5. Two-tailed hypotheses • In general our hypothesis is: • Did the sample come from some particular population? • If the sample mean is too high or too low, we suspect that it did not. • Thus, we must check to see if the sample mean is either significantly higher, or significantly lower. • This is called a two-tailed test. • When in doubt, most tests are best done as two-tailed ones

6. The One Tailed Hypothesis • Sometimes we suspect, or hypothesize, direction • e.g. The average income for West Virginia will be significantly lower than the country as a whole. • HA: Xbar <  • This is a one-tailed test • We ignore the tail in the direction not hypothesized

7. The Z-test • The z-test is based upon the standard normal distribution. • In this case we are making statements about the sample mean, instead of the actual data values

8. The Z-test – (cont.) • Note that the Z-test is based upon two parts. • The standard normal transformation • The standard deviation of the sampling distribution.

9. The Z-test – an example • Suppose that you took a sample of 25 people off the street in Morgantown and found that their personal income is $24,379 • And you have information that the national average for personal income per capita is $31,632 in 2003. • Is the Morgantown significantly different from the National Average • Sources: • (1) Economagic • (2) US Bureau of Economic Analysis

10. What to conclude? • Should you conclude that West Virginia is lower than the national average? • Is it significantly lower? • Could it simple be a randomly “bad” sample” • Assume that it is not a poor sampling technique • How do you decide?

11. Example (cont.) • We will hypothesize that WV income is lower than the national average. • HA: WVInc < USInc (Alternate Hypothesis) • H0: WVInc = USInc (Null Hypothesis) • Since we know • the national average ($31,632), • and standard deviation (15000), • OK – I made this up to make the problem simpler • we can use the z-test to make decide if WV is indeed statistically significantly lower than the nation

12. Example (cont.) • Using the z-test, we get • OK – so what?

13. The Probability of a Type I error • We would like to infer that WV had a lower income than the national average, but we must examine whether we simply got these numbers by chance in a random sample • We would like to not make mistakes when we make statistical decisions. • We know we will. • With statistical inference, we have the ability to decide how often we find it acceptable to be wrong – by random chance. • Thus we set the probability of making a Type I error. • P(Type I error) =  = ? • By convention =.05

14. The Critical Value of Z (cont) • Ok, now we know z… • We know that we can make probability statements about z, since it is from the standard normal distribution • We know that if z =1.96 then the area out in the tail past 1.96 is equal to .025 • This means that the likelihood of obtaining a value of z > 1.96 by random chance in any given sample is less than .025.

15. The Critical Values of Z to memorize • Two tailed hypothesis • Reject the null (H0) if z  1.96, or z  -1.96 • One tailed hypothesis • If HA is Xbar > , then reject H0 if z  1.645 • If HA is Xbar < , then reject H0 if z  -1.645

16. Z test example (cont.) • Suppose we decided to look at a different state, say Oregon. • Would you try a 1-tailed test? • Which way? HA: Xbar >  or HA: Xbar <  • Without an a priori reason to hypothesize higher oir lower, use the 2-tailed test • Assume Oregon has a mean of 29,340, and that we collected a somewhat larger sample, say 100. • Using the z-test, we get • What would we conclude? What if n=25? 1000?

17. The applicability of the z-test • We frequently run into a problem with trying to do a z test. • The sample size may be below the number needed for the CLT to apply (N~30) • While the population mean () may be frequently available, the population standard deviation () frequently is not. • Thus we use our best estimate of the population standard deviation – the sample standard deviation (s).

18. The t test • When we cannot use the population standard deviation, we must employ a different statistical test • Think of it this way: • The sample standard deviation is biased a little low, but we know that as the sample size gets larger, it becomes closer to the true value. • As a result, we need a sampling distribution that makes small sample estimates conservative, but gets closer to the normal distribution as the sample size gets larger, and the sample standard deviation more closely resembles the population standard deviation. • Thus we need the Student’s t

19. The t-test (cont.) • The t-test is a very similar formula. • Note the two differences • using s instead of  • The resultant is a value that has a t-distribution instead of a standard normal one.

20. The t distribution • The t distribution is a statistical distribution that varies according to the number of degrees of freedom (Sample size – 1) • As df gets larger, the t approximates the normal distribution. • For practical purposes, the t-distribution with samples greater than 100 can be viewed as a normal distribution.

21. Selecting the critical value – t-dist • Selecting the critical value of the t-distribution requires these steps. • Determine whether one- or two-tailed test. • Select α level (α=.05) • Determine degrees of freedom (n-1) • Find value for t in appropriate column (table if one- & two-tailed tests are separate tables) • Critical value of t is at intersection of df row and α-level column.

22. Interpreting t-value • The t-test formula gives you a value that you can compare to the critical value. • If: • Conducting a two tailed test, if the calculated t-value is outside the range of –t to +t, we conclude that the sample is significantly different that the population. • Note that a t-value that exceeds the critical value means that the probability of that t is less than the selected α-level. • Hence if t > C.V . of t, then p(t) < α (say .05)

23. Interpreting t-value – one tailed test • The t-test formula gives you a value that you can compare to the critical value. • If: • Conducting a one-tailed test, if the calculated t-value is greater that the critical value of t, or less than –(critical value of t), we conclude that the sample is significantly different that the population. • Choice of t or –t is determined by the one-tailed test direction. • Note that a t-value that exceeds the critical value means that the probability of that t is less than the selected α-level. • Hence if t > C.V . of t, then p(t) < α (say .05)

24. T-test example • Suppose we decided to look at Oregon, but do not know the population standard deviation • Would you try a 1-tailed test? • Which way? HA: Xbar >  or HA: Xbar <  • Like the z-test, without an a priori reason to hypothesize higher or lower, use the 2-tailed test • Assume Oregon has a mean of 29,340, and that we collected a sample of 169. • Using the t-test, we get • What would we conclude? What if n=25? 1000?

25. Two-sample t-test • Frequently we need to compare the means of two different samples. • Is one group higher/lower than some other group? • e.g. is the Income of blacks significantly lower than whites? • The two-sample t difference of means test is the typical way to address this question.

26. Examples • Is the income of blacks lower than whites? • Are teachers salaries in West Virginia and Mississippi alike? • Is there any difference between the background well and the monitoring well of a landfill?

27. The Difference of means Test • Frequently we wish to ask questions that compare two groups. • Is the mean of A larger (smaller) than B? • Are As different (or treated differently) than Bs? • Are A and B from the same population? • To answer these common types of questions we use the standard two-sample t-test

28. The Difference of means Test • The standard two-sample t-test is:

29. The standard two sample t-test • In order to conduct the two sample t-test, we only need the two samples • Population data is not required. • We are not asking whether the two samples are from some large population. • We are asking whether they are from the same population, whatever it may be.

30. Assumptions about the variance • The standard two-sample t-test makes no assumptions about the variances of the underlying populations. • Hence we refer to the standard test as the unequal variance test. • If we can assume that the variances of the tow populations are the same, then we can use a more powerful test – the equal variance t-test.

31. The Equal Variance test • If the variances from the two samples are the same we may use a more powerful variation • Where

32. Which test to Use? • In order to choose the appropriate two-sample t-test, we must decide if we think the variances are the same. • Hence we perform a preliminary statistical test – the equal variance F-test.

33. The Equal Variance F-test • One of the fortunate properties on statistics is that the ratio of two variances will have an F distribution. • Thus with this knowledge, we can perform a simple test.

34. Interpretation of F-test • If we find that P(F) > .05, we conclude that the variances are equal. • If we find that P(F)  .05, we conclude that the variances are unequal. • We then select the equal– or unequal-variance t-test accordingly. • The F distribution

35. Degrees of freedom • Note that the degrees of freedom is different across the two tests • Equal variance test • Df = n1 +n2-2 • Unequal variance test • Df = complicated – real number not integer

36. Contingency Tables • Often we have limited measurement of our data. • Contingency Tables are a means of looking at the impact of nominal and ordinal measures on each other. • They are called contingency tables because one variables value is contingent upon the other. • Also called cross-tabulation or crosstabs.

37. Contingency Tables • The procedure is quite simple and intuitively appealing • Construct a table with the independent variable across the top and the dependent variable on the side • This works fairly well for low numbers of categories (r,c < 6 or so)

38. Contingency Tables An example • Presidents are often suspected of using military force to enhance their popularity. • What do you suppose the data actually look like? • Any conjectures • Let’s categorize presidents as using force,or not, and as having popularity above and below 50% • Are there definition problems here? • Which is independent and which is dependent?

39. Contingency Tables

40. Measures of Independence • Are the variables actually contingent upon each other? • Is the use of force contingent upon the president’s level of popularity? • We would like to know if these variables are independent of each other, or does the use of force actually depend upon the level of approval that the president have at that time?

41. 2 Test of Independence • The 2 Test of Independence gives us a test of statistical significance. • It is accomplished by comparing the actual observed values to those you would expect to see if the two variables are independent.

42. 2 Test of Independence • Formula • Where

43. Chi-Square Table (2)

44. Interpreting the 2 • The Table gives us a 2 of 5.55 with 1 degree of freedom [d.f. = (r-1)*(c-1)] • The critical value of 2 with 1 degree of freedom is 3.84 (see 2 Table) • We therefore conclude that Presidential popularity and use of force are related. • We technically “reject the null hypothesis that Presidential popularity and use of force are independent.” • Note: 2 is influenced by sample size. • It ranges from 0.0 to .

45. Corrected 2 measures • Small tables have slightly biased measures of 2 • If there are cell frequencies that are low, then there are some adjustments to make that correct the probability estimates that 2 provides.

46. Yate’s Corrected 2 • For use with a 2x2 table with low cell frequencies (5<n<10) • If there are any cell frequencies < 5, the 2 is invalid. • Use Fisher’s Exact Test

47. Measures of Association • Not only do we want to see whether the variables of a cross-tabulation are independent, we often want to see if the relationship is a strong or weak one. • To do this, we use what are referred to as measures of association. • The level of measurement determines what measure of association we might use.

48. Measures of Association • We group them according to whether the variables are nominal or ordinal. • If one variable is nominal, use nominal measures. • If both are ordinal, use an ordinal measure. • If either is interval, generally we use a different statistical design.

49. Measures based on 2 • Contingency Coefficient • Kramer’s V

50. Yule’s Q • May be used on any 2x2 table, nominal or ordinal • If we define out table with cell counts as • Yule’s Q is calculated as: • Q ranges from 0 to 1.0 • Q compares concordant pairs to discordant pairs