More About Hypothesis Test

1. More About Hypothesis Test Chapter 21

2. The Null Hypothesis • To perform a hypothesis test, the null must be a statement about the value of a parameter for a model. • How do we choose the null hypothesis? The appropriate null arises directly from the context of the problem—it is not dictated by the data, but instead by the situation. • A good way to identify both the null and alternative hypotheses is to think about the Why of the situation.

3. The Null Hypothesis (cont.) • There is a temptation to state your claim as the null hypothesis. • However, you cannot prove a null hypothesis true. • So, it makes more sense to use what you want to show as the alternative. • This way, when you reject the null, you are left with what you want to show.

4. Review of Necessary Conditions • We must state the assumption and check the corresponding conditions to determine whether we can model the sampling distribution of the proportion with a Normal model. • Conditions to check: (same conditions as used for C.I.) • 1. Random Sampling Condition • 2. 10% Condition • 3. Success/Failure Condition

5. How to Think about P-Values • A P-value is a conditional probability • Tells us the probability of the observed statistic given that the null hypothesis is true • P-value = P[observed statistic value (or even more extreme) | Ho] • ***P-value is not the probability that the null hypothesis is true • The smaller the P-value, the more confident we can be in declaring that we doubt the null hypothesis

6. Alpha Levels • We can define “rare event” arbitrarily by setting a threshold for our P-value. • If our P-value falls below that point, we’ll reject H0. We call such results statistically significant. • The threshold is called analpha level, denoted by .

7. Alpha Levels (cont.) • Common alpha levels are 0.10, 0.05, and 0.01. • You have the option—almost the obligation—to consider your alpha level carefully and choose an appropriate one for the situation. • The alpha level is also called the significance level. • When we reject the null hypothesis, we say that the test is “significant at that level.”

8. Alpha Levels (cont.) • What can you say if the P-value does not fall below ? • You should say that “The data have failed to provide sufficient evidence to reject the null hypothesis.” • Don’t say that you “accept the null hypothesis.”

9. Making Errors • When testing a null hypothesis, we make a decision either to reject it or fail to reject it. • Our conclusions are sometimes correct and sometimes wrong (even if we do everything correctly). • There are two types of errors that can be made. • Type I error: The mistake of rejecting the null hypothesis when it is actually true. • Type II error: The mistake of failing to reject the null hypothesis when it is actually false.

10. Making Errors (cont.) Here’s an illustration of the four situations in a hypothesis test: Which type of error is more serious depends on the situation at hand. In other words, the gravity of the error is context dependent.

11. Making Errors (cont.) How can we remember which error is type I and which is type II? Let’s try a mnemonic device - “ROUTINE FOR FUN” Using only the consonants from those words: RouTiNe FoR FuN We can easily remember that a type I error is RTN: reject true null (hypothesis), whereas a type II error is FRFN: failure to reject a false null (hypothesis).

12. Making Errors (cont.) • How often will a Type I error occur? • Since a Type I error is rejecting a true null hypothesis, the probability of a Type I error is our  level. • When H0 is false and we reject it, we have done the right thing. • A test’s ability to detect a false hypothesis is called the power of the test.

13. Making Errors (con’t) • When H0 is false and we fail to reject it, we have made a Type II error. • We assign the letter  to the probability of this mistake. • It’s harder to assess the value of  because we don’t know what the value of the parameter really is. • There is no single value for --we can think of a whole collection of ’s, one for each incorrect parameter value.

14. Making Errors (cont.) • One way to focus our attention on a particular  is to think about the effect size. • Ask “How big a difference would matter?” • We could reduce  for all alternative parameter values by increasing . • This would reduce  but increase the chance of a Type I error. • This tension between Type I and Type II errors is inevitable. • The only way to reduce both types of errors is to collect more data. Otherwise, we just wind up trading off one kind of error against the other.

15. Power • The power of a test is the probability that it correctly rejects a false null hypothesis. • The power of a test is 1 – . • The value of the power depends on how far the truth lies from the null hypothesis value. • The distance between the null hypothesis value, p0, and the truth, p, is called the effect size. • Power depends directly on effect size.

16. Controlling Type I and Type II Errors • a, b, and sample size (n) are all related, so when you choose or determine any two of them, the third is automatically determined. • Try to use the largest a that you can tolerate. However, for type I errors with more serious consequences, select smaller values of a. • Then choose a sample size as large as reasonable, based on considerations of time, cost and other relevant factors.

17. Controlling Type I and Type II Errors (cont.) • For any fixed sample size n, a decrease in a will cause an increase in b. • For any fixed a, an increase in the sample size n will cause a decrease in b. • To decrease both a and b, increase the sample size.

18. Example - Radio Ads • A company is willing to renew its advertising contract with a local radio station only if the station can prove that more than 20% of the residents of the city have heard the ad and recognize the company’s product. The radio station conducts a random phone survey of 400 people. • A.) What are the hypotheses? • B.) What would a Type I error be? • C.) What would a Type II error be?

19. Example - Radio Ads (cont.) D.) The station plans to conduct this test using a 10% level of significance, but the company wants the significance level lowered to 5%. Why? E.) What is meant by the power of the test. F.) For which level of significance will the power of this test be higher? Why? G.) They finally agree to use a =0.5, but the company proposes that the station call 600 people instead of the 400 initially proposed. Will that make the risk of Type II error higher or lower. Explain.

20. Assignment • Read Chapter 23 • Try the following problems from Ch. 21: • #1, 3, 13, 15, 19, 21, and 25