Statistical Significance and  (alpha) level

1. Statistical Significanceand (alpha) level Lesson 10.2.2

2. Starter 10.2.2 • Write the definition of an unbiased statistic.

3. Today’s Objectives • Determine the significance level () to be used in a hypothesis test • Determine whether the results of a hypothesis test are statistically significant • Incorporate P-value and  into the conclusion statement of a hypothesis test California Standard 18.0 Students determine the P- value for a statistic for a simple random sample from a normal distribution.

4. Significance level  • If the P-value of a hypothesis test is above 20% or 25%, we would easily agree that there is not enough evidence to reject Ho • If the P-value is less than 20%, we begin to believe that there is some evidence to reject Ho and support Ha • If the P-value is in the range of 10% or 5% or less, we believe that there is good evidence to reject Ho and support Ha • So at what P-value do we decide there is enough evidence? • We resolve this by determining in advance the level of evidence we require to reject Ho • This is called the significance level and denoted by the Greek letter  • We decide the  level before gathering evidence and state it along with the hypotheses • Then if the P-value is less than  the results of a hypothesis test are UNLIKELY TO HAPPEN BY CHANCE and considered STATISTICALLY SIGNIFICANT.

5. Example • Refer back to the ice cream scooping example • If we define  = .10, are the results statistically significant • For Gina? • For Jim? • In Gina’s case, the P-value was about 25% • This is not less than  so it is not statistically significant • We fail to reject Ho (or: We continue to assume Ho true) • In Jim’s case, the P-value was about 0.4% • Because this is less than  it is statistically significant • we reject Ho and say there is good evidence to support Ha

6. We need to modify the conclusion statement • The three-phrase conclusion statement for hypothesis tests needs to incorporate a comparison between P and  • Instead of saying “Because P is so (low/high)…” say “Because (P <  / P > )…” • Continue with the same two phrases as before: • “There (is / is not) good evidence to support the claim that…” • Restate Ha in context • Notice that we still do not “accept” or “prove” Ho • It was assumed true from the outset of the test • If any statement is made about Ho, it would be only that we reject (or fail to reject) the null hypothesis

7. Example Concluded • Here’s the new conclusion statement for Gina: • Because P (25%) >  (10%), there is not good evidence to support the claim that Gina’s mean scoop weight is more than 3.5 oz • Here’s the new conclusion statement for Jim: • Because P (0.4%) <  (10%), there is good evidence to support the claim that Jim’s mean scoop weight is more than 3.5 oz

8. New Example • Suppose we know that the SAT math test nationally follows the N(515, 100) distribution • Mrs. Riley claims that Northgate students score significantly better than the national average • She supports the claim with a study of 100 randomly chosen Northgate students whose average score was 535 • Clearly these results are above the national average. Is this just random chance based on the 100 scores she happened to choose, or is it statistically significant? • Perform a significance test to support the claim • Decide for yourself what significance level is appropriate and write it along with your hypotheses • Do the math (What is the distribution of x-bar? What is P?) • Write a conclusion sentence following the three-phrase form

9. State hypotheses and significance level • Ho: µ = 515 • Ha: µ > 515 •  = .10 or .05 or .01 (depends on your choice) • Gather evidence • We chose 100 random students • The sample mean was:  = 535 • Calculate the P-value • The distribution of  is N(515, 10) if Ho is true • Then the probability that one particular  is 535 or greater is found by the command: normalcdf(535, 999, 515, 10) = .023 • Write the conclusion • If you chose  = .01: Since P(2.3%) > (1%) there is not enough evidence to support the claim that the mean Northgate SAT math score is higher than the national average of 515 • If you chose  = .10 OR .05: Since P(2.3%) < (5%) there is good evidence to support the claim that the mean Northgate SAT math score is higher than the national average of 515

10. So what is the true NHS mean? • Use the calculator’s Stat:Test:ZInterval screen to form a 90% confidence interval for the true mean • Remember that σ = 100, Ë = 535, n = 100 • You should have found (518.55, 551.45) • Write a sentence that summarizes this finding • I am 90% confident that the true mean score of Northgate students is between 518.55 and 551.45

11. Today’s Objectives • Determine the significance level () to be used in a hypothesis test • Determine whether the results of a hypothesis test are statistically significant • Incorporate P-value and  into the conclusion statement of a hypothesis test California Standard 18.0 Students determine the P- value for a statistic for a simple random sample from a normal distribution.

12. Homework • Read pages 540 – 542 • Do problems 33, 35 – 38