Thinking by Hypothesis Testing

1. Thinking by Hypothesis Testing ...when you have eliminated the impossible, whatever remains, however improbable, must be the truth. - Sherlock Holmes / Arthur Conan Doyle

2. Thinking by Hypothesis Testing • You go out to your car and it will not start. How do you explain this? How do you solve this problem? • Step 1: Generate possible explanations and solutions • Step 2: Evaluate each one for possible outcomes • Step 3: Execute the solution that results in the best outcome • The possible explanations are hypotheses • The process of evaluation is an experiment

3. Simple Example of Hypothesis Testing • Traumatic Brain Injury causes millions of neurons to become hyperactive and this damages the neurons. • A drug company invents a medication that suppresses this process. • Will the drug work as an effective treatment for Traumatic Brain Injury? • Research Hypothesis (H1): The drug reduces the damage and the patient will have a better recovery. The drug is effective. • Null Hypothesis (H0): The drug is not effective

4. Steps of Hypothesis Testing • Step 1: Restate the research question as a research hypothesis and a null hypothesis - refer to populations • Population 1: Trauma patients who receive the drug • Population 2: Trauma patients who receive placebo • Research Hypothesis: The mean IQ of Population 1 will be higher than the mean IQ of Population 2 • The population means: 1 > 2 • Null Hypothesis: The mean IQ of Population 1 will be the same as the mean IQ of Population 2 • The population means: 1 = 2

5. Steps of Hypothesis Testing • Step 1(continued) • Research Hypothesis: The mean IQ of Population 1 will be higher than the mean IQ of Population 2. The population means: 1 > 2

6. Steps of Hypothesis Testing • Step 1 (continued): • Null Hypothesis: The mean IQ of Population 1 will be the same as the mean IQ of Population 2. The population means: 1 = 2

7. Steps of Hypothesis Testing • Step 2: Determine the characteristics of the comparison distribution • This is the distribution of scores you expect if there is no experimental manipulation (e.g. no treatment of the trauma patient = mean IQ of 83; SD=12)

8. Steps of Hypothesis Testing • Step 3: Determine a cutoff sample score or critical value • How extreme must the score be before you conclude it could not occur by chance? • How extreme must a score be to reject the Null Hypothesis? • Researchers usually reject the Null Hypothesis if the probability of the sample result is less than 5% (p<.05). This is the level of significance.

9. Steps of Hypothesis Testing • Step 4: Determine your sample’s score on the comparison distribution • Carry out the study and collect a sample • Figure the Z score for the sample’s raw score using the comparison distribution

10. Steps of Hypothesis Testing • Step 5: Decide whether to reject the Null Hypothesis • Compare the Z score for the sample to the cutoff Z score for the critical value • If the Z score for the sample is less than the Z score critical value then the Null Hypothesis is rejected and you accept the Research Hypothesis • Since it is based on probabilities, Rejecting the Null Hypothesis is not a proof

11. One and Two-Tailed Tests If the hypothesis specifies the direction of the effect then you use only one tail of the comparison distribution. This is a one-tailed, or directional hypothesis. Hence, you will use a one-tailed test of the hypothesis If the hypotheses do not specify a direction for the effect then you use both tails of the comparison distribution. This is a two-tailed test of the hypothesis. Since you are using both tails then the significance probability (e.g. 5%) must be split over both tails. This makes the two-tailed test more conservative.

12. One-Tailed Test • Use the significance level to make one cutoff on one side of the comparison distribution • If the cutoff is on the low side, determine if your sample score is lower than the cutoff • If the cutoff is on the high side, determine if your sample score is higher than the cutoff • If the sample score is more extreme than the cutoff, reject the null hypothesis

13. Two-Tailed Test • Divide the significance level in half • Use the halved sig. level to make one cutoff on the low side of the comparison distribution • Then use it to make another cutoff on the high side of the comparison distribution • Determine if your sample score is either lower than the low cutoff, or higher than the high cutoff • If it is, reject the null hypothesis

14. One-Tailed Example • General Motors knows that the average time to manufacture a car in their factories is 60 minutes, with an SD of 5 minutes. They want to know if a new robot can reduce this time. They try it out on one car, and the car takes 55 minutes to make. At the 5% significance level, did the robot work?

15. Two-Tailed Example • General Motors knows that the average time to manufacture a car in their factories is 70 minutes, with an SD of 8 minutes. They want to know if a new robot can make the process faster, but they are worried that the robot might actually make the process slower. They try it out on one car, and the care takes 88 minutes to make. At the 5% significance level, did the robot make a difference?

16. The Downside of Hypothesis Testing Many researchers conclude that the Null Hypothesis is true when they get an insignificant result. An insignificant result may occur for a number of reasons. Significance testing usually does not report effect sizes. Statistically significant results may be clinically insignificant.