S519: Evaluation of Information Systems

1. S519: Evaluation of Information Systems Social Statistics Inferential Statistics Chapter 8: Significantly significant

2. The “4 Steps” of Hypothesis Testing: • State the hypothesis • Set decision criteria • Collect data and compute sample statistic • Make a decision (accept/reject)

3. This week • Types of variables • Central Limit Theorem • Steps of hypothesis testing • Type I and Type II errors

4. Types of Variables (Between-subjects vs. Within-subjects) • Between-subjects variable: a characteristic that varies between subjects (e.g. age, gender etc.). • Within-subjects variable: a characteristic of individuals that varies with time. (the same individuals are compared at different points in time).

5. Types of Variables (independent vs. dependent ) • The independent variable is the one that is manipulated. • The dependent variable is the one that is observed.

6. Example I • A sport psychologist is interested in the relationship between reach and knockouts in boxers. She measures the reach of 20 pro boxers, and then observes the number of knockouts each scores over one year of fighting. • Between-subjects or within-subjects?

7. Example II • A researcher is testing the effect of alcohol on memory performance. He gives one group of subjects a bottle of vodka, and another a nonalcoholic substance that tastes like vodka. Each group then learns a list of words, and attempts to recall them. Number of words correctly recalled for each group is recorded. • Between-subjects or within-subjects? • What are the independent and dependent variables?

8. Example III • A social psychologist is interested in gender differences in math performance. She randomly selects men and women from IU and has them solve a series of equations. Number of equations correctly solved for each participant is recorded. • Between-subjects or within-subjects? • What are the independent and dependent variables?

9. A fallacy • In a survey, one of the questions asks respondents whether they had breakfast today, another asks them whether they are satisfied with their work today. • You found that 80% of the respondents who answered yes on the breakfast question said they were satisfied with their work. • You concluded that eating breakfast will make people feel satisfied about their work. • Justified or not?

10. A fallacy • Alternative I (between-subjects) • Two groups (same number of people, similar backgrounds, randomly chosen) • One have breakfast; one without having breakfast • Test their satisfaction towards work • Alternative II (within-subjects) • One group • Having breakfast the first day; without having breakfast the second day • Test their satisfaction towards work • Still, there are confounding factors (weather, order, etc.)

11. Distribution of Sample Means • So far, we’ve talked about samples of size 1. • In an experiment, we take a sample of several observations and try to make generalizations back to the population. • How do we estimate how good a representation of the population the sample we obtain is? • We can rely on the assumption that most populations are normally distributed, and apply the Central Limits Theorem.

12. Sampling Distributions • The distribution of sample means contains all sample means of a size n that can be obtained from a population.

13. Central Limit Theorem • For any population with mean μ and standard deviation σ, the distribution of sample means for sample size n will have a mean of μ and a standard deviation of , and will approach a normal distribution.

14. Central Limit Theorem • The Central Limit Theorem tells us that for any DSM of samples of size n: • μM =μ

15. Central Limit Theorem • The mean of the distribution of sample means is called the expected value of M. • The standard deviation of the distribution of sample means is called the standard error of M. standard error = σM • Standard deviation: standard distance between a score X and the population mean μ. • Standard error: standard distance between a sample mean M and the population mean μ.

16. Probability and the SampleMeans Distribution • SAT-scores (μ=500, σ=100). • What is the probability (M>540)? • If • Choose one randomly from the population; • Take sample n=25.

17. Step 1: State Hypothesis • Hypothesis is stated in terms of a population parameter (e.g. μ): assume null unless sufficient evidence to reject it • Null Hypothesis: no change in population parameter, treatment has no effect. H0 • Alternate (Scientific) Hypothesis: opposite of H0, treatment has an effect. H1 (directional or non directional)

18. Step 2: Set Criteria • By selecting a probability value (level of significance or alpha level), e.g. α = 0.05, 0.01, 0.001 • Critical region is region of extreme sample values (unlikely to be obtained if H0 is true). If sample data fall within the critical region, H0 is rejected. • Determine exact values for boundaries of critical region by using α and unit normal table.

19. Step 2: Set Criteria

20. Step 3: Collect Data/Statistics • Select random sample and perform “experiment”. • Compute sample statistic, e.g. sample mean. • Locate sample statistic within hypothesized distribution (use z-score). • Is sample statistic located within the critical region?

21. Step 4: Decision • Possibility: sample statistic is within critical region. Reject H0. • Possibility: sample statistic is not within critical region. Do not reject H0.

22. Hypothesis Testing: An Example • It is known that corn in Bloomington grows to an average height of μ=72 σ=6 six months after being planted. • We are studying the effect of “Plant Food 6000” on corn growth. We randomly select a sample of 40 seeds from the above population and plant them, using PF-6000 each week for six months. At the end of the six month period, our sample has a height of M=78 inches. Go through the steps of hypothesis testing and draw a conclusion about PF-6000 • 1. State hypotheses; 2. Chance model/critical region; 3. Collect data; 4. Decision and conclusion

23. Step 1: State Hypotheses • In words • Null: PF6000 will not have an effect on corn growth • Alt: PF6000 will have an effect on corn growth • In “code” symbols: • H0 :μ = 72 • H1 :μ ≠ 72

24. Step 2: Chance Model and Critical Values • a) Distribution of Sample Means: • b) Set alpha level α=.05 ∴ zcrit = ±1.96

25. Step 2: Chance Model and Critical Values • c) Compute critical values to correspond to zcrit

26. Step 3: Do Experiment • For the question, this part has already been done for us, we just need to compare this obtained sample mean to our chance model to determine if any discrepancy between our sample and the original population is due to: • 1. Sampling Error • 2. A true effect of our manipulation

27. Step 4: Decision and Conclusion • Mcrit is 70.14 (lower) or 73.86 (upper) • If Mobt exceeds either of these critical values (i.e., is out of the “chance” range, we reject H0. Otherwise, cannot reject H0) • Mobt = 78 Mcrit = 73.86 • Mobt exceeds Mcrit ∴ Reject H0 • Conclusion: We must reject the null hypothesis that the chemical does not produce a difference. Conclude that PF6000 has an effect on corn growth.

28. Uncertainty and Errors

29. Critical regions for different values of α

30. A template for significant test • 1. a statement of the null hypothesis • 2. setting a level of risk associated with the null hypothesis (level of significance or Type I error, p) • 3. select a proper statistical test (see Fig 8.1) • 4. set up the sample and experiment, and compute the test statistic value • 5. determine the value needed for rejection of the null hypothesis using proper tables – critical value (see appendix) • 6. compare the computed value and the obtained value • 7. if computed value > critical value: reject the null; if computed value < critical value: accept the null