Chapter Eleven

1. Chapter Eleven Inferential Tests of Significance I: t tests – Analyzing Experiments with Two Groups PowerPoint Presentation created by Dr. Susan R. BurnsMorningside College Smith/Davis (c) 2005 Prentice Hall

2. Inferential Statistics • Once you have conducted a two-group experiment, you need to perform a statistical test on the data you gathered. • The results of this test will help you decide if the IV was effective or not. • Inferential statistical tests allow you to make an inference about your results – you will decide if your results are significant. Unlike the warning on this truck, psychologists view data and statistical procedures as tools to help answer research questions Smith/Davis (c) 2005 Prentice Hall

3. The Relation Between Experimental Design and Statistics • Selecting the appropriate design determines the particular statistical test you will use to analyze your data. • Thus, you should decide upon your experimental design before you start collecting data to ensure there will be an appropriate statistical test to analyze your data. Smith/Davis (c) 2005 Prentice Hall

4. The Logic of Significance Testing • The result of an experiment is statistically significant is when it occurs rarely by chance. Smith/Davis (c) 2005 Prentice Hall

5. The Logic of Significance Testing • What is Significant? • The results of an inferential statistical test tell us whether the results of an experiment would occur frequently or rarely by chance. • Inferential statistics with small values occur frequently by chance (accept the null hypothesis), whereas large values occur rarely by chance (accept the alternative hypothesis). • Traditionally psychologists say that any event that occurs by chance alone 5 times or fewer in 100 occasions is a rare event. (i.e., .05 level of significance). Smith/Davis (c) 2005 Prentice Hall

6. The Logic of Significance Testing • Samples and Populations – our interest is not in the samples we have tested in an experiment, but in what these samples tell us about the population from which we drew them. That is, we want to generalize, or infer, from our samples to the larger population using the following steps: • Random selecting a sample from a specified population. • Randomly assigning participants in the sample to different groups. • Apply the manipulation of our independent variable. • Assuming the groups are significantly different, we generalize the findings from our experiment to the population of interest. Smith/Davis (c) 2005 Prentice Hall

7. Samples and Populations • Random samples are drawn from a population (Part A) • The administration of the IV causes the samples to differ significantly (Part B) • The experimenter generalizes the results from the experiment to the general population (Part C) Smith/Davis (c) 2005 Prentice Hall

8. Analyzing Two-Group Designs • The appropriate statistical test is a t test (assuming you have interval or ratio-level data). • For a two-independent-groups design, you would use a t test for independent samples (known as an independent t test). This test is used when you randomly assign your participants to the two groups. • For a two-correlated-groups design, you would analyze the data with a t test for correlated samples (known as a dependent t test, within-groups t test, or paired t test). This test is used when you use repeated measures, matched pairs, or natural pairs. Smith/Davis (c) 2005 Prentice Hall

9. The Logic of Significance Testing • An example of a study where you could use the t-test for independent samples is examining how long it takes salesclerks to wait on customers in dressy versus sloppy clothes. Smith/Davis (c) 2005 Prentice Hall

10. The t Test for Independent Samples • The formula for an independent samples t test is as follows: • Calculating the standard error of the difference between means takes several steps and requires different formulas depending on whether the two groups are equal or different in size. • We must calculate the standard error of the difference because we have two groups that contribute to overall variability rather than only one as when we previously encountered the standard error of the mean (SEM). Smith/Davis (c) 2005 Prentice Hall

11. Standard Error of the Mean (SEM) Smith/Davis (c) 2005 Prentice Hall

12. Calculating t – Test for Independent Groups • Step by step calculation of t for the example is provided in Table 11-2 Smith/Davis (c) 2005 Prentice Hall

13. The t Test for Independent Samples • Once you have calculated the t value, you must follow several steps in order to interpret its meaning: • Determine the degrees of freedom (df) involved: • df = (NA – 1) + (NB – 1) • Use the degrees of freedom to enter a t table that contains t values that occur by chance. • To be significant, the calculated t must be equal or larger than the one in the table. • Look for the appropriate level of significance (i.e., .05 level) to determine significance. Smith/Davis (c) 2005 Prentice Hall

14. The t Test for Independent Samples • Statistical Issues • Matched-group tests tend to reduce error variability. • However, this advantage occurs only if you have matched your participants on a relevant variable. • If you match on an irrelevant variable, you error variability does not decrease, but your degrees of freedom do. Smith/Davis (c) 2005 Prentice Hall

15. Computer Analysis of t Test for Independent Samples • Different statistical packages provide slightly different information. • Typically you will first see descriptive statistics (i.e., means and standard deviations). • Next, you will see two statistical values: F and p. • The Fmaxis a statistic used to test the assumption of homogeneity of variance for the two groups (i.e., the variability of the scores of the two groups is similar). • To use a t test, we assume the variances are similar. Smith/Davis (c) 2005 Prentice Hall

16. Computer Analysis of t Test for Independent Samples • Different statistical packages provide slightly different information. • Typically you will first see descriptive statistics (i.e., means and standard deviations). • Next, you will see two statistical values: F and p. • The Fmaxis a statistic used to test the assumption of homogeneity of variance for the two groups (i.e., the variability of the scores of the two groups is similar). • If the p value for this test exceeds .05 we assume homogeneity of variance, if it is smaller than .05, we would find heterogeneity of variance, meaning that the variability of the scores of the two groups is not comparable. • Thus, you would be violating a mathematical assumption for using the t test. In that instance, you can use the “Equal Variances Not Assumed” information. Smith/Davis (c) 2005 Prentice Hall

17. Computer Analysis of t Test for Independent Samples • You, next, will find the t value and probability level. • Some statistical packages may not automatically print the degrees of freedom, so it is important for you to remember how to calculate df. • Some statistical packages also do not provide the probability of your result as a part of the printout; and thus you would need to make this determination for yourself. • Finally, your printout may show Cohen’s d, which is a measure of the effect size. • Translating Statistics Into Words – to communicate statistical results of an experiment, we use APA format for clarity and conciseness. Smith/Davis (c) 2005 Prentice Hall

18. Calculating d after t Test for Independent Groups Smith/Davis (c) 2005 Prentice Hall

19. The t Test for Correlated Samples • If you used correlated groups such as matched pairs, repeated measures, or natural pairs for your design, then you will need to use the t test for correlated samples. • Calculating t by hand – • The formula for the correlated t test is as follows: • In words, t equals the mean of x minus the mean of y, divided by the standard error of the deviation scores. Smith/Davis (c) 2005 Prentice Hall

20. Smith/Davis (c) 2005 Prentice Hall

21. Computer Analysis of t Test for Correlated Samples • Again you will first see descriptive statistics followed by the block of information that shows the size of the differences between the two means, as well as its standard deviation and standard error (this information is rarely used by researchers and thus may not be included in the printout). • The next bit of information explains about the relation between the pairs of participants (or the same participant if using repeated measures). • Here is where you determine whether the paired scores were correlated (you want them to be so you can gain additional statistical control). Smith/Davis (c) 2005 Prentice Hall

22. Computer Analysis of t Test for Correlated Samples • Next, you will see the inferential statistic. • The degrees of freedom for correlated sample cases are equal to the number of pairs of participants minus one. • Translating Statistics Into Words – to communicate statistical results of an experiment, we use APA format for clarity and conciseness. Smith/Davis (c) 2005 Prentice Hall

23. Computer Analysis of t Test for Correlated Samples Smith/Davis (c) 2005 Prentice Hall

24. Important Reminders • One-Tailed and Two-Tailed Significance Tests – if you stated your experimental hypothesis in a directional manner, then you will use a one-tailed test of significance. If you used a nondirectional hypothesis you would use a two-tailed test of significance. • Type I and Type II Errors – Type I error (alpha, α) refers to accepting the experimental hypothesis when the null hypothesis is in fact true. Rejecting a true experimental hypothesis is called a Type II error. Smith/Davis (c) 2005 Prentice Hall

25. The Continuing Research Problem • Research is a continuous process. • It is rare for a psychologist to conduct a single research project and stop at that point because that one project had answered all the questions about the particular topic. • Rather, one experiment usually answers some of your questions, does not answer others, and raises new ones for consideration. Smith/Davis (c) 2005 Prentice Hall