Some basic statistical tests & more on basic statistical analysis

Some basic statistical tests & more on basic statistical analysis Communication Research Week 11 with help from: Carey, J & Dimmitt, C. (2003) Statistical Analysis: Is Change Real? www.umass.edu/schoolcounseling/ WelcometoAmherstMassachusetts/StatisticalAnalysis.ppt [accessed 10 Oct 2006] http://www.statsoft.com/textbook/stathome.html

Why Statistical Analysis? • After we gather and compute our data, we want to be sure that the scores of two groups really are different. • We want to be sure that the differences we see are not just due to chance. • If we are basing decisions on real differences our behavior is directed and purposeful. • If we are basing decisions on differences that are only due to chance our behavior is random and chaotic. Communication Research

Statistical Tests • Allow us to estimate the likelihood that the apparent differences between groups are real and not due to chance. • These tests have the built in capacity to take the number of people per group and the variability of the data into account when making these estimates. Communication Research

Measuring Variables • Variables – the things we measure, control or manipulate • Independent variables(IV) are usually those that are manipulated • Dependent variables(DV) are only measured or registered • They differ in how well they can be measured and the type of measurement scale used • Two or more variables are related if, in a sample of observations, the values systematically correspond to each other for these observations • eg height is considered related to weight because typically tall people are heavier than short ones; IQ is related to the number of errors in a test, if people with higher IQs make fewer errors Communication Research

Why are relations between variables considered important? • The philosophy of science believes that there is no other way of representing “meaning” except in terms of relations between some quantities or qualities • Statistical significance(p-value) of a result is the probability that the observed relationship (eg between the variables) or a difference (eg between the means) in a sample occurred by pure chance (“luck of the draw”) and that in the populations from which the sample was drawn, no such relationship or differences exist • In other words, the statistical significance of a result tells us something about the degree to which the result is “true” (ie representative of the population) Communication Research

Example – "Baby boys to baby girls ratio." • Consider the following example from research on statistical reasoning (Nisbett, et al., 1987). There are two hospitals: in the first one, 120 babies are born every day, in the other, only 12. On average, the ratio of baby boys to baby girls born every day in each hospital is 50/50. However, one day, in one of those hospitals twice as many baby girls were born as baby boys. In which hospital was it more likely to happen? • The answer is obvious for a statistician, but as research shows, not so obvious for a lay person: It is much more likely to happen in the small hospital. • The reason for this is that technically speaking, the probability of a random deviation of a particular size (from the population mean), decreases with the increase in the sample size. Communication Research

Data characteristics that help determine the statistical test used • Type of data used – nominal, ordinal, interval, ratio • Two groups vs more than two groups • Whether groups are matched (“paired”) or unmatched • Whether groups are small or large • Whether the data are normally distributed (continuous data) Communication Research

Different data/variable types Communication Research

Example 2: After implementation of a family math education intervention, Latino/a students average 4th Grade MCAS scaled score increased from 206 to 215. • Hypothesis (Ha). The two groups are really different. • Null Hypotheses (Ho). The two groups are not different, the apparent difference is due to chance. Communication Research

Example 2: After implementation of a family math education intervention, Latino/a Students’ average 4th Grade MCAS score increased from 206 to 215. • The variability of the outcome data is a major factor in determining whether the differences are real or due to chance. • At the TAB, in a straight bet, how much would you be willing to wager the Ha is true if you knew that, if students retake the MCAS within a month • 90% of the time their two scores differ by less than 2 points. • 90% of the time their two scores differ by less that 10 points. • 90% of the time their two scores differ by less than 50 points. Communication Research

Example 1: 70% of White and 40% of African American 3rd graders score Advanced or Proficient on the MCAS Reading Test. • The number of people in two groups is a major factor in determining whether differences are real or due to chance. • At the TAB, in a straight bet, how much would you be willing to wager the hypothesis is true if you knew that: • The percentages are based on 10 students from each group. • The percentages are based on 50 students from each group. • The percentages are based on 100 students from each group. Communication Research

Parametric vs non parametric tests • Since we have two types of data we need two types of statistical tests. • Parametric – the DV is a continous variable (eg age in years) so it makes sense to calculate the mean and SD • Non parametric – the DV is a count (nominal data) or a ranking (ordinal data) and so it makes no sense to measure a means eg “the average gender of Australians is 1.5” • Parametric Tests are generally more powerful, meaning that if there is a real difference between the groups its easier to find it with a Parametric Test Communication Research

Examples of parametric tests • Independent t-test or a comparison of two means • Looks for a difference between two groups (eg men and women) on a particular variable (eg whether they kiss on the first date) • Paired t-test eg such as how someone feels about drink driving before they get caught, and how they feel about it afterwards. • In an SPSS output table, the Sig (2-tailed) value is the significance value – the likelihood that the result could happen by pure chance. • If the value is less than 0.05, the chance is less than 5%, so the significance of the difference is 95% – this is therefore highly significant Communication Research

Choosing a significance level • Statistical Tests do not give us information that allows us to definitively say whether an observed difference between groups is real or just due to chance. • Statistical Tests do give us an estimate of the likelihood that observed difference between groups results from chance. • We must decide what criteria we will use for deciding whether a difference is real. Communication Research

Choosing a Significance Level • We do this by choosing a Significance Level • .25 25% chance difference is due to chance • .10 10% chance difference is due to chance • .05 5% chance difference is due to chance • .01 1% chance difference is due to chance Communication Research

T-Test in SPSS SPSS will allow you to do all of these tests quite easily Communication Research

If the value is less than 0.05, the chance is less than 5%, so the significance of the difference is 95% which is highly significant. Levene’s test, checks to see whether the variances of the two variables are relatively similar. If the significance for Levene's test is 0.05 or below, then the ‘Equal Variances Not Assumed’ t-test result (the one on the bottom) is used. Otherwise you use the ‘Equal Variances Assumed’ test (the one on the top) Communication Research

T-Test for Independent Samples • Remember, we need to know two other things in order to ascertain the likelihood of chance creating this size of a difference: • The number of people in each group • The variability of the scores • The number of people is easy, and is counted in the frequency table (n) Communication Research

Variability • In order to know whether a difference between two means is important, we need to know how much the scores vary around the means. Communication Research

Variability • Holding the difference between the means constant • With High Variability the two groups nearly overlap • With Low Variability the two groups show very little overlap Communication Research

Measuring Variability • Medium Variance • High Variance • Low Variance Communication Research

Measuring Variability • Usually it’s easier to work with the square root of the variance. • This statistic is called the Standard Deviation. • SPSS statistical tests will calculate the SD for you Communication Research

ANOVA • ANOVA is an acronym; ANalysis Of VAriance. • It is an extension of the two-tailed t-test, and is generally used to test for significant differences between means. • The name is derived from the fact that in order to test for statistical significance between means, we actually compare (or analyse) variances. • For two-group comparisons, ANOVA will give results identical to a t-test, but when the design is more complex, ANOVA offers numerous advantages that t-tests cannot provide (even if you run a series of t- tests comparing various cells of the design). • For example, it often happens in research practice that you need to compare more than two groups (e.g., drug 1, drug 2, and placebo), or compare groups created by more than one independent variable while controlling for the separate influence of each of them (such as Gender, Type of Drug, and Size of Dose). Communication Research

Communication Research

Independent T test Communication Research

Dependent T-test Communication Research

Oneway ANOVA Communication Research

Repeated Measures ANOVA Communication Research

Factorial ANOVA Communication Research

Mixed ANOVA Communication Research

Chi2 Test • Chi Square (X2) - is a non-parametric test • It uses nominal data and checks to see if there are significant differences between/among groups compared to what would be expected. • The crosstabulation table tells you whether selected variables are related to other selected variables; the chi-square table tells you what the degree of certainty is. • Chi-square is based on the fact that for a two-way table, we can compute the frequencies that we would expect if there was no relationship between the variables. Communication Research

Chi2 Example • Suppose we ask 20 men and 20 women to choose between two brands of soft drink - brands A and B. If there is no relationship between preference and gender, then we would expect about an equal number of choices of brand A and brand B for each gender. • The chi-square test becomes increasingly significant as the numbers deviate further from this expected pattern; that is, the more this pattern of choices for men and women differs. Communication Research

If our Chi2 test statistic has exceeded the critical value for: • The .10 significance level it would mean that there was only a 10% chance of seeing a difference that large that resulted from chance. • The .05 significance level it would mean that there was only a 5% chance of seeing a difference that large that resulted from chance. • The .025 significance level it would mean that there was only a 2.5% chance of seeing a difference that large that resulted from chance. Communication Research

If our Chi2 test statistic has exceeded the critical value for: • The .01 significance level it would mean that there was only a 1% chance of seeing a difference that large that resulted from chance. • The .005 significance level it would mean that there was only a 0.5% chance of seeing a difference that large that resulted from chance. Communication Research

Pearson correlation • The Pearson correlation looks for a relationship between two variables and generates a mathematical index of the relationship between them. • The value lies between -1.00 and +1.00, and the bigger the number the stronger the relationship. • Pearson correlation indicates trends - one thing increases (or decreases) as another thing increases (or decreases). • A negative value indicates that low scores on one variable go with high scores on the other variable, while a positive value indicates that a high score of one variable goes with a high score on the other variable. Communication Research

Note that SPSS will allow you to calculate different tests of significance as well as note significant correlations Communication Research

Some basic statistical tests & more on basic statistical analysis