Introduction to Biostatistical Analysis Using R Statistics course for first-year PhD students

1. Introduction to Biostatistical AnalysisUsing RStatistics course for first-year PhD students Session 2 Lecture: Introduction to statistical hypothesis testing Null and alternate hypothesis. Types of error. Two-sample hypotheses. Correlation. Analysis of frequency data. Introduction to statistical modeling Lecturer: Lorenzo Marini DAFNAE, University of Padova, Viale dell’Università 16, 35020 Legnaro, Padova. E-mail: lorenzo.marini@unipd.it Tel.: +39 0498272807 http://www.biodiversity-lorenzomarini.eu/

2. Inference A statistical hypothesis test is a method of making statistical decisions from and about experimental data. Null-hypothesis testing just answers the question of "how well do the findings fit the possibility that chance factors alone might be responsible?”. sampling Sample Estimation (Uncertainty!!!) Population testing Statistical Model

3. Key concepts: Session 1 Statistical testing in five steps: 1. Construct a null hypothesis (H0) and alterantive hypothesis 2. Choose a statistical analysis (assumptions!!!) 3. Collect the data (sampling) 4. Calculate P-value and test statistic 5. Reject/accept (H0) if P is small/large Remember the order!!! Concept of replication vs. pseudoreplication 1. Spatial dependence (e.g. spatial autocorrelation) 2. Temporal dependence (e.g. repeated measures) 3. Biological dependence (e.g. siblings) n=6 yi Key quantities residual y mean x

4. 1. Costruire e testare un’ipotesi Ipotesi: affermazione che ha come oggetto accadimenti nel mondo reale, che si presta ad essere confermata o smentita dai dati osservati sperimentalmente Esempio: gli studenti maschi e femmine presentano gli stessi voti

5. 1. Costruire e testare un’ipotesi Ipotesi nulla (H0): è un’affermazione riguardo alla popolazione che si assume essere vera fino a che non ci sia una prova evidente del contrario (status quo, mancanza di effetto etc.) Ipotesi alterantiva (Ha): è un’affermazione riguardo alla popolazione che è contraria all’ipotesi nulla e che viene accettata solo nel caso in cui ci sia una prova evidente in suo favore

6. 1. Costruire e testare un’ipotesi 1. Rifiutare H0 (e quindi accettare  Ha) Test di ipotesi consiste in una decisione fra H0 e Ha 2. Accettare H0 (e quindi rifiutare Ha)

7. 1. Costruire e testare un’ipotesi ? 1. Rifiutare H0 2. Accettare H0 La statistica inferenziale ci permette di quantificare delle probabilità per decidere se accettare o rifiutare l’ipotesi nulla: Quanto attendibile è H0?

8. Livello di significatività (alpha) Devo definire a priori una probabilità (alpha) per rifiutare l’ipotesi nulla Il livello di significatività di un test: probabilità di rifiutare H0, quando in realtà è vera (quanto confidenti siamo nelle nostre conclusioni?) Più piccola è alpha maggiore sarà la certezza nel rifiutare l’ipotesi nulla Valori usuali sono 10%, 5%, 1%, 0.1% I valori più comuni

9. Hypothesis testing • 1 – Hypothesis formulation (Null hypothesis H0 vs. alternative hypothesis H1) • 2 – Compute the probability P P-value is often described as the probability of seeing results as or more extreme as those actually observed if the null hypothesis was true • 3 – If this probability is lower than a defined threshold (level of significance: 0.01, 0.05) we can reject the null hypothesis

10. Hypothesis testing:Types of error As power increases, the chances of a Type II error decreases Statistical power depends on: -The statistical significance criterion used in the test -The size of the difference or the strength of the similarity (effect size) -Variability of the population -Sample size -Type of test

11. Statistical analyses Mean comparisons for 2 populations Test the difference between the means drawn by two samples Correlation In probability theory and statistics, correlation, (often measured as a correlation coefficient), indicates the strength and direction of a linear relationship between two random variables. In general statistical usage, correlation refers to the departure of two variables from independence. Analysis of count or proportion data Whole number or integer numbers (not continuous, different distributional properties) or proportion

12. Mean comparisons for 2 samples The t test H0: means do not differ H1: means differ • Assumptions • Independence of cases (work with true replications!!!) - this is a requirement of the design. • Normality - the distributions in each of the groups are normal • Homogeneity of variances - the variance of data in groups should be the same (use Fisher test or Fligner's test for homogeneity of variances). • These together form the common assumption that the errors are independently, identically, and normally distributed

13. Normality Before we can carry out a test assuming normality of the data we need to test our distribution (not always before!!!) Graphics analysis In many cases we must check this assumption after having fitted the model (e.g. regression or multifactorial ANOVA) hist(y) lines(density(y)) library(car) qq.plot(y) or qqnorm(y) RESIDUALS MUST BE NORMAL Test for normality Shapiro-Wilk Normality Test shapiro.test() Skew + kurtosis (t test)

14. Normality:Histogram

15. Normality:Histogram Normal distribution must be symmetrical around the mean library(animation) ani.options(nmax = 2000 + 15 -2, interval = 0.003) freq = quincunx(balls = 2000, col.balls = rainbow(1)) # frequency table barplot(freq, space = 0)

16. Normality:Q-Q Plot

17. Normality:Quantile-Quantile Plot Quantiles are points taken at regular intervals from the cumulative distribution function (CDF) of a random variable. The quantiles are the data values marking the boundaries between consecutive subsets

18. Normality In case of non-normality: 2 possible approaches 1. Change the distribution (use GLMs) Advanced statistics E.g. Poisson (count data) E.g. Binomial (proportion) 2. Data transformation Logaritmic (skewed data) Square-root Arcsin (percentage) Probit (proportion) Box-Cox transformation

19. Homogeneity of variance: two samples Before we can carry out a test to compare two sample means, we need to test whether the sample variances are significantly different. The test could not be simpler. It is called Fisher’s F To compare two variances, all you do is divide the larger variance by the smaller variance. E.g. Students from TESAF vs. Students from DAFNAE F calculated F<-var(A)/var(B) qf(0.975,nA-1,nB-1) F critical if the calculated F is larger than the critical value, we reject the null hypothesis Test can be carried out with the var.test()

20. Homogeneity of variance : > two samples It is important to know whether variance differs significantly from sample to sample. Constancy of variance (homoscedasticity) is the most important assumption underlying regression and analysis of variance. For multiple samples you can choose between the Bartlett test and the Fligner–Killeen test. Bartlett.test(response,factor) Fligner.test(response,factor) There are differences between the tests: Fisher and Bartlett are very sensitive to outliers, whereas Fligner–Killeen is not

21. Mean comparison In many cases, a researcher is interesting in gathering information about two populations in order to compare them. As in statistical inference for one population parameter, confidence intervals and tests of significance are useful statistical tools for the difference between two population parameters. Ho: the two means are the same H1: the two means differ • - All Assumptions met?Parametric t.test() • - t test with independent or paired sample -Some assumptions not met?Non-parametric Wilcox.test() - The Wilcoxon signed-rank test is a non-parametric alternative to the Student's t-test for the case of two samples.

22. Il test t Misura legata alla differenza fra le medie tcalcolato= Misura di variabilità dentro i gruppi Differenza medie Variabilità dei gruppi

23. Il test t Caso 1 Caso 2 Variabile Differenza fra le medie Variabilità A B A B A Variabilità B Caso 3 Caso 4 Variabile B A B A

24. Il test t Differenza fra le medie tcalcolato= Errore standard della differenza t t Differenza fra medie Variabilità dentro i gruppi Più estremo sarà t calcolato minore sarà P maggiore sarà la probabilità di rifiutare H0

25. Il test t Differenza fra le medie tcalcolato= Errore standard della differenza + estremo sarà tcalcolato maggiore la probabilità di rifiutare H0 P Tcritico -Tcritico

26. Come scegliere il test t giusto a partire dalle assunzioni Indipendenza NO SÌ Test t appaiato Test t non appaiati Test t per pop. omoschedastiche Test t per pop. eteroschedastiche Welch t-test (formula complessa richiesto un PC)

27. Campioni independenti omoschedastici: Test t! ? Varianza combinata (”pooled”) I gradi di libertà sono n1 + n2-2 per Tcritico

28. Campioni independenti omoschedastici: Test t! H0: le due medie sono uguali Ha: le due medie sono diverse • Test di ipotesi: • Calcolo la varianza combinata dei due campioni • Determino il valore di tcalcolato • Decido il livello di significatività (alpha, 1 o 2 code?) • Determino il valore di tcritico • Se |tcalcolato|> |t critico| rifiuto H0 • Conclusione: le medie sono DIVERSE! I gradi di libertà sono n1+n2-2 per Tcritico

29. Campioni appaiati: 2 casi 1. Misure ripetute 2. Correlazione nello spazio Misura a monte Misura a valle Fiume B Fiume C Fiume A Industria tessile [Ammoniaca] in acqua

30. Campioni appaiati: Test t Media delle differenze Deviazione standard delle differenze Numero di coppie I gradi di libertà sono n-1 per tcritico

31. Campioni appaiati: Test t H0: le due medie sono uguali Ha: le due medie sono diverse ? • Test di ipotesi: • Determino il valore di tcalcolato • Decido il livello di significatività (alpha, 1 o 2 code?) • Determino il valore di tcritico • Se |tcalcolato|> |tcritico| rifiuto H0 • Conclusione: le medie sono DIVERSE! I gradi di libertà sono n-1 per tcritico

32. Non parametrica: Wilcoxon I ranghi n1 and n2 sono I numeri delle osservazioni R1 è la somma dei rnaghi nel campione 1 Test can be carried out with the wilcox.test() function

33. Correlation Correlation, (often measured as a correlation coefficient), indicates the strength and direction of a linear relationship between two random variables Bird species richness Plant species richness 1 2 3 4 … 458 x1 x2 x3 x4 … x458 l1 l2 l3 l4 … l458 Sampling unit Three alternative approaches 1. Parametric - cor() 2. Nonparametric - cor() 3. Bootstrapping - replicate(), boot()

34. Correlation: causal relationship? Which is the response variable in a correlation analysis? NONE Bird species richness Plant species richness 1 2 3 4 … 458 x1 x2 x3 x4 … x458 l1 l2 l3 l4 … l458 Sampling unit

35. Correlation Plot the two variables in a Cartesian space A correlation of +1 means that there is a perfect positiveLINEARrelationship between variables. A correlation of -1 means that there is a perfect negative LINEAR relationship between variables. A correlation of 0 means there is no LINEAR relationship between the two variables.

36. Correlation Same correlation coefficient! r= 0.816

37. Parametric correlation: when is it significant? Pearson product-moment correlation coefficient Correlation coefficient: Hypothesis testing using the t distribution: Ho: Is cor = 0 H1: Is cor ≠ 0 t critic value for d.f. = n-2 • Assumptions • Two random variables from a random populations • - cor() detects ONLY linear relationships

38. Nonparametric correlation Rank procedures Distribution-free but less power Spearman correlation index The Kendall tau rank correlation coefficient P is the number of concordant pairs n is the total number of pairs

39. Issues related to correlation 1. Temporal autocorrelation Values in close years are more similar Dependence of the data 2. Spatial autocorrelation Values in close sites are more similar Dependence of the data Moran's I = 0 Moran's I = 1 Moran's I or Geary’s C Measures of global spatial autocorrelation

40. Three issues related to correlation 2. Temporal autocorrelation Values in close years are more similar Dependence of the data Working with time series is likely to have temporal pattern in the data E.g. Ring width series Autoregressive models (not covered!)

41. Three issues related to correlation 3. Spatial autocorrelation Values in close sites are more similar Dependence of the data ISSUE: can we explain the spatial autocorrelation with our models? Moran's I or Geary’s C (univariate response) Measures of global spatial autocorrelation Raw response Residuals after model fitting Hint: If you find spatial autocorrelation in your residuals, you should start worrying

42. Estimate correlation with bootstrap BOOTSTRAP Bootstrapping is a statistical method for estimating the sampling distribution of an estimator by sampling with replacement from the original sample, most often with the purpose of deriving robust estimates of SEs and CIs of a population parameter Sampling with replacement >a<-c(1:5) > a [1] 1 2 3 4 5 > replicate(10, sample(a, replace=TRUE)) [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [1,] 2 3 2 1 4 2 1 2 1 3 [2,] 1 5 2 3 5 3 1 1 3 2 [3,] 4 4 4 5 4 4 5 1 1 5 [4,] 4 1 1 3 3 2 3 1 5 2 [5,] 5 5 1 3 5 2 4 1 5 4 1 original sample 10 bootstrapped samples

43. Estimate correlation with bootstrap Why bootstrap? It doesn’t depend on normal distribution assumption It allows the computation of unbiased SE and CIs Statistic distribution Sample Bootstrap Quantiles N samples with replacement …

44. Estimate correlation with bootstrap CIs are asymmetric because our distribution reflects the structure of the data and not a defined probability distribution If we repeat the sample n time we will find 0.95*n values included in the CIs

45. Frequency data • Properties of frequency data: • Count data • Proportion data Count data: where we count how many times something happened, but we have no way of knowing how often it did not happen (e.g. number of students coming at the first lesson) Proportion data: where we know the number doing a particular thing, but also the number not doing that thing (e.g. ‘mortality’ of the students who attend the first lesson, but not the second)

46. Count data Straightforward linear methods (assuming constant variance, normal errors) are not appropriate for count data for four main reasons: • The linear model might lead to the prediction of negative counts. • The variance of the response variable is likely to increase with the mean. • The errors will not be normally distributed. • Many zeros are difficult to handle in transformations. - Classical test with contingency tables - Generalized linear models with Poisson distribution and log-link function (extremely powerful and flexible!!!)

47. Count data: contingency tables We can assess the significance of the differences between observed and expected frequencies in a variety of ways: - Pearson’s chi-squared (χ2) - G test - Fisher’s exact test H0: frequencies found in rows are independent from frequencies in columns

48. Count data: contingency tables - Pearson’s chi-squared (χ2) We need a model to define the expected frequencies (E) (many possibilities) – E.g. perfect independence Critic value X

49. Count data: contingency tables - G test 1. We need a model to define the expected frequencies (E) (many possibilities) – E.g. perfect independence χ2 distribution - Fisher’s exact test fisher.test() If expected values are less than 4 o 5

50. Proportion data Proportion data have three important properties that affect the way the data should be analyzed: • the data are strictly bounded (0-1); • the variance is non-constant (it depends on the mean); • errors are non-normal. - Classical test with probit or arcsin transformation - Generalized linear models with binomial distribution and logit-link function (extremely powerful and flexible!!!)