2 nd Level GLM

1. 2nd Level GLM Emily Falk, Ph.D.

2. (De-noise) Realign Slice Timing Correct Smooth Predictors AcquireFunctionals Y X y = Xβ + ε Template 1st level (Subject) GLM Determine Scanning Parameters β βhow - βwhy Co-Register Normalize Acquire Structurals (T1) Contrast All subjects 2nd level (Group) GLM Threshold

3. Groups of Subjects • So far: Analyzing each individual voxel from one person • How do subjects combine data from groups of subjects? • Often referred to as 2nd-level random effects analysis • Basic approach: • Normalize SPMs from each subject into a standard space • Test whether statistic from a given voxel is significantly different from 0 across subjects • Correct for multiple comparisons

4. An Example -

5. Region A: β1-β2 (repeat for all regions) Subject 1: 32 Subject 2: 18 Subject 3: -4 Subject 4: 45 Subject 5: 23 Mean : * 22.8 (Aron et al., 2005)

6. Fixed and Random Effects • Fixed effect • Always the same, from experiment to experiment, levels are not draws from a random variable • Sex (M/F) • Drug type (Prozac) • Random effect • Levels are not randomly sampled from a population • Subject • Day, in a longitudinal design • If effect is treated as fixed, error terms in model do not include variability across levels • Cannot generalize to unobserved levels • e.g., if subject is fixed, cannot generalize to new subjects Courtesy of Tor Wager

7. Fixed vs. Random Effects: Bottom Line • If I treat subject as a fixed effect, the error term reflects only scan-to-scan variability, and the degrees of freedom are determined by the number of observations (scans). • If I treat subject as a random effect, the error term reflects the variability across subjects, which includes two parts: • Error due to scan-to-scan variability • Error due to subject-to-subject variability and degrees of freedom are determined by the number of subjects. Courtesy of Tor Wager

8. Random Effects Analysis • Subjects treated as “random” effect • Randomly sampled from population of interest • Sample is used to make estimates of population effects • Results lead to inferences on the population

9. Random vs. Fixed Effects • Whereas some early studies used fixed effects models, virtually all current studies use random effects models • Use random effects • All analysis that follow treat subject as a random effect

10. More specifically…

11. Model specification Parameter estimation Hypothesis Statistic Voxel-Wise 2nd-Level Analysis Subject Normalized SPMs from different subjs Statistic at that voxel single voxel subj series SPM

12. Model Specification:Building the Design Matrix Stat Value Design matrix Residuals Model parameters = + Subjects X intercept

13. Parameter Estimation/Model Fitting Find  values that produce best fit to observed data y =  0 + ERROR

14. The SPM Way of Plotting the Variables y X e + = X

15. One-sample t-test @ 2nd level Group Analysis Using Summary Statistics: A simple kind of ‘random effects’ modelThe “Holmes and Friston” approach (HF) First level Second level DataDesign MatrixContrast Images SPM(t) Courtesy of Tor Wager

16. Summary Statistic Approach: 2 Sample t-test from Mumford & Nichols, 2006

17. Summary Statistic Approach: Inference • In a 1-sample t-test, the contrast C = 1 derives the group mean • If images taken to a second level represent the contrast A – B, then • C = 1 is the mean difference (A > B) • C = -1 is the mean difference (B > A) • Dividing by the standard error of the mean yields a t-statistic • Degrees of freedom is N – 1, where N is the number of subjects • Comparison of the t-statistic with the t-distribution yields a p-value • P(DataNull)

18. Tech Note: Sufficiency of Summary Statistic Approach • With simple t-tests under the summary statistic approach, within-subject variance is assumed to be homogenous (within a group) • SPM’s approach, but other packages can act differently • If all subjects (within a group) have equal within-subject variance (homoscedastic), this is ok • If within-subject variance differs among subjects (heteroscedastic), this may lead to a loss of precision • May want to weight individuals as a function of within-subject variability • Practically speaking, the simple approach is good enough (Mumford & Nichols, 2009, NeuroImage) • Inferences are valid under heteroscedasticity • Slightly conservative under heteroscedasticity • Near optimal sensitivity under heteroscedasticity • Computationally efficient

19. For extended example of ways that you could do this wrong, check out Derek Nee’s second level GLM lecture from last year

20. The GLM Family DV Predictors Analysis Regression Continuous One predictor Multiple Regression Continuous Two+ preds One continuous 2-sample t-test Categorical 1 pred., 2 levels General Linear Model One-way ANOVA Categorical 1 p., 3+ levels Factorial ANOVA Categorical 2+ predictors Two measures, one factor Paired t-test Repeated measures More than two measures Repeated measures ANOVA

21. Correlations • To perform mass bi-variate correlations, use SPM’s “Multiple Regression” option with a single co-variate • Can also specify multiple co-variates and perform true multiple regression • Be cautious of multi-collinearity! • Correlations are done voxel-wise • % of explained variance necessary to reach significance with appropriate correction for multiple comparisons may be very high • Interpret location, not effect size (more later) • May be more realistic to perform correlations on a small set of regions-of-interest (more later)

22. Examples • First level: Why > How • Regression with… • trait empathy • trait narcissism • scan on weekday or weekend • friends on facebook • First level: Loved one > Other • Regression with… • relationship closeness • relationship satisfaction • age

23. Example • Costly exclusion predicts susceptibility to peer influence Falk et al., 2013, JAH

24. Correlations and Outliers Same data, with one outlier Null-hypothesis data, N = 50 Courtesy of Tor Wager

25. Robust Regression • Outliers can be problematic, especially for correlations • Robust regression reduces the impact of outliers • 1) Weight data by inverse of leverage • 2) Fit weighted least squares model • 3) Scale and weight residuals • 4) Re-fit model • 5) Iterate steps 2-4 until convergence • 6) Adjust variances or degrees of freedom for p-values • Can be applied to simple group results or correlations • Whole brain: http://wagerlab.colorado.edu/ • ROI: whatever software you prefer (more later)

26. Null-hypothesis data, N = 50 Same data, with one outlier Robust IRLS solution Courtesy of Tor Wager

27. Case Study: Visual Activation Visual responses Courtesy of Tor Wager

28. (De-noise) Realign Slice Timing Correct Smooth Predictors AcquireFunctionals Y X y = Xβ + ε Template 1st level (Subject) GLM Determine Scanning Parameters β βhow - βwhy Co-Register Normalize Acquire Structurals (T1) Contrast All subjects 2nd level (Group) GLM Threshold

29. (De-noise) Realign Slice Timing Correct Smooth Predictors AcquireFunctionals Y X y = Xβ + ε Template 1st level (Subject) GLM Determine Scanning Parameters β βhow - βwhy Co-Register Normalize Acquire Structurals (T1) Contrast All subjects 2nd level (Group) GLM Threshold

30. Up Next… • Hypothesis Testing • Levels of Inference • Multiple Comparisons • Family-wise Error Correction • False-Discovery Rate Correction • Non-parametric Correction

31. Hypothesis Testing • Null Hypothesis H0 • No effect • T-test: No difference from zero • F-test: No variance explained • α level • Set to an acceptable false positive rate • Level α = P( T > μα | H0) • Threshold μα controls false positive rate at level α • P-value • Test statistics are compared with appropriate distributions • Changes as a function of degrees of freedom • T-distribution: bell-shaped • F-distribution: skewed • Assessment of probability of test statistic assuming H0 • P(Data | Null) • But not P(Null | Data)!

32. Information for Making Inferences on Activation • Where? Signal location • Local maximum – no inference in SPM • Could extract peak coordinates and test (e.g., Woods lab, Ploghaus, 1999) • How strong? Signal magnitude • Local contrast intensity – Main thing tested in SPM • How large? Spatial extent • Cluster volume – Can get p-values from SPM • Sensitive to blob-defining-threshold • When? Signal timing • No inference in SPM; but see Aguirre 1998; Bellgowan 2003; Miezin et al. 2000, Lindquist & Wager, 2007

33. Unit of Analysis • Fundamental unit of analysis is voxel • GLM is run voxel-by-voxel • Statistical parametric maps (SPM’s) are calculated voxel-by-voxel • Unit of interest may instead by a “region” • Functional unit • Pool data across voxels • May also be broadly interested in the brain as a whole • Considering the brain as a whole, do these 2 conditions differ?

34. Levels of Inference • Inferences can be made at any “level” depending upon your unit of interest • Voxel-level • This/these particular voxels are significant • Most spatially specific, least sensitive • Cluster-level • These contiguous voxels together are significant • Less spatially specific, more sensitive • Set-level • The brain shows an effect • No spatial specificity, but can be most sensitive SPM’s results table shows p-values for voxel-level, cluster-level, and set-level tests.

35. Voxel-Level Inference • Retain voxels above α-level threshold uα • Gives best spatial specificity • The null hyp. at a single voxel can be rejected uα space Significant Voxels No significant Voxels Courtesy of Tor Wager

36. Cluster-Level Inference • Two step-process • Define clusters by arbitrary threshold uclus • Retain clusters larger than α-level threshold kα uclus space Cluster not significant Cluster significant kα kα Courtesy of Tor Wager

37. Cluster-Level Inference • Typically better sensitivity • Worse spatial specificity • The null hyp. of entire cluster is rejected • Only means that one or more voxels in cluster active uclus space Cluster not significant Cluster significant kα kα Courtesy of Tor Wager

38. Multiple Comparisons Problem • Often over 100,000 voxels in the brain • Voxel-level tests are repeated over 100,000 times • If α = 0.05 (i.e. p < 0.05), over 5,000 false positive voxels! • Need to control false positive rate at α across all tests • Otherwise, difficult to know if result is believable

39. Multiple Comparisons • Perform statistical tests at every voxel tens and tends of thousands • Quite likely that some would pass threshold by chance even if there was absolutely no effect • Need to correct for multiple comparisons.

40. Some Approaches • Bonferroni correction: Insist on p<.05/#voxels • Severely reduces sensitivity, but works with small ROIs • Gaussian random field theory: Suppose there is no effect but data is spatially smooth. What’s the chance of seeing a blob of X contiguous voxels all of which are above a threshold V? • Default approach to controlling familywise error (FWE) in SPM • False Discovery Rate (FDR): Set threshold so that less than 5% of the voxels above threshold would be false positives under null hypothesis

41. Family-Wise Error (FEW) • FWE-rate is the probability of finding one or more false positives among all hypothesis tests • If FWEα = 0.05, probability of finding one or more false positives is 5% • Based on maximum distribution • If no true positives are present, most significant voxel will exceed the threshold 5% of the time • Several approaches to control FWE

42. Bonferroni Correction • Simplest method for controlling FWE • αcorrected = α/V • α is the desired alpha level • αcorrected is the alpha level corrected for FWE • V is the number of voxels/tests • 0.05/100,000 = 0.0000005 • E.g. t(20) = 6.93 When examining results in SPM, you can find the # of voxels in the statistics table (bottom of the table under Volume). Divide α by the # of voxels to determine a Bonferroni corrected threshold.

43. BonferroniCorrection: Limitations • Correction assumes that each test is independent • Data are actually spatially smooth, so not independent! • Correction tends to be overly conservative • False positives appropriately controlled • But threshold is too high to detect many true positives

44. Gaussian Random Fields • SPM’s default method of FWE correction takes into account smoothness of data • Intuition • Smooth data  lower the resolution of the search space  fewer comparisons  less stringent correction • Assumes that an image of residuals can be descripbed by Gaussian noise convolved with a 3D kernel • Forms a Gaussian Random Field • FWHM of the kernel describes the smoothness of the data

45. 1 2 3 4 5 6 7 8 9 10 1 2 3 4 Tech Note – Estimating Smoothness: RESELS • RESELS = RESolution Elements • 1 RESEL = FWHMx x FWHMy x FWHMz voxels RESELS Note, when examining results in SPM, you can find the # of resels and FWHM in the statistics table (bottom of the table under Volume)

46. Threshold needed to correct • Increases with greater search volume • Need more stringent correction • Decreases with greater smoothness, RESEL • Greater smoothness leads to less stringent correction

47. 5mm FWHM 10mm FWHM 15mm FWHM Gaussian Random Fields: Clusters 1) Threshold at voxel-level 2) Estimate chance of clusters of size ≥ k, taking into account Mean expected cluster size search volume smoothness Threshold -> puncorrected of cluster of size ≥k 3) Apply previously described correction pcorrected * z2 Courtesy of Tor Wager

48. Gaussian Random Fields: Limitations • Requires sufficient smoothness of data • FWHM 3-4x voxel size • Performs poorly with low df • Better with df > 20 and sufficient smoothness • Tends to be conservative, especially with rough data (FWHM < 6) • Based on approximations • Approximations can be thrown off by “roughness spikes” • Approximations will vary on a contrast by contrast basis • Different contrasts in same data will have different thresholds • Typically regarded as better at individual level where df are high Select “FWE” in SPM results to threshold using Gaussian Random Field Theory. Expect a conservative threshold.

49. False-Discovery Rate • Correction of FWE ensures that false positives will be controlled per family of tests • αFWE-corrected = 0.05, 5% of contrasts (across all voxels) will have a single false positive • False-Discovery Rate (FDR) controls the number of false positives within a family of tests • αFDR-corrected = 0.05, 5% of reportedly active voxels in a contrast will be false positives • Upside: will find more true signal • Downside: will have a few false positives

50. False-Discovery Rate: Method • Establish a rate, q, of acceptable proportion of false-positives (e.g. 0.05) • Sort observed p-values from smallest to largest • Find max(i) such that Pi < i*(q/V) • V is the # of voxels • In other words • Smallest p-value must pass Bonferroni, second smallest Bonferroni*2, third smallest Bonferroni*3, etc • i.e. i = 1: Bonferroni*1, i = 2: Bonferroni*2, i = 3: Bonferroni*3, etc. • Highest such i gives threshold • If no p-value passes, threshold cannot be determined (SPM will say the threshold is t = infinity) 1 p(i) p-value (i/V)q 0 0 1 i/V