1 / 50

Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson

Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston). A Generative Model of M/EEG Group inversion (optimising priors across subjects) Multimodal integration: 3.1 Symmetric integration (fusion) of MEG + EEG

kennethcash
Télécharger la présentation

Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)

  2. A Generative Model of M/EEG • Group inversion (optimising priors across subjects) • Multimodal integration: • 3.1 Symmetric integration (fusion) of MEG + EEG • 3.2 Asymmetric integration of M/EEG + fMRI • 3.3 Full fusion of M/EEG + fMRI? Overview

  3. (Linear) Forward Model for MEG/EEG (for one timepoint): Y = Data n sensors J = Sources p>>n sources L = Leadfields n sensors x p sources E= Error n sensors 1. A PEB Framework for MEG/EEG(Bayesian inference) (Gaussian) Likelihood: C(e)= n x n Sensor (error) covariance Prior: C(j)= p x p Source (prior) covariance Posterior: Phillips et al (2005), Neuroimage

  4. # sensors # sensors # sensors # sensors # sources # sources # sources # sources 1. A PEB Framework for MEG/EEG(Covariance Components/Priors) Specifying (co)variance components (priors/regularisation): C = Sensor/Source covariance Q= Covariance components λ= Hyper-parameters 1. Sensor components, (error): “IID” (white noise): Empty-room: 2. Source components, (priors/regularisation): Multiple Sparse Priors (MSP): “IID” (min norm): Friston et al (2008) Neuroimage

  5. Prestim Baseline Anti-Averaging Sensor priors Depth-Weighting Smoothness Source priors projected to sensors 1. A PEB Framework for MEG/EEG(Hyperpriors) When multiple Q’s are correlated, estimation of hyperparameters λ can be difficult (eg local maxima), and they can become negative (improper for covariances) To overcome this, one can: 1) impose positivity on hyperparameters: 2) impose weak, shrinkage hyperpriors: uninformative priors are then “turned-off” (cf. “Automatic Relevance Detection”) Henson et al (2007) Neuroimage

  6. 1. A PEB Framework for MEG/EEG(Generative Model) Source and sensor space Fixed Variable Data Friston et al (2008) Neuroimage

  7. 1. A PEB Framework for MEG/EEG(Estimation/Inversion) 1. Obtain Restricted Maximum Likelihood (ReML) estimates of the hyperparameters (λ) by maximising the variational “free energy” (F): 2. Obtain Maximum A Posteriori (MAP) estimates of parameters (sources, J): cf. Tikhonov 3. Maximal F approximates Bayesian (log) “model evidence” for a model, m: Accuracy Complexity (…where and are the posterior mean and covariance of hyperparameters) Friston et al (2002) Neuroimage

  8. 1. A PEB Framework for MEG/EEG Summary: • Automatically “regularises” in principled fashion… • …allows for multiple constraints (priors)… • …to the extent that multiple (100’s) of sparse priors possible… • …(or multiple error components or multiple fMRI priors)… • …furnishes estimates of model evidence, so can compare constraints

  9. A Generative Model of M/EEG • Group inversion (optimising priors across subjects) • Multimodal integration: • 3.1 Symmetric integration (fusion) of MEG + EEG • 3.2 Asymmetric integration of MEG + fMRI • 3.3 Full fusion of MEG/EEG + fMRI? Overview

  10. # sensors # sensors # sensors # sensors # sources # sources # sources # sources Specifying (co)variance components (priors/regularisation): C = Sensor/Source covariance Q = Covariance components λ= Hyper-parameters 2. Group Inversion(Covariance Components) 1. Sensor components, (error): “IID” (white noise): Empty-room: 2. Source components, (priors/regularisation): Multiple Sparse Priors (MSP): “IID” (min norm): Friston et al (2008) Neuroimage

  11. # sensors # sensors # sensors # sensors # sources # sources Specifying (co)variance components (priors/regularisation): C = Sensor/Source covariance Q = Covariance components λ= Hyper-parameters 2. Group Inversion(Covariance Components) 1. Sensor components, (error): “IID” (white noise): Empty-room: 2. Optimise Multiple Sparse Priors by pooling across participants Litvak & Friston (2008) Neuroimage

  12. 2. Group Inversion (one subject)(Generative Model) Source and sensor space Fixed Variable Data Litvak & Friston (2008) Neuroimage

  13. 2. Group Inversion (multiple subjects)(Generative Model) Source and sensor space Fixed Variable Data Litvak & Friston (2008) Neuroimage

  14. 2. Group Inversion (multiple subjects)(Re-referencing leadfield matrices) Concatenate data across subjects …having projected to an “average” leadfield matrix Subject-specific sensor-level priors: Common source-level priors: Litvak & Friston (2008) Neuroimage

  15. 2. Group Inversion(Generative Model) MSP MMN MSP (Group) Litvak & Friston (2008) Neuroimage

  16. 2. Group Inversion(Generative Model) MMN + 3 fMRI priors (Group) MMN + 3 fMRI priors Henson et al (submitted) Frontiers

  17. A Generative Model of M/EEG • Group inversion (optimising priors across subjects) • Multimodal integration: • 3.1 Symmetric integration (fusion) of MEG + EEG • 3.2 Asymmetric integration of MEG + fMRI • 3.3 Full fusion of MEG/EEG + fMRI? Overview

  18. “Neural” Activity Causes (hidden): 3. Types of Multimodal Integration (inversion) Balloon Model Head Model Head Model Generative (Forward) Models: ? Data: fMRI MEG EEG ? (future) Henson (2010) Biomag

  19. “Neural” Activity Causes (hidden): 3. Types of Multimodal Integration Symmetric Integration (Fusion) Balloon Model Head Model Head Model Generative (Forward) Models: ? Data: fMRI MEG EEG ? (future) Asymmetric Integration Daunizeau et al (2007), Neuroimage

  20. # sensors # sensors # sensors # sensors # sources # sources # sources # sources 3.1 Fusion of MEG+EEG(Sensor Components) Specifying (co)variance components (priors/regularisation): C = Sensor/Source covariance Q= Covariance components λ= Hyper-parameters 1. Sensor components, (error): “IID” (white noise): Empty-room: 2. Source components, (priors/regularisation): Multiple Sparse Priors (MSP): “IID” (min norm): Friston et al (2008) Neuroimage

  21. # sensors # sources # sources # sensors # sensors # sensors # sources # sources 3.1 Fusion of MEG+EEG(Sensor Components) Specifying (co)variance components (priors/regularisation): Ci(e)= Sensor error covariance for ith modality Qij= jth component for ith modality λij= Hyper-parameters 1. Sensor components, (error): E.g, white noise for 2 modalities: 2. Source components, (priors/regularisation): Multiple Sparse Priors (MSP): “IID” (min norm): Henson et al (2009) Neuroimage

  22. 3.1 Basic Model for MEG or EEG(Generative Model) Source and sensor space Fixed Variable Data Henson et al (2009) Neuroimage

  23. 3.1 Fusion of MEG+EEG(Generative Model) Source and sensor space Fixed Variable Data Henson et al (2009) Neuroimage

  24. Stack data and leadfields for d modalities: 3.1 Fusion of MEG+EEG(Theory) (note: common sources and source priors, but separate error components) • Where data / leadfields scaled to have same average / predicted variance: mi = Number of spatial modes (e.g, channels) Henson et al (2009) Neuroimage

  25. ERs from 12 subjects for 3 simultaneously-acquired Neuromag sensor-types: Magnetometers (MEG, 102) (Planar) Gradiometers (MEG, 204) Electrodes (EEG, 70) 3.1 Fusion of MEG+EEG(Application) fT μV RMS fT/m Faces Scrambled ms ms ms Faces - Scrambled 150-190ms Henson et al (2009) Neuroimage

  26. +19 -48 -6 +31 -51 -15 MEG mags MEG grads Faces Scrambled 3.1 Fusion of MEG+EEG Faces – Scrambled, 150-190ms +43 -67 -11 +44 -64 -4 FUSED EEG IID noise for each modality; common MSP for sources Henson et al (2009) Neuroimage (fixed number of spatial+temporal modes)

  27. Fusing magnetometers, gradiometers and EEG increased the conditional precision of the source estimates relative to inverting any one modality alone • (when equating number of spatial+temporal modes) • The maximal sources recovered from fusion were a plausible combination of the ventral temporal sources recovered by MEG and the lateral temporal sources recovered by EEG • (Simulations show the relative scaling of mags and grads agrees with empty-room data) 3.1 Fusion of MEG+EEG(Conclusions) Henson et al (2009) Neuroimage

  28. # sensors # sensors # sensors # sensors # sources # sources # sources # sources Specifying (co)variance components (priors/regularisation): C = Sensor/Source covariance Q = Covariance components λ= Hyper-parameters 3.2 Asymmetric Integration of M/EEG+fMRI 1. Sensor components, (error): “IID” (white noise): Empty-room: 2. Source components, (priors/regularisation): Multiple Sparse Priors (MSP): “IID” (min norm): Friston et al (2008) Neuroimage

  29. # sensors # sensors # sensors # sensors # sources # sources 3.2 Asymmetric Integration of M/EEG+fMRI Specifying (co)variance components (priors/regularisation): C = Sensor/Source covariance Q = Covariance components λ= Hyper-parameters 1. Sensor components, (error): “IID” (white noise): Empty-room: 2. Each suprathreshold fMRI cluster becomes a separate prior # sources fMRI Priors: “IID” (min norm): # sources Henson et al (2010) Hum. Brain Map.

  30. 3.2 Basic model for MEG or EEG(Generative Model) Source and sensor space Fixed Variable Data Friston et al (2008) Neuroimage

  31. 3.2 Asymmetric Integration of M/EEG+fMRI(Generative Model) Source and sensor space Fixed Variable Data Henson et al (2010) Hum. Brain Map.

  32. 3.2 Integration of M/EEG+fMRI (Priors) T1-weighted MRI {T,F,Z}-SPM Anatomical data Functional data … 1. Thresholding and connected component labelling Cortical surfaceextraction Gray matter segmentation … 2. Projection onto the cortical surface using the Voronoï diagram … 3D geodesicVoronoï diagram 3. Prior covariance components Henson et al (2010) Hum. Brain Map.

  33. 1 2 3.2 Integration of M/EEG+fMRI (Application) SPM{F} for faces versus scrambled faces, 15 voxels, p<.05 FWE 3 4 5 5 clusters from SPM of fMRI data from separate group of (18) subjects in MNI space Henson et al (2010) Hum. Brain Map.

  34. Magnetometers (MEG) * * 3.2 Fusion of MEG+fMRI (Application) * * Gradiometers (MEG) * * Negative Free Energy (a.u.) (model evidence) * * Electrodes (EEG) * * * None Global Local (Valid) Local (Invalid) Valid+Invalid (binarised, variance priors) Henson et al (2010) Hum. Brain Map.

  35. Magnetometers (MEG) * * 3.2 Fusion of MEG+fMRI (Application) * * Gradiometers (MEG) * * Negative Free Energy (a.u.) (model evidence) * * Electrodes (EEG) * * * None Global Local (Valid) Local (Invalid) Valid+Invalid (binarised, variance priors) Henson et al (2010) Hum. Brain Map.

  36. Magnetometers (MEG) * * 3.2 Fusion of MEG+fMRI (Application) * * Gradiometers (MEG) * * Negative Free Energy (a.u.) (model evidence) * * Electrodes (EEG) * * * None Global Local (Valid) Local (Invalid) Valid+Invalid (binarised, variance priors) Henson et al (2010) Hum. Brain Map.

  37. Magnetometers (MEG) * * 3.2 Fusion of MEG+fMRI (Application) * * Gradiometers (MEG) * * Negative Free Energy (a.u.) (model evidence) * * Electrodes (EEG) * * * None Global Local (Valid) Local (Invalid) Valid+Invalid (binarised, variance priors) Henson et al (2010) Hum. Brain Map.

  38. Magnetometers (MEG) * * 3.2 Fusion of MEG+fMRI (Application) * * Gradiometers (MEG) * * Negative Free Energy (a.u.) (model evidence) * * Electrodes (EEG) * * * None Global Local (Valid) Local (Invalid) Valid+Invalid (binarised, variance priors) Henson et al (2010) Hum. Brain Map.

  39. IID sources and IID noise (L2 MNM) Magnetometers (MEG) 3.2 Fusion of MEG+fMRI (Application) Gradiometers (MEG) Electrodes (EEG) None Global Local (Valid) Local (Invalid) Henson et al (2010) Hum. Brain Map.

  40. IID sources and IID noise (L2 MNM) Magnetometers (MEG) 3.2 Fusion of MEG+fMRI (Application) Gradiometers (MEG) Electrodes (EEG) None Global Local (Valid) Local (Invalid) Henson et al (2010) Hum. Brain Map.

  41. IID sources and IID noise (L2 MNM) Magnetometers (MEG) 3.2 Fusion of MEG+fMRI (Application) Gradiometers (MEG) Electrodes (EEG) None Global Local (Valid) Local (Invalid) fMRI priors counteract superficial bias of L2-norm Henson et al (2010) Hum. Brain Map.

  42. IID sources and IID noise (L2 MNM) Magnetometers (MEG) 3.2 Fusion of MEG+fMRI (Application) Gradiometers (MEG) Electrodes (EEG) None Global Local (Valid) Local (Invalid) fMRI priors counteract superficial bias of L2-norm Henson et al (2010) Hum. Brain Map.

  43. Right Posterior Fusiform (rPF) Right Medial Fusiform (rMF) Right Lateral Fusiform (rLF) +41 -43 -24 +32 -45 -12 +26 -76 -11 Differential Response (Faces vs Scrambled) 3.2 Fusion of MEG+fMRI (Application) Left occipital pole (lOP) R -27 -93 0 Differential Response (Faces vs Scrambled) Gradiometers (MEG) (5 Local Valid Priors) Left Lateral Fusiform (lLF) -43 -47 -21 L Differential Response (Faces vs Scrambled) NB: Priors affect variance, not precise timecourse… Henson et al (2010) Hum. Brain Map. Prior 4. Prior 5.

  44. 3.2 Fusion of MEG+fMRI (Conclusions) • Adding a single, global fMRI prior increases model evidence • Adding multiple valid priors increases model evidence further • Helpful if some fMRI regions produce no MEG/EEG signal (or arise from neural activity at different times) • Adding invalid priors does not necessarily increase model evidence, particularly in conjunction with valid priors • Can counteract superficial bias of, e.g, minimum-norm • Affects variance but not not precise timecourse Henson et al (2010) Hum. Brain Map.

  45. “Neural” Activity Causes (hidden): 3.3 Fusion of fMRI and MEG/EEG? Fusion of fMRI + MEG/EEG? Balloon Model Head Model Head Model ? Data: fMRI MEG EEG ? (future) Henson (2010) Biomag

  46. 3.3 Fusion of fMRI and MEG/EEG? Source and sensor space Fixed Variable Data Henson (submitted) Frontiers

  47. 3.3 Fusion of fMRI and MEG/EEG? Source and sensor space Fixed Variable Data Henson (submitted) Frontiers

  48. Overall Conclusions • The PEB (in SPM8) framework is advantageous • Group optimisation of MSPs can be advantageous • Full fusion of MEG and EEG is advantageous • Using fMRI as (spatial) priors on M/EEG is advantageous • Unclear that fusion of fMRI and M/EEG is advantageous

  49. The End

  50. Magnetometers (MEG) Gradiometers (MEG) Electrodes (EEG) ln(λ)+32 3.2. Fusion of MEG+fMRI Participant fMRI hyperparameters Local Valid ln(λ)+32 Participant Local Invalid Henson et al (2010) Prior 4. Prior 5.

More Related