Multimodal Brain Imaging

1. Multimodal Brain Imaging Will D. Penny FIL, London Guillaume Flandin, CEA, Paris Nelson Trujillo-Barreto, CNC, Havana

2. Neuronal Activity Experimental Manipulation Optical Imaging MEG,EEG PET fMRI FORWARD MODELS Single/multi-unit recordings Spatial convolution via Maxwell’s equations Sensorimotor Memory Language Emotion Social cognition Temporal convolution via Hemodynamic/Balloon models

3. Neuronal Activity Experimental Manipulation MEG,EEG fMRI INVERSION 1. Spatio-temporal deconvolution Spatial deconvolution via beamformers 2. Probabilistic treatment Temporal deconvolution via model fitting/inversion

4. Overview • Spatio-temporal deconvolution for M/EEG • Spatio-temporal deconvolution for fMRI • Towards models for multimodal imaging

5. Add temporal constraints in the form of a General Linear Model to describe the temporal evolution of the signal. Puts M/EEG analysis into same framework as PET/fMRI analysis. Work with Nelson. Described in chapter of new SPM book. Spatio-temporal deconvolution for M/EEG

7. Generative Model: Hyperpriors:

8. Repeat L • Update source estimates, q(j) • Update regression coefficients, q(w) • Update spatial precisions, q(a) • Update temporal precisions, q(l) • Update sensor precisions, q() KL F Until change in F is small Variational Bayes: Mean-Field Approximation

9. Mean-Field Approximation: Approximated posteriors:

11. Corr(R3,R4)=0.47

14. 700ms 500ms 2456ms 600ms Fa o + + + Time Sa + o Sb + + High Symmetry Low Symmetry Low Asymmetry High Asymmetry Phase 1 Ub Henson R. et al., Cerebral Cortex, 2005

15. A1 Faces minus Scrambled Faces B8 170ms post-stimulus

16. B8 A1 Faces Scrambled Faces

17. Daubechies Cubic Splines Wavelets

18. Daubechies-4 28 Basis Functions 30 Basis Functions

19. ERP Faces ERP Scrambled

20. t = 170 ms

21. Faces – Scrambled faces: Difference of absolute values t = 170 ms

22. Temporal evolution is described by GLM in the usual way. Add spatial constraints on regression coefficients in the form of a spatial basis set eg. spatial wavelets. Automatically select the appropriate basis subset using a mixture prior which switches off irrelevant bases. Embed this in a probabilistic model. Work with Guillaume. To appear in Neuroimage very soon. Spatio-temporal deconvolution for fMRI

23. Spatial Model eg. Wavelets

24. Mixture prior on wavelet coefficients • Wavelet switches: d=1 if coefficient is ON. Occurs with probability p • If switch is on, draw z from the fat Gaussian.

25. Probabilistic Generative Model Switch priors Wavelet switches Wavelet coefficients Spatial Model General Linear Model Temporal Model fMRI data

27. Compare to (i) GMRF prior used in M/EEG and (ii) no prior

28. Inversion using wavelet priors is faster than using standard EEG priors

29. Results on face fMRI data

30. Use simultaneous EEG- fMRI to identify relationship Between EEG and BOLD (MMN and Flicker paradigms) EEG is compromised -> artifact removal Testing the `heuristic’ Start work on specifying generative models Ongoing work with Felix Blankenburg and James Kilner Towards multimodal imaging

31. fMRI results

32. fMRI results

33. MRI Gradient artefact removal from EEG We have “synchronized sEEG-fMRI” – MR clock triggers both fMRI and EEG acquisition; after each trigger we get 1 slice of fMRI and 65ms worth of EEG. Synchronisation makes removal of GA artefact easier

34. Ballistocardiogram removal Could identify QRS complex from ECG to set up a ‘BCG window’ for subsequent processing

35. Ballistocardiogram removal

36. Ballistocardiogram removal

37. Testing the heuristic • The EEG-BOLD heuristic (Kilner, Mattout, Henson & Friston) contends that • increases in average EEG frequency predict BOLD activation. g(w) = spectral density

38. RMSF for Marta’s data at Cz

39. Log of Bayes factor for Heuristic versus Null

40. Log of Bayes factor for Heuristic versus Alpha

41. Tentative probabilistic generative model

42. THANK-YOU FOR YOUR ATTENTION !