Distributed Computational Architectures for Integrated Time-Dynamic Neuroimaging

1. Distributed Computational Architectures forIntegrated Time-Dynamic Neuroimaging Dr. Allen D. Malony malony@cs.uoregon.edu Computer & Information Science Department Computational Science Institute CIBER University of Oregon

2. Outline • Computational science and cognitive neuroscience • Brain dynamics analysis problem • integrated electromagnetic analysis system • Motivating case studies • observations • Computational architectures • models and technology • key ideas • Neuroinformatics GRID • Final Thoughts HBP Neuroinformatics Conference

3. Computational Science & Cognitive Neuroscience • Computational methods applied to scientific research • high-performance simulation of complex phenomena • large-scale data analysis and visualization • Understand functional activity of the human cortex • multiple cognitive domains • multiple experimental paradigms and methods • Need for coupled/integrated modeling and analysis • electrical and magnetic, cortical and theoretical • Need for robust tools: computational, informatic • Problem solving environment for brain analysis HBP Neuroinformatics Conference

4. Brain Dynamics Analysis Problem • Identifyfunctional components in cognitive contexts • Interpret with respect to cognitive theoretical models • Requirements: spatial (structure), temporal (activity) • Imaging techniques for analyzing brain dynamics • blood flow neuroimaging (PET, fMRI) • good spatial resolution  functional brain mapping • temporal limitations to tracking of dynamic activities • electromagnetic measures (EEG/ERP, MEG) • msec temporal resolution to distinguish components • spatial resolution sub-optimal (source localization) • potential to map electrical activity to cortex surface HBP Neuroinformatics Conference

5. Electromagnetic Analysis Methodology • Multi-trial analysis • signal analysis and response analysis • averaging across subjects and trials • distortion (smearing) of estimated source response • noise artifacts, signal variation (individuals, trials) • improvements: artifact removal, selective averaging • create component response models • factor analysis: PCA, ICA • error in source factors: variability, statistics • Multi-subject and single-subject analysis • quantify differences of individual from population HBP Neuroinformatics Conference

6. Single-Trial Analysis Capability • Improve fidelity of single-subject response model • higher information content than multi-trial/subject • reduce analysis error from trial/subject variability • knowledge of subject population, stimulus deviations • Diagnosis (identification) of cognitive state • known stimulus • blind stimulus • match response to known component response model • Problems • greater noise • greater complexity HBP Neuroinformatics Conference

7. Single-Trial Analysis Methodology • Integrate methods for analyzing brain dynamics • Improve resolution and robustness of techniques • increase measurement density (128 to 256 channels) • Coupled modeling: constraints and cross-validation • component response model  cortical activity model • tuned models for single individual • Build models in experimental paradigm context • Match single-trial measurements to models • known stimulus  multiple trial models • blind stimulus  multiple stimulus/trial models • Training and learning HBP Neuroinformatics Conference

8. Integrated Electromagnetic Brain Analysis Cortical Activity Knowledge Base Head Analysis Source Analysis Structural / Functional MRI spatial pattern recognition temporal dynamics Cortical Activity Model Experiment subject Constraint Analysis Single-trial Analysis EEG MEG Component Response Model neural constraints Dense Array EEG / MEG temporal pattern recognition Signal Analysis Response Analysis Component Response Knowledge Base HBP Neuroinformatics Conference

9. Case Study: Readiness Potential • Self-paced button pressing task • slow negative shifts in potential contralateral to hand • Single subject examination • multi-trial (150 trials) averaged ERP analysis • Dense-array scalp electrical measurement • 129 electrode array (EGI Geodesic Sensor Net) • Modeling of brain electrical activity • MRI and CT data analysis with tissue segmentation • realistic boundary element meshes (2K ’s for brain) • source localization • Can ERP analysis accurately localize cortical activity? HBP Neuroinformatics Conference

10. Experimental Methodology CT and MRI EEG segmentedtissues NetStation processed EEG BrainVoyager mesh generation, source localization EMSE Interpolator 3D HBP Neuroinformatics Conference

11. Electrical Activity of Scalp and Brain Lateralize Readiness Potential (LRP) • Expected brain activity • Correlated with fMRI experimental studies • Topographic and cortex mapped spatial analysis -404 ms -56 ms 0 ms 160 ms HBP Neuroinformatics Conference

12. Case Study: Self-Monitored Motivated Action • Learning task with feedback (Gehring et al. (1993)) • left- or right-hand button press response • "incorrect" feedback on error • "OK" or “late” feedback if correct • timed expectancy and motivated response • Error-Related Negativity (ERN) • large medial negative response on error • self-monitoring when motivated action goes wrong • What is the nature and complexity of the ERN with respect to dynamic components of brain activity? HBP Neuroinformatics Conference

13. Visualize the dynamic operations of brain Example: fMRI blood flow response to reading a word Dense-array EEG / MEG frontal lobe activity (ERN) significant changes in milliseconds frontal oscillations and separate time courses BrainVoyager Cognitive Experiments and Brain Dynamics HBP Neuroinformatics Conference

14. ERN Analysis using ICA (Makeig, Salk Institute) • Average analysis smears temporal/spatial dynamics • Single-trial analysis may expose greater detail • Independent Components Analysis (ICA) • find independent EEG component contributors • temporal and spatial • components accounting for artifacts • components accounting for functional sources (ERN) • analysis over single trials • Two components account for averaged ERN • response-locked ERN difference wave dominated • show temporal and functional independence HBP Neuroinformatics Conference

15. ERP and Component Envelopes (Left/Correct) Component #2 Component #7 • Complementary • behavior • Both active at strongest ERN channels HBP Neuroinformatics Conference

16. ERPs averaged across response hand neither C2 nor C7 explain the waveforms component sum does explain the waveforms and shows ERN response HBP Neuroinformatics Conference

17. Topographic Imaging and Dipole Modeling Component #2 Component #2 Averaged ERN Brain Electrical Source Analysis (BESA) HBP Neuroinformatics Conference

18. ICA Component #2 Dynamics • Stimulus locked HBP Neuroinformatics Conference

19. ICA Component #7 Dynamics • Phase reset byresponse, largestafter incorrect HBP Neuroinformatics Conference

20. Case Study Observations • Diverse set of tools • function and implementation • separate and not integrated • incompatibilities and limitations for interoperation • Complex analysis processes • scientific discovery through integrated techniques • heterogeneous, flexible, extensible capabilities • increasingly high computational demands • high-level process methodology • Multiple, interdisciplinary scientific domains HBP Neuroinformatics Conference

21. High-Performance Computational Environments • Integrated database, analysis, and visualization • Distributed tool infrastructure • diverse tools across multiple platforms • interoperation requirements • user interaction requirements • support portability, flexibility, extensibility • Scalable, high-performance parallel computing • increase data resolution • minimize solution time • High-level access to tools • web-based access HBP Neuroinformatics Conference

22. Domain-specific, problem-specific environments (PSE) TIERRA Scientific “workbench” SCIRun Programming environments numerical frameworks POOMA application coupling PVM / MPI CUMULVS PAWS SILOON / PDT Metacomputing / GRID Legion Globus Heterogeneous distributed computing / coupling NetSolve INTERLACE HARNESS Web-based environments ViNE PUNCH VNC Computational Systems: Models and Technology HBP Neuroinformatics Conference

23. SCIRun (Johnson, University of Utah) • Scientific programming environment • large-scale simulations • “computational workbench” • visual programming interface • dataflow model of computing • modules: operation or algorithm with I/O ports • network: set of modules and their interconnections • widgets: 3D user interaction • data types: Mesh, Surface, Matrix, Field, Geometry • extensible module library • computational steering HBP Neuroinformatics Conference

24. Visual programming lets users select, arrange, and connect modules into a desired network Interactive steering of design, computation, and visualization allows more rapid convergence SCIRun User Interface HBP Neuroinformatics Conference

25. ICA for EEG Source Localization with SCIRun • PCA decomposition forEEG signal/noise subspaces • ICA activity map separationon signal subspace • Solution to a single dipolesource forward problem • underlying model is shownin the MRI planes • dipole source is indicated by red and blue spheres • electric field visualized by cropped scalp potential map and wire-frame equipotential isosurface HBP Neuroinformatics Conference

26. PDT (Malony, University of Oregon) • Program Database Toolkit • Program analysis • multi-language(Fortran, C,C++, Java) • commercial-grade parsers • IL to programdatabase (PDB) • API for PDBaccess / query • Tools: instrumentation, code wrapping, documentation HBP Neuroinformatics Conference

27. SILOON (Advanced Computing Lab, LANL; UO) • Scripting Interface Language for OONumerics • Toolkit and run-time support for building easy-to-use external interfaces to existing numerical codes • Scripting language to “glue” components together HBP Neuroinformatics Conference

28. Metasystems and Metacomputing • Many resources accessible on the internet • computers, data, devices, people • Extend single system model to internet domain • wide-area (department, campus, region, country) • scalable, transparent access to resources • hides network complexity (“as if on your machine”) • Extend computing model to internet domain • shared persistent space of objects (data, execution) • heterogeneous distributed and parallel processing • meta-applications (multi-component, hierarchical) • Deal with complex environment / primitive tools HBP Neuroinformatics Conference

29. Characteristics of Meta-applications • Multiple components • programs, databases, instruments, devices • Different authors • Different languages • Different applications • legacy, COTS, ... • coupled modeling • Parallelism • internal: task/data • external: components HBP Neuroinformatics Conference

30. “The GRID” • New applications based on high-speed coupling of people, computers, databases, instruments, ... • computer-enhanced instruments • collaborative engineering • browsing of remote datasets • use of remote software • data-intensive computing • very large-scale simulation • large-scale parameter studies HBP Neuroinformatics Conference

31. GRID Architectural Picture HBP Neuroinformatics Conference

32. GRID Technical Challenges • Complex application structures, combining aspects of parallel, multimedia, distributed, collaborative computing • Dynamic varying resource characteristics, in time and space • Need for high and guaranteed “end-to-end” performance, despite heterogeneity and lack of global control • Inter-domain issues of security, policy, payment HBP Neuroinformatics Conference

33. NetSolve (Dongarra, University of Tennessee) • Client-server systemto access distributedcomputational / DBHW/SW resources • Distributed computing:resources, processes,data, users • Load-balancing policy for efficiency / performance • Integration with arbitrary software components • C, Fortran, Java, MatLab, Mathematica, Excel • BLAS, (Sca)LAPACK, MINPACK, FFTPACK HBP Neuroinformatics Conference

34. NetSolve – 1999 R&D Winner HBP Neuroinformatics Conference

35. NetSolve Usage • “Blue collar” GRID-based computing • users can set things up (without “su” privileges) • no deep network programming knowledge required • Scenarios • clients, servers, and agents anywhere on Internet • clients, servers, and agents on an Intranet • clients, servers, and agent on the same machine • Focus on MATLAB users • OO-style language (objects are matrices) • one of most popular desktop systems for numerical computing (> 400K users) HBP Neuroinformatics Conference

36. NetSolve – The Client • NetSolve API hides complexity of numerical software • Computation is location transparent • Provides access to virtual libraries: • Component GRID-based framework • Central management of library resources • User not concerned with most up-to-date versions • Automatic tie to Netlib repository • Synchronous or asynchronous calls • User-level parallelism HBP Neuroinformatics Conference

37. NetSolve – The Agent and Server • Agent • gateway to computational services • performs load balancing and resource management • Server • various software installed on various hardware • configurable and extendable • framework to easily add software • many numerical libraries being integrated • supports parallel computing HBP Neuroinformatics Conference

38. Using MCell with NetSolve HBP Neuroinformatics Conference

39. MCell (Bartol, Salk Institute; Salpeter, Cornell) • Monte Carlo simulator of cellular microphysiology • Study how neurotransmitters diffuse and activate receptors in synapses between different cells • NetSolve distributesprocessing workloadand allows access tocomputational resources • Simultaneous evaluationof large number ofdifferent parametercombinations HBP Neuroinformatics Conference

40. ViNE (Malony, University of Oregon) • Virtual Notebook Environment • High-level, sharednotebooks, data, andtools in distributed,heterogenous system • Architecture • leaves: notebookfunctions and data • stems: notebookcommunication • Web-based access HBP Neuroinformatics Conference

41. ViNE Experiment Builder • List of available, named data, tools, and experiments • Visual dataflow model of experiment process • Wrapped tools and databases wrapped MATLAB “tool” HBP Neuroinformatics Conference

42. Brain Electrophysiology Lab Notebook • Dense array EEG datasets • Commercial of the shelf statistical and numerical packages • Multiple machines types • Notebook content automatically generated from experiment results HBP Neuroinformatics Conference

43. PUNCH • Purdue University Network-Computing Hubs • Educational and research computing “portals” • across the Purdue “enterprise” • with affiliated institutions • Resource sharing by Purdue users • computers, software, laboratory equipment • educational materials • Distance education • allows sharing of courses and instructors • Collaborative research HBP Neuroinformatics Conference

44. PUNCH – User’s and Developer’s View • Set of network-based laboratories that provide software tools for various fields • Specialized WWW-server interfaces WWW-browsers • access software and download data • run tools and view results • Tool specification • Virtual laboratorydevelopmentenvironment HBP Neuroinformatics Conference

45. PUNCH Web Page Hubs HBP Neuroinformatics Conference

46. PUNCH Software Components HBP Neuroinformatics Conference

47. PUNCH Across the Internet HBP Neuroinformatics Conference

48. PUNCH Tool Display Support via VNC X Windows display MATLABcommand window MATLABinteractive window MATLABgraphics window HBP Neuroinformatics Conference

49. Virtual Network Computing (VNC) • Remote access to graphical user interfaces • VNC “thin client” protocol • based on concept of remote frame buffer • server updates a frame buffer displayed on a viewer • OS independent: Unix, Linux, MacOS, Windows, PDA • Communications independent – reliable transport HBP Neuroinformatics Conference

50. VNC Clients X (Windows) Mac-IE (Windows) Mac-IE (X) PDA (X) Mac (X) X-NS (Windows) HBP Neuroinformatics Conference