1 / 51

A Novel Multi-GPU Neural Simulator

A Novel Multi-GPU Neural Simulator. Corey Thibeault , Roger Hoang Frederick C. Harris, Jr. Brain Computation Laboratory University of Nevada, Reno. Reno, Nevada. University of Nevada, Reno. Brain Computation Laboratory University of Nevada, Reno in funded collaboration with

rigg
Télécharger la présentation

A Novel Multi-GPU Neural Simulator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Novel Multi-GPU Neural Simulator Corey Thibeault, Roger Hoang Frederick C. Harris, Jr. Brain Computation Laboratory University of Nevada, Reno

  2. Reno, Nevada

  3. University of Nevada, Reno

  4. Brain Computation Laboratory University of Nevada, Reno in funded collaboration with U de Cergy-Pontoise and CNRS, Paris, France Brain Mind Institute (Blue Brain Project), EPFL, Lausanne, Switzerland • Laurence Jayet Fred Harris, Jr Sergiu Dascalu • Director • Mathias Quoy • René Doursat • Henry Markram • Jim King UNR Brain Computation Lab

  5. Scope of Work Neuroscience Modeling & Neural Engineering Robotic/Human Loops ONR N000140710018: “Large-Scale Biologically Realistic Models of Cortical & Subcortical Dynamics with Social Robotic Applications” DURIP N000140510525: “Robotic Platform for Security and Service Applications” DURIP N000140710704: “Parallel Robotic Brains” HRL0011-09-C-001: “Project Synapse”

  6. A Little Neuroscience • Neurons • Excitatory • Interneurons (inhibitory) • Neurons • Excitatory • Interneurons (inhibitory) • Neurons • Excitatory • Interneurons (inhibitory) • Columns • High connectivity within columns. • Less connectivity across columns

  7. A Little Neuroscience • Channels • Potassium Family • M, A, AHP Channels • Suppressing behavior on parent cell • Synapses • Analog converter of binary spike event. • Contextual filters.

  8. A Little Neuroscience • Brain Slice technology from EPFL

  9. A Little Neuroscience • Voltage Injection & Measurements

  10. A Little Neuroscience • But we need more than 1 • Currently a 12 clamp

  11. A Little Neuroscience • This technology is used to measure the neurons for reverse engineering • Connectivity • Voltage Response • Yun Wang, Henry Markram, Philip H. Goodman, Junying Ma, Patricia S. Goldma-Rakic “Novel Microcircuit Specializations in the Prefrontal Cortex” Nature Neuroscience 2006 Apr;9(4):534-42.

  12. Early Brain Simulation • Artificial Neural Networks (ANNs) • based on the nonlinear propagation of average activity (analogous to ring rates) • Some Biologically Realistic Simulators • Neuron & Genesis • Very accurate, • But small models (<10 cells)

  13. NCS History • Version 1:1999 • Matlab • 160-cell, 2-column architecture • Each cell was modeled as a single integrative compartment (point neuron) with a spike mechanism, • calcium-dependent (AHP) channels, and • voltage-sensitive A and M (muscarinic) potassium channels. • M.M. Kellog, H.R. Wills, and P.H. Goodman. “A biologically realistic computer model of neocortical associative learning for the study of aging and dementia.” J. Investig. Med., 47(2), February 1999

  14. NCS History • Version 1b: 1999 • Direct translation to C from Matlab • Ali Etazadi-Amoli and Keith Weslowski • 24 times faster. • tested on mixed excitatory-inhibitory networks of up to 1,000 cells.

  15. NCS History • Version 2: 1999 • code was then redesigned and rewritten for distributed processing on an existing 20-cpu cluster (Pentium II). • Initial trials of this code were performed on cortical networks of 2 to 1,000 cells.

  16. NCS History • Version 3: 2001 • completely redesigned using object-oriented design principles and recoded in C++ • objects, such as cells, compartments, channels, and the like, model the corresponding cortical entities. • The cells, in turn, communicate via messages passed through synapse objects. • Input parameters allow the user to create many variations of the basic objects, in order to model measured or hypothesized biological properties. • E. Courtenay Wilson, Phillip H. Goodman, and Frederick C. Harris, Jr. “Implementation of a biologically realistic parallel neocortical-neural network simulator” in Proceedings of the 10th SIAM Conf. on Parallel Process. for Sci. Computing, Portsmouth, Virginia, March 2001.

  17. NCS History E. Courtenay Wilson, Frederick C. Harris, Jr., and Phillip H. Goodman. “A large-scale biologically realistic cortical simulator” in Proceedings of SC 2001, Denver, Colorado, November 2001

  18. Hardware • Several grants from DoD for hardware • 2001 – 30 dual PIII (2GB/core) • 2002 – 34 dual PIV (2GB/core), Myrinet connection • 2004 – 40 dual opterons (2 GB/core) • 2007 – 4 16core opteron boxes (128GB/box) and 3 24TB disk arrays, Infiniband connection

  19. Web Interface • Goals: • Allow rapid creation of brain models • Allow remote collaboration Kishor K. Waikul, Lianjun Jiang, E. Courtenay Wilson, Frederick C. Harris, Jr., and Philip H. Goodman, “Design and Implementation of a Web Portal for a NeoCortical Simulator,” in Proceedings of the 17th International Conference on Computers and Their Applications (CATA 2002) pp. 349-353, April 4-6, 2002, San Francisco, CA

  20. Code Optimization & Revisions • Rewrote the input parser • Worked on code base • sevenfold sequential speedup over the version 3 code • added new features while shrinking our code base by more than 25%. • Added More Biological Parameters. • 35,000 cells and approximately 6.1 million synapses using 72% of the available 4GB of memory per node.

  21. Code Optimization James Frye, James G. King, Christine J. Wilson, and Frederick C. Harris, Jr. “QQ: Nanoscale timing and profiling” In Proceedings of PMEO-PDS, Denver, CO, April 3-8 2005.

  22. Modeling & Neural Engineering Ripplinger MC, Wilson CJ, King JG, Frye J, Drewes R, Harris FC, and Goodman PH, “Computational Model of Interacting Brain Networks,” Journal of Investigative Medicine, Vol. 52, No. 1, Jan 2004, pp. S155.

  23. Modeling & Neural Engineering

  24. (bAC) KAHP Modeling & Neural Engineering

  25. Modeling & Neural Engineering Romain Brette, Michelle Rudolph, Ted Carnevale, Michael Hines, David Beeman, James M. Bower, Markus Diesmann, Abigail Morrison, Philip H. Goodman, Frederick C. Harris, Jr., Milind Zirpe, Thomas Natschlager, Dejan Pecevski, Bard Ermentrout, Mikael Djurfeldt, Anders Lansner, Olivier Rochel, Thierry Vieville, Eilif Muller, Andrew P. Davison, Sami El Boustani and Alain Destexhe, “Simulation of networks of spiking neurons: A review of tools and strategies,” Journal of Computational Neuroscience Vol. 23, December, 2007, pgs 349-398.

  26. extensive domain of self-sustained asynchronous irregular firing R N Modeling & Neural Engineering

  27. Modeling & Neural Engineering • RAIN Firing Characteristics

  28. Robotic/Human Loops • Juan C. Macera, Philip H Goodman, Frederick C. Harris, Jr., Rich Drews, and James B. Maciokas • “Remote-Neocortex Control of Robotic Search and Threat Identification,” Robotics and Autonomous Systems,  Vol. 46, No. 2, February 2004, pp 97-110.`

  29. Robotic/Human Loops Qunming Peng “Brainstem: A NeoCortical Simulator Interface for Robotic Studies”, MS Thesis December 2006, University of Nevada,Reno

  30. Robotic/Human Loops Coarse Gabor Filters transduced into a raster of spikes VC 30

  31. Robotic/Human Loops Engaging & Rewarding Evocative 31

  32. Robotic/Human Loops • Virtual Robots Philip H. Goodman, Sermsak Buntha, Quan Zou, Sergiu M. Dascalu, :Virtual neurorobotics (VNR) to accelerate the development of plausible neuromorphic brain architectures” Frontiers in Neurorobotics 1(1) pp. 1-7 November 2007.

  33. Other Preliminary Work • General Movement Disorders • Parkinsons • Autism • Goal: model properly • That way we can study changes

  34. Blue Brain • EPFL: IBM BlueGene, 8096-CPU cluster, 22 Trillion Flops

  35. Objectives: • Understand scientific basis for superiority of human intelligence over current machine learning and AI • Create neurally-based cognitively intelligent systems • Develop neuromorphic robots which interact with humans • Complement Neuroscience wet lab and cognitive research

  36. Impacts: • Multi-compartment and million-cell brain models • New theory of “mesocircuit” to link biology and behavior • Robotic prototype advanced from CANINE to HUMANOID robot • Enhanced new field of “virtual neurorobotics” (VNR)

  37. Present Scope of Work Goals Neuroscience Modeling & Neural Engineering Robotic/Human Loops

  38. A Novel Multi-GPU Neural Simulator • The proof-of-concept simulation code described here is presented as an illustration of both the design's scalability and performance potential once integrated to the existing environment. • Written in CUDA • With hooks for MPI

  39. A Novel Multi-GPU Neural Simulator • Supports the same input file as NCS • We are adding support for NeuroML • Once input has been read, Initialization begins.

  40. Initialization Overview • The neurons are sorted based on the number of synaptic connections. • These are then distributed to the respective GPUs in a round-robin fashion. • Then simulation begins

  41. Initialization Overview • Local Data structures are then created.

  42. Simulation • Now that we are finished with setup we go on to the simulation:

  43. Simulation • The simulation begins by updating (numerically integrating) the neurons.

  44. Simulation • The cell firings are exchanged and we update the cell computations

  45. Testing • Some basic benchmarks were run to illustrate the scalability and functionality of the design. • The test network was based on the polychronization models from Izhikevich et al. [10] and Szatet al. [11].

  46. Testing • This network utilizes a ratio of four excitatory neurons to 1 inhibitory neuron. • M/N is the probability of connections • M is the number of connections per Neuron. N is the number of Neurons.

  47. Results

  48. Results • Small amount of data to transfer • 100,000 neurons with 50 connections per neuron can run at 1.2 times faster than real time.

  49. Conclusions/Future Work • This was a worst case scenario. • And it is better than real time. • We have already moved to a cluster of boxes of GPUs [and have a proposal out for money to purchase a better/larger GPU cluster]

  50. A Novel Multi-GPU Neural Simulator Corey Thibeault, Roger Hoang Frederick C. Harris, Jr. Brain Computation Laboratory University of Nevada, Reno

More Related