Parallel neural Circuit SIMulator (PCSIM) - Tutorial

1. Parallel neural Circuit SIMulator (PCSIM) - Tutorial Dejan Pecevski Institute for Theoretical Computer Science Graz University of Technology FIAS Theoretical Neuroscience Summer School, Frankfurt, August, 2008

2. Outline of the Tutorial • General Info: What is PCSIM? • Features: What is PCSIM useful for? • Network elements: neurons and synapses • Scalability: Using clusters for parallel simulation. • Python Interface • PCSIM basic operations • Creating and connecting neurons • Simulating • Recording signals • Populations and Projections • Summary

3. Who am I? • Ph.D. Student at theInstitute for Theoretical Computer Science, Graz University of Technology • One of the developers of PCSIM. • Research Interests • Plasticity and Learning mechanisms in neural circuits • Spiking Neural Networks • Simulation technologies/algorithms for biological neural networks

4. PCSIM in a nutshell • Simulator for parallel simulation of spiking and analog neural networks with point neuron models • Implemented in C++, with Python and Java interfaces • High-level definition of neural networks • Library of built-in neuron and synapse models • Part of FACETS standardization efforts: the pyNN interface http://www.neuralensemble.org • Open-source, free, released under the GPL license, web page at: http://www.igi.tugraz.at/pcsim

5. Developers Thomas Natschläger Ph.D.Software Competence Center Hagenberg Hagenberg, Austria Dejan Pecevski DIInstitute for theoretical computer scienceGraz University of Technology Graz, Austria Other Contributors: Klaus Schuch DI (IGI), Christian Ernstbrunner DI (SCCH), Walter Hargassner DI (SCCH)

6. Feature Overview • Object Oriented Design • The user interface/view is object oriented • General Communication System • Analog and spiking messages • Network elements with multiple input and output ports • Supported in distributed/multi-threaded mode • Flexible construction of more complex network architectures • Populations of simulated objects • Projections for connectivity specification

7. Feature Overview • Parallel simulation • Mixed multi-thread and distributed • Easy to use, transparent to the user • Easily extensible • A compilation tool for creating PCSIM extension packages • Usable as backend component in C++, Python, Java, … • Easier and faster setup of simulations in Python and Java • Runs on Linux or other Unix-like platforms • can be compiled under Windows (still not tested).

8. PCSIM high-level structure

9. Type of models PCSIM is suitable for • Large recurrent networks with simple point neurons • Distributed and multi-threaded capabilities • Efficient simulation engine in C++ • More than 105 spiking neurons and 108 synapses • Hybrid models • Abstract modules (filters etc.) combined together with circuits of analog neurons and spiking neurons • Extensible with new custom elements • Closed loop/feedback models

10. Type of models PCSIM is suitable for • Structured models • Networks composed of smaller subnetworks (Populations) • Probabilistic connectivity patterns based on neuron’s attributes • Diversity of neurons and synapses • Different neuron types in the neuron populations • Specifying random distribution for parameter values • New neuron and synapse types can be implemented

11. Examples of PCSIM usage • Spiking neural network model of the V1 area in the visual cortex of mammals (Schuch & Rasch) • Testing the computational performance of laminar cortical models based on experimental data. (Schuch & Haeusler) • Experiments to examine the learning capabilities of reward-modulated spike-timing-dependent plasticity (Legenstein, Pecevski, Maass)

12. A simple model represented in PCSIM Model equations • The equations can be decoupled into separate simulated elements. • Presynaptic neurons send spikes to the synapses. • Synapses inject currents (or modify conductances) in the postsynaptic neuron.

13. Network element: basic building block • Nodes of the network (dynamical systems) • Advanced/integrated each time step of the simulation • Represented by class SimObject • Neurons, synapses, recorders are derived from this class • Multiple input and output ports, either analog or spiking. • Connections are formed from output to input ports of the same type. • Every element in PCSIM

14. Neurons and Synapses as Network Elements • Synapses as network elements are attached to the postsynaptic neuron • Synapses have one input port, no output ports • Neurons have only one output, no inputs • The spiking and analog connections can connect neurons on different nodes

15. Spiking Models LifNeuron, CbLifNeuron Leaky integrate-and-fire HHNeuron Hodgkin-Huxley Neuron IzhiNeuron(Izhikevich, 2004) aeIFNeuron Adaptive Exponential I&F (Brette and Gerstner, 2005) LinearPoissonNeuronLeaky integrate with Poisson output Input Neurons PoissonInputNeuron SpikingInputNeuron Analog Neurons LinearAnalogNeuron Built-in Neuron Models

16. Type current based conductance based PSR Kernel exponential, alpha, double-exponential, Square Short-term Plasticity (Markram et al. 1998) Spike-timing-dependent Plasticity (Froemke and Dan, 2002) (Gütig et al, 2003) each pair and nearest neighbour Other Mechanisms NMDA(Gabbiani et al. 1994) GabaB (Mainen et al. 1994) Reward-modulated STDP (Izhikevich, 2007) Homeostatic plasticity(Buonomano et al. 2005) Ornstein-Uhlenbeck noise synapse (Destexhe et al. 2001) Analog Synapse Built-in Synapse Models

17. Simulation Strategy • Clock-driven simulation with a fixed time step. • Hybrid Integration Strategy • Neurons are integrated every time step. • Synapses are event driven, i.e. they are integrated only after receiving a spike. • Numeric Integration Algorithms • Exponential Euler Method • ODE solvers from GSL

18. Parallel Simulation • Each MPI process advances subset of neurons • MPI as underlying communication layer • spiking and analog messages • Each process can have multiple-threads • Transparent to the user • Default round-robin distribution of the neurons over nodes • Custom distribution strategies • Easy retrieval of recordings (as in single-thread case)

19. Parallel Simulation • Why? • Speed-up simulation of an existing model by using more processors • Enables simulation of larger models: no memory limit problem • Scale up the number of neurons while keeping the simulation time the same. • Comparison between single-threaded and distributed simulation • < 104 neurons and 107 synapses on a single machine • > 105 neurons and 108 synapses on clusters • > 106 neurons and 1010 synapses on supercomputers?

20. Scalability of PCSIM Hardware: 3.4 Ghz, 512 kb cache, 4GB RAM Model: • 50000 LifNeurons, 4.0 Hz average firing rate • 50106 synapses, 1.5 ms transmission delay

21. Scalability of PCSIM Hardware: 3.4 Ghz, 512 kb cache, 4GB RAM Cuba Model: • 5000 LifNeurons per node, 4.0 Hz average firing rate • 103 synapses per neuron, 1.0 ms transmission delay, broad distribution of time synaptic constants

22. Extending PCSIM • What are extensions? • User implemented classes that are plugged into the PCSIM OO framework • usualy coded in C++ (can be coded also in Python) • Compilation of additional separate Python extension packages, without the need to recompile the main PCSIM code. • The extension classes are automatically wrapped in Python • Usual extensions for PCSIM • new neuron/synapse types • other user defined network elements • custom rules for specific network construction

23. Python Interface What is Python? (taken from www.python.org) • Python is a dynamic object-oriented programming language • offers strong support for integration with other languages and tools. • comes with extensive standard libraries (“batteries included”). • can be learned in a few days. • Many Python programmers report substantial productivity gains and feel the language encourages the development of higher quality, more maintainable code. • many scientific packages: scipy, numpy, matplotlib, ipython, Rpy etc.

24. Python Interface • PCSIM can be imported as a package within Python • The model is represented as a network object: • Individual neurons and synapses are accessible as python objects. • Population and Projection objects for high-level construction of networks composed of populations • Together with the other Python packages for scientific computing provides a complete working environment for neural simulations. from pypcsim import * net = SingleThreadNetwork()

25. Creating and connecting neurons # Creates one LIF neuron with default parameter values nrn_model = LifNeuron() nrn_id = net.create( nrn_model ) # Creates array of 10 LIFneurons nrn_array = net.create( LifNeuron(), 10) # Connects the first and the second neuron in the array syn_id = net.connect( nrn_array[0], nrn_array[1], StaticSpikingSynapse())

26. Specifying neuron/synapse parameters nrn = net.create(LifNeuron( Cm=2e-10, Rm=1e8, Vthresh=-50e-3, Vresting=-60e-3, Vreset=-60e-3, Trefract=5e-3, Vinit=-60e-3 )) ; syn = net.connect( pre_nrn, post_nrn, StaticSpikingSynapse( W=1e-10, tau= 5e-3, delay=1e-3 ))

27. Recording # record the output spikes of a neuron rec_spk = net.record( nrn, SpikeTimeRecorder() ) # record the membrane potential rec_vm = net.record( nrn, “Vm”, AnalogRecorder() ) # record the weight of a synapse rec_syn = net.record( syn, “W”, AnalogRecorder() ) # record any field in an element rec = net.record( myElement, “anyFieldName”, AnalogRecorder())

28. Simulate the model # simulate the network for 1 second net.simulate(1.0) # or advance it for 200 time steps net.advance(200)

29. Accessing individual neurons/synapses # creates return an ID of the neuron nrn_id = net.create( LifNeuron() ) # get the actual neuron object from the network nrn_object = net.object(nrn_id) # Manipulate the neuron object nrn_object.Vthresh = -59e-3 # getting the recorded values from a recorder values = list(net.object(rec_id).getRecordedValues())

30. Synapse classes nomenclature • StaticCurrExpSynapse (StaticSpikingSynapse) • DynamicCondDblExpSynapse • StaticStdpCondAlphaSynapse • DynamicCurrAlphaSynapse • DynamicNMDAAlphaSynapse Current/conductance based? Shape of PSR kernel Short-term plasticity

31. Hello, world! from pypcsim import * net = SingleThreadNetwork() pre_nrn = net.create( LifNeuron(Iinject = 5e-8) ) post_nrn = net.create( LifNeuron(Iinject = 5e-8) ) net.connect( pre_nrn, post_nrn, StaticSpikingSynapse()) rec = net.record( post_nrn, SpikeTimeRecorder() ) net.simulate(0.2) print “spike times:”, list(net.object(rec).getRecordedValues())

32. Specifying inputs # Create input neuron which spikes at specific times net.create(SpikingInputNeuron( [0.1, 0.2, 0.4])) # Create input neuron with Poisson process spike times net.create( PoissonInputNeuron(rate = 10, duration = 10) ) # Create analog input neuron with an # output analog signal specified as array net.create( AnalogInputNeuron( arange(0,10,1e-4) % 1 ) )

33. # create a population of 1000 LIF neurons popul = SimObjectPopulation(net, LifNeuron(), 1000) # create the projection proj = ConnectionsProjection( popul, popul, StaticSpikingSynapse(), RandomConnections( 0.1 ) ) # create a recorder population rec_popul = popul.record( SpikeTimeRecorder() ) Population of 1000 neurons randomly interconnected with 0.1 probability Population of recorders: one for each neuron in the population Creating and connecting many neurons

34. Random distributions for parameter values • Factory – an object that generates PCSIM elements • Neuron and synapse objects (used as models for creation) are simple clone factories • SimObjectVariationFactory – factory that attaches random distributions to parameter names nrn_factory = SimObjectVariationFactory( LifNeuron() ) nrn_factory.set( “Vinit”, NormalDistribution( -55e-3, 0.1 ) ) popul = SimObjectPopulation(net, nrn_factory, 1000)

35. exc_nrn = LifNeuron( Cm = 2e-10, Inoise = 1e-10) inh_nrn = LifNeuron( Cm = 3e-10 ) popul = SpatialFamilyPopulation(net, [ exc_nrn, inh_nrn], RatioBasedFamilies((4,1) ), CuboidIntegerGrid3D(20,10,10)) exc_popul, inh_popul = popul.splitFamilies() Each neuron has coordinates in 3D space The population is composed of different families of neurons (that can be of different type) Spatial heterogenous populations

36. Distance dependent random connections proj = ConnectionsProjection( popul, popul, StaticSpikingSynapse(), EuclideanDistanceRandomConnections( C, lambda ) ) Creates random connections with probability where D is the euclidean distance between the two neurons

37. Saving recorded data from pypcsimplus import * # Create the recording class r = Recordings() # Add the recorder populations as attributes to r r.spikes = popul1.record( SpikeTimeRecorder() ) r.vm_traces = popul2.record( “Vm”, AnalogRecorder() ) # Some additional variables to be saved r.v = 5 # Save the Recordings class to a HDF5 file r.saveInOneH5File(“results.h5”)

38. Running a parallel simulation • Change the network class from SingleThreadNetwork to either: • MultiThreadNetwork( nThreads = 4 ) • DistributedSingleThreadNetwork • DistributedMultiThreadNetwork • then run the script with mpirun $ mpirun –n 10 python my_experiment.py

39. Summary • PCSIM is a tool for simulation structured random neural networks composed of spiking and analog point neuron models • Has various built-in neuron and synapse models • Supports parallel simulation: distributed and multi-threaded • Flexible high-level construction of networks based on probabilistic rules • The Object-oriented framework is designed to support easy extensions • Can be used as a package within the Python interpreting programming language, together with other useful scientific packages

40. PCSIM Resources • Web page http://www.lsm.tugraz.at/pcsim • On Sourceforge http://www.sourceforge.net/projects/pcsim • Mailing listhttp://sourceforge.net/mailarchive/forum.php?forum_name=pcsim-users • User manualhttp://www.lsm.tugraz.at/pcsim/usermanual/html/index.html • C++ class referencehttp://www.lsm.tugraz.at/pcsim/cppclassreference/html/hierarchy.html

41. Thank you for your attention!