Caching and replay of place sequences in a Temporal Restricted Boltzmann Machine model of the hippocampus

1. Caching and replay of place sequences in a Temporal Restricted Boltzmann Machine model of the hippocampus Sue Becker, Dept. of Psychology Neuroscience & Behaviour, McMaster Univ., becker@mcmaster.ca and Geoff Hinton, Dept. of Computer Science, Univ. of Toronto 1. Summary 3. Learning in RBMs and TRBMs 5. Training patterns Place tuning in linear track environment The hippocampus is thought to be critical for the rapid formation and cued recall of complex event memories. For example, a set of hippocampal place cells that fires in a particular sequence during exploration may fire in the same sequence during sleep [1,2,3]. We propose a novel model of the hippocampal memory system that reconciles a wide range of neurobiological data. Here we address the question of how the hippocampus encodes sequences rapidly, and what is the function of sequence replay. Our model draws upon two recent developments of the Restricted Boltzmann Machine (RBM): 1) a hierarchy of sequentially trained RBM's [4], and 2) the extension to sequential inputs employed in the Temporal RBM [5]. The top two layers of the model, representing the dentate gyrus (DG) and CA fields, are connected via undirected links to form an autoassociator, allowing the model to generate memories of coherent events, rather than generating top-level unit states independently. The CA region also has directed connections from previous CA states, representing the CA3 recurrent connections. Thus the probability distribution learned over the visible and hidden units is conditioned on earlier states of the autoassociator. The model is trained by contrastive Hebbian learning, with data-driven and generative phases providing the statistics for the positive and negative Hebbian updates respectively. This is broadly consistent with Hasselmo's proposal that the hippocampus oscillates between encoding and recall modes within each theta cycle [6]. When trained on a variety of spatial environments (fig 5), the hippocampal units develop place fields (fig 6), while the CA time-delayed recurrent collaterals encode the sequential structure of the data (fig 7). It has been widely assumed that sequence replay is required for memory consolidation, in which rapidly stored hippocampal memories are gradually transferred to cortex. Alternatively, the hippocampus may always be required to recall complex associative memories, while replay allows the brain to maintain consistent forward and generative models in the hippocampal-cortical system [7]. 3.1 Restricted Boltzmann Machines Figure 4: Spatial locations are coded by Boundary Vector Cells (BVCs) [11]. Each BVC’s tuning curve is a product of 2 Gaussians, tuned to distance from and direction to environmental boundaries Probabilities of states P(V,H) are proportional to their Boltzmann free energies. In an RBM, conditional probabilities of H & V can be computed efficiently: Figure 6: Place tuning of several CA units Sequence replay X Figure 5: The model was trained on 3 environments, a linear track, a U-shaped maze and a rectangular box. Left: U-maze, with training locations shown by blue dots. Right: the reconstruction of the boundaries from the BVC representation of the location marked X in the U-maze. Time 3.2 Contrastive divergence learning [10] Maximizing likelihood requires settling to equilibrium. Instead, minimize diff. between 2 K.L. divergence terms: 2. Data not explained by most HC models 6. Simulation results Figure 7: Model’s reconstruction of a sequence of spatial locations along the linear track. • Anatomy (fig 1) • Cascaded architecture • Reciprocal connectivity btw DG<-->CA3 and CA<-->EC • Activation dynamics • (fig 2) • Theta vs sharp waves • Sequence replay (fig 3) • slow during REM sleep • fast during SWS, sharp waves • Learning dynamics • LTP in phase w theta rhythm • LTD in anti-phase w theta rhythm Input reconstruction 3.3 Temporal Restricted Boltzmann Machines 7. Discussion C2 This is one example of a class of models called TRBM discussed in [5]. The model learns spatial environments gradually, but can rapidly cache state sequences in its temporal generative model (CA recurrent collaterals) Making the temporal connections bi-directional requires propagation of states backwards in time, accounting for reverse sequence replay. The addition of sparseness constraints sharpens place-tuning and leads to a grid-like representation C1 Figure 1. Cascaded architecture of the HC. W Figure 2. Left:Theta-modulated gamma ripples (fig 2A from Chrobak et al 2000) [8]. Right: Sharp waves (fig 1 A, C from Chrobak & Buszaki, 1994). [9] Reproduced with permission. 8. References [1] Wilson, M.A. and McNaughton, B.L. (1994), Science 265(5172):676-679 [2] Louie, K; Wilson, MA (2001), Neuron 145-156 [3] Lee, A.K. and Wilson, M.A. (2002) Neuron 36(6):1183-1194 [4] Hinton, G. E., Osindero, S. and Teh, Y. (2006), Neural Computation 18(7):1527-1554. [5] Sutskever, I. and Hinton, G. E. (2006) Technical Report UTML TR 2006-003. [6] Hasselmo, M.E. et al (2002), Neural Computation, 14(4): 793-817. [7] Kali, S. and Dayan, P. (2004) Nature Neuroscience 7(3):286-294 [8] Chrobak,J.J., Lorincz,A. & Buszaki,G. (2000) Hippocampus 10(4):457-65 [9] Chrobak & Buszaki, (1994) J Neurosci 14:6160-6170 [10] Hinton, G.E. (2002) Neural Computation, 14(8):1711–1800 [11] Hartley, T., Burgess, N., Lever, C., Cacucci, F. and O'Keefe, J. (2000), Hippocampus 10(4):369-379 4. A TRBM model of the hippocampus Reproduced with permission from Fig 2 Lee & Wilson 2002.[3] Figure 3. Top: Smoothed raster plots of spikes vs sequentially traversed spatial locations (5-second time scale). Bottom:Same sequence of 6 place cells fires during slow wave sleep (150msec time scale). Figure 5: Examples of boundary reconstructions from BVC input patterns (left) and from model’s reconstruction of input (right) averaged over 5 binary stochastic samples Figure 4. Simplified HC architecture used here. This work was supported by funding from the Natural Sciences and Engineering Research Council of Canada (SB, GEH) and CIAR (GEH)