Learning Rules 1

1. Learning Rules 1 Computational Neuroscience 03 Lecture 8

2. Last week showed how to model neurons and networks of neurons using firing rate models. And we then discussed how to add them together to form networks of neurons

3. Feedforward and Recurrent networks Where the weight vector is replaced by a matrix. Also often replace feedforward input with a vector

4. Recurrent networks can also do this but have much more complex dynamics than feedforward nets. Also more difficult to analyse Much analysis focuses on looking at the eigenvectors of the matrix M Can show for instance that networks can exhibit selective amplification if there is one dominant eigenvector (cf PCA)

5. Or if an eigenvalue is exactly equal to 1 and others < 1can get integration of inputs and therefore persistent activity as activity does not stop when input stops But how are we to generate such precise weight changes? Need some synaptic modification rules

6. Hebb’s postulate Hebb's postulate of learning (or simply Hebb's rule) (1949), is the following: "When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth processes or metabolic changes take place in one or both cells such that A's efficiency as one of the cells firing B, is increased". This rule forms the basis of much of the research on role of synaptic plasticity in memory and learning Has been generalised to include decreases of strength when neuron A repeatedly fails to be involved in activation of B and generally look at the correlation or covariance of activities of pre-and postsynaptic neurons

7. Learning in the hippocampus Here see examples of long-term potentiation and depression (LTP and LTD). High frequency stimulation induces LTP while long-lasting low frequency leads to LTD ‘Long Term’ refers to changes > 10 mins

8. Learning Rules • We will consider 3 types of learning and will focus on Hebbian type learning though others exist (modifcations on pre/postsynaptic activity only etc) • Unsupervised learning: Network responds during training solely as a result of its connections and intrinsic dynamics. Net self-organises in a manner dependent on inputs and synaptic plasticity rule • Supervised learning: Here the network also has a “teacher” in the form of a set of desired ‘target’ outputs for each input. Not especially biologically plausible, but good for existence proofs • Reinforcement learning: Somewhere in between the 2. The net does not know the target output but gets feedback via reward punishment

9. Unsupervised Learning Start with single postsynaptic neuron and a linear activation function. As synaptic changes will be much longer time scale than these dynamics firing rate equation reduces to: Start with simplest Hebbian style plasticity for a single neuron:

10. As u, v denote firing rates vu can be interpreted as a probability that pre- and post fire together. Each different set of u is known as an input pattern. As weight changes are slow, rather than summing all changes separately can average the input patterns and thus compute the average change. To do this use < > to denote averages over the ensemble of input patterns. Thus get: Remembering that v = w.u this gives the correlation-based rule Where Q is correlation matrix: Qbb’ = <ubub’>

11. eg for 2 patterns u1 and u2, Qij = ½(u1i u1j + u2i u2j) So suppose u1= (1, 0) and u2= (0, 1) then Alternatively u1= (1, 1) and u2= (1, 0) then Suppose tw=10 and w=(0.1,0.1) and we have the 2nd matrix then: As wn+1= wn + Dw, w1=(0.115, 0.11), w2=(0.13, 0.12) …. Which leads to instability and uncontrolled growth of w

12. W=(0.1, 0.1) W=(0.1, -0.3) Outcome dependent on eiegenvectors of Q and initial conditions, but unstable

13. To avoid unbounded growth can impose a saturation constraint. However, this means all weights go to max or min and thus we have no competition between different synapses This means that the neuron cannot distinguish between presynaptic inputs Also since u, v are firing rates and therefore positive, Hebb rule only describes LTP. However, earlier figure showed that synapses can depress in strength if presynaptic activity is accompanied by a low level of postsynaptic activity Can also get results where opposite is true

14. Analysis focuses on looking at eigenvectors of Q the correlation matrix of the input vectors For instance, can show that after training Hebbian rule leads to v ae1.u for arbitary vector u and that weight vector expressed as a sum of eigenvectors is dominated by e1 ie wae1 That is v is projection of u onto the principal eigenvector of Q Eg for a Q with principal eigenvector (1, -1)/sqrt(2) we would expect w to end up as (wmax,0) or (0, wmax). However because of saturation constraints can get (wmax, wmax)

15. Covariance rule To get LTD can introduce a postsynaptic or presynaptic threshold: Below thresholds get depression, above potentiation. A convenient choice for the thresholds is the average pre/post synaptic input <u> or <v>. If we now replace v with w.u we get the covariance rule Where C is the covariance matrix of the input data ie:

16. Note that as <v> keeps changing we need to keep updating the postsynaptic threshold while the presynaptic one is independent of the weights Although both average to the same thing, they do have differences. The postsynaptic threshold means that only modifies weights for non-zero presynaptic activities. If v is below threshold then this results in homosynaptic depression Alternatively, presynaptic threshold reduces the strength of inactive synapses for v>0: heterosynaptic depression Although the covariance rule allows LTD it is still unstable due to positive feedback Also we do not have competition, but this can be introduced to allowing threshold to slide as follows

17. BCM rule As covariance rule allows LTD without postsynaptic/presynaptic activity, Bienenstock, Cooper and Munro (82) proposeed an alternative for which there is experimental evidence where the postsynaptic threshold is dynamic

18. This is again unstable if q is fixed. However, if we allow the threshold to grow faster than v we get stability. For instance use q as low pass filtered version of v2 Usually set tq to be less than tw so that changes in q faster than changes in v Now get competition between synapses since strengthening some synapses results in threshold increasing meaning that it is harder for others to be strengthened

19. Synaptic normalisation A more direct way of enforcing competition is through synaptic normalisation Idea is that postsynaptic neuron can only support a certain amount of total synaptic weight so strengthening one leads to weakening others Can either hold the sum of weights constant if all are +ve or –ve or can constrain the sum of squares of the weights (cf ANN network pruning) 2 types: subtractive normalisation and multiplicative normalisation

20. Subtractive normalisation Where n is a vector of ones of length Nu so n.u is the sum of all the inputs u. Thus the second term is simply a vector Nu long with the same values in ie (k, k, ….., k) whose sum over all the elements is equal to the sum over all the elements of vu. Thus the total increase in the weights is 0 This rule must be augmented by a saturation constraint to prevent the weights becoming negative, that if a weight becomes zero, it is not moved downwards Also, without upper saturation often leads to all weights bar one being zero. Note also rule involves global knowledge of weights

21. Multiplicative normalisation Where a is a +ve constant: known as Oja’s rule (1982) This rule is more local than previous as it only involves the weight in question and pre-and post synaptic activities. However, its form is based on theoretical arguments rather than experimental data Previous rule was rigid as it had to be satisfied at all times whereas this is more dynamic with |w|2 gradually relaxing to 1/ a This induces competition as if one weight increases, the maintenance of constant length of the weight vector forces others to decrease

22. Both Hebbian and Oja’s rule run for long enough generate vectors parallel to principal eigenvector of correlation matrix as in A This is basically principal component analaysis (PCA) which is theoretically the optimal in terms of retaining info way to encode high dimensional info onto lower dimensional subspace However, B shows what happens if input vectors don’t have zero mean (as in real systems), but this problem is alleviated by using covariance-based rules

23. Timing based rules Previous rules don’t take timing into account Can be crucial since if pre-synaptic spike occurs after postsynaptic get LTD rather than LTP

24. Therfore need to integrate over time as in the following: Where H is a function like the solid line in previous figure Such functions still require saturation constraints but timing can generate competition

25. Multiple Postsynaptic Neurons Can extend the rules defined previously to nets with multiple postsynaptic neurons. In these networks the output rates are: Thus where And the Hebbian rule becomes:

26. Can also use feature-based models where the net is indexed by input features rather than buy individual inputs Network models can have adaptive feedforward wieghts and fixed recurrent ones, or vice versa or both layers can be adaptive Can get competition through mainly inhibitory recurrent connections Can then get eg self-organising maps and elastic nets where K can have forms like: