1. Progress Report R. Schuler 27 Feb 07

2. Since last time… • Studied: • Bar-Gad, I., Ruppin, E., Bergman, H., “Reinforcement Driven Dimensionality Reduction – A Model For Information Processing In The Basal Ganglia” • O’Reilly, R., Frank, M., “Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia” • Dominey, P., Arbib, M., Joseph, J.-P., “A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and Sequences” • Discussed: • Rahul Jaitly and I discussed integration of our models and agreed to define interfaces for the BG module such that we can incorporate his work into my WCST model • Explored: • Read about NSL/SCS alternatives of GENESIS, NEURON • Followed an Eclipse Plugin Development Environment (PDE) tutorial to develop an example plugin • Read up on custom Ant task development

3. Good: Implements end-to-end simulation of WCST task using corticothalmic loop Novel implementation of dynamic gating in WM Bad: TestManager needs improvement in order to properly measure perseverative errors BasalGanglia does not learn the task nor is there an actor-critic architecture Missing lateral inhibition and a WTA mechanism Current State of WCST Model

4. WorkingMemory Separate the WM module from the WCST model Document and enter in BODB as standalone module Parameterize WM’s NN layers’ bias, gain, noise, etc. Task 1: Factor out WM

5. BG Module Inteface Work with Rahul to agree on and to define an interface to the BG module for input, output, and reinforcement signals Rework BG Module Update my BG to comply with new interface Factor out my BG into interchangeable module (so that in the future I can swap it out for Rahul’s BG) Task 2: Rework BG

6. Two major improvements to the Test Manager are needed Improvement 1: Minor bug in the way my TM administers the WCST task -- should keep target cards fixed until a rule category is completed Improvement 2: Major bug in the way my TM evaluates the WCST task – does not record perseverative errors until at least ONE category is successfully completed Task 3: Improve TestManager

7. In my work leading to Project 4 of CS 564, I chose NN values (weights, gains, biases) pragmatically based on what “worked” I need to revisit these NN parameters according to the literature (Amos 2000 and others) Also, I left out lateral inhibition from the SNr due to time limitation Task 4: Revisit NNs

8. Based on: Factored out WM Refactored BG Revised NNs Fixed TestManager Now: Rerun WCST tests for simulated “healthy” and “patient” subjects Compare with simulated and experimental “healthy” and “patient” subjects from (Amos 2000) Milestone I

9. Integrate Rahul’s BG This component will be implemented by Rahul and used in my WCST model Given an agreed upon interface for input, output, and reinforcement signals this should not require too much effort (wishful thinking?) Reinforcement Signal It may (or may not!) be possible to resuse the TM’s “reward” signal for the BG’s reinforcement signal If not, the TM along with the entire WCST model will need some significant improvement to support a training cycle and a reinforcement signal sufficient to train the BG Task 5: Integrate Rahul’s BG Reinforcement

10. Based on: Milestone I tasks Integration of Rahul BG Upgrades to WCST model to support training cycle and reinforcement signal Now: Rerun WCST tests for simulated “healthy” and “patient” subjects Compare with Milestone I results Compare with simulated and experimental “healthy” and “patient” subjects from (Amos 2000) Milestone II Reinforcement

11. …Beyond Milestone II • I think Milestone II is achievable within the timeframe of this semester, beyond Milestone II there are some interesting things to explore if time permits • WM module in DAJ Model • The DAJ Model, specifically the Sequence Learning task, depends on working memory • I do not yet have a plan for how to integrate my WM module in the DAJ model but I’d like to explore a visual task as an alternative to the WCST planning task and compare the performance of a dynamically gated WM module • O’Reilly & Frank BG Model • The O’Reilly & Frank model is interesting BUT quite complex and beyond the scope of what I think can be implemented in a semester of work • Having said that, time permitting, a nice alternative to my current BG and the Rahul BG would be an O’Reilly & Frank-inspired BG – i.e., one that incorporates a more sophisticated learning mechanism and actor-critic architecture