Neural Decision Making: Imprecision & Noise in Computational Neuroscience
170 likes | 270 Vues
Explore the bewilderingly vast topic of neural decision making with models playing a central role. Beware of self-confirmation battles in this fusion of ethology, economics, and psychology, where economic choices and instrumental/Pavlovian conditioning impact decision-making algorithms. Delve into neural computations involving neuromodulators like amygdala and prefrontal cortex, while navigating uncertainty with Bayesian sensory inference and game theory. Understand the implementation of neural decision-making through evidence accumulation, Q-learning, and dopamine-related processes. Discover diffusion to bound models, neural correlates like prediction errors, and metacognitive aspects. Further, ponder upon the structural correlates and the intriguing interplay between economics and neuroscience.
Neural Decision Making: Imprecision & Noise in Computational Neuroscience
E N D
Presentation Transcript
A View from the Bottom Peter Dayan Gatsby Computational Neuroscience Unit
Neural Decision Making • bewilderingly vast topic • models playing a central role • so beware of self-confirmation + battles
Ethology/Economics(?) optimality logic of the approach Psychology economic choices instrumental/Pavlovian conditioning Computation Algorithm Neural Decision Making prediction: of important events control: in the light of those predictions • Implementation/Neurobiology neuromodulators; amygdala; prefrontal cortex nucleus accumbens; dorsal striatum
Imprecision & Noise • computation • Bayesian sensory inference • Kalman filtering and optimal learning • metacognition • exploration/exploitation • game theory
Imprecision & Noise • algorithm • multiple methods of choice • instrumental: model-based; model-free • (note influence on RTs) • Pavlovian: evolutionary programming • uncertainty-modulated inference and learning • DFT/drift diffusion decision-making • MCMC methods for inference
Imprecision & Noise • implementation • (where does the noise come from?) • evidence accumulation • Q-learning and dopamine • metacognition and the PFC • acetylcholine/norepinephrine and uncertainty-sensitive inference and learning
Diffusion to Bound Britten et al, 1992
Diffusion to Bound • expected reward, priors affect starting point • some evidence for urgency signal • works for discrete evidence (WPT) • less data on >2 options • micro-stimulation works as expected • decision via striatum/superior colliculus/etc? • choice probability for single neurons Gold & Shadlen, 2007
dopamine and prediction error TD error L R Vt R no prediction prediction, reward prediction, no reward
Fiorillo et al, 2003 Tobler et al, 2005 Probability and Magnitude
Risk Processing < 1 sec 5 sec ISI 0.5 sec 2-5sec ITI You won 40 cents 5 stimuli: 40¢ 20¢ 0/40¢ 0¢ 0¢ 19 subjects (dropped 3 non learners, N=16) 3T scanner, TR=2sec, interleaved 234 trials: 130 choice, 104 single stimulus randomly ordered and counterbalanced
Neural results: Prediction errors what would a prediction error look like (in BOLD)?
Neural results I: Prediction errors in NAC raw BOLD (avg over all subjects) unbiased anatomical ROI in nucleus accumbens(marked per subject*) * thanks to Laura deSouza
Value Independent of Choice Roesch et al, 2007
Metacognition • Fleming et al, 2010 • contrast staircase for performance; type II ROC for confidence
Structural Correlate • also associated white matter (connections)
Discussion • what can economics do for us? • theoretical, experimental ideas • experimental methods • like behaviorism… • what can we do for economics? • large range of constraints • objects of experimental inquiry precisely aligned with economic notions • grounding/excuse for complexity…
