Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern California

1. Itti: CS564 - Brain Theory and Artificial IntelligenceUniversity of Southern California • Lecture 28. Overview & Summary • Reading Assignment: • TMB2 Section 8.3 • Supplementary reading: Article on Consciousness in HBTNN

2. You said “brain” theory?? • First step: let’s get oriented!

5. Major Functional Areas • Primary motor: voluntary movement • Primary somatosensory: tactile, pain, pressure, position, temp., mvt. • Motor association: coordination of complex movements • Sensory association: processing of multisensorial information • Prefrontal: planning, emotion, judgement • Speech center (Broca’s area): speech production and articulation • Wernicke’s area: comprehen- • sion of speech • Auditory: hearing • Auditory association: complex • auditory processing • Visual: low-level vision • Visual association: higher-level • vision

6. Neurons and Synapses

7. http://www.radiology.wisc.edu/Med_Students/neuroradiology/fmri/http://www.radiology.wisc.edu/Med_Students/neuroradiology/fmri/

14. Limbic System • Cortex “inside” the brain. • Involved in emotions, sexual behavior, memory, etc • (not very well known)

15. Major Functional Areas

16. Some general brain principles • Cortex is layered • Retinotopy • Columnar organization • Feedforward/feedback

17. Layered Organization of Cortex • Cortex is 1 to 5mm-thick, folded at the surface of the brain • (grey matter), and organized as 6 superimposed layers. • Layer names: • 1: Molecular layer • 2: External granular layer • 3: External pyramidal layer • 4: internal granular layer • 5: Internal pyramidal layer • 6: Fusiform layer • Basic layer functions: • Layers 1/2: connectivity • Layer 4: Input • Layers 3/5: Pyramidal cell bodies • Layers 5/6: Output

18. Retinotopy • Many visual areas are organized as retinotopic maps: locations next • to each other in the outside world are represented by neurons close • to each other in cortex. • Although the topology is thus preserved, the mapping typically is highly non-linear (yielding large deformations in representation). • Stimulus shown on screen… and corresponding activity in cortex!

19. Columnar Organization • Very general principle in cortex: neurons processing similar “things” are grouped together in small patches, or “columns,” or cortex. • In primary visual cortex… as in higher (object recognition) visual areas… • and in many, non-visual, areas as well (e.g., auditory, motor, sensory, etc).

21. Interconnect Felleman & Van Essen, 1991

22. Neurons??? • Abstracting from biological neurons to neuron models

23. The "basic" biological neuron • The soma and dendrites act as the input surface; the axon carries the outputs. • The tips of the branches of the axon form synapses upon other neurons or upon effectors (though synapses may occur along the branches of an axon as well as the ends). The arrows indicate the direction of "typical" information flow from inputs to outputs.

24. Transmenbrane Ionic Transport • Ion channels act as gates that allow or block the flow of • specific ions into and out of the cell.

25. Action Potential and Ion Channels • Initial depolarization due to opening sodium (Na+) channels • Repolarization due to opening potassium (K+) channels • Hyperpolarization happens because K+ channels stay open longer than Na+ channels (and longer than necessary to exactly come back to resting potential).

26. Warren McCulloch and Walter Pitts (1943) • A McCulloch-Pitts neuronoperates on a discrete time-scale, t = 0,1,2,3, ... with time tick equal to one refractory period • At each time step, an input or output is • on or off — 1 or 0, respectively. • Each connection or synapse from the output of one neuron to the input of another, has an attached weight.

27. From Logical Neurons to Finite Automata 1 Brains, Machines, and 1.5 Mathematics, 2nd Edition, 1987 AND 1 Boolean Net 1 ® X Y 0.5 OR 1 X NOT Finite 0 Automaton -1 Y Q

28. Leaky Integrator Neuron • The simplest "realistic" neuron model is a continuous time model based on using • the firing rate (e.g., the number of spikes traversing the axon in the most recent 20 msec.) • as a continuously varying measure of the cell's activity • The state of the neuron is described by a single variable, the membrane potential. • The firing rate is approximated by a sigmoid, function of membrane potential.

29. Leaky Integrator Model m(t) m(t) m(t) • t = - m(t) + h • has solution m(t) = e-t/t m(0) + (1 - e-t/t)h •  h for time constant t > 0. • We now add synaptic inputs to get the • Leaky Integrator Model: • t = - m(t) + i wi Xi(t) + h • where Xi(t) is the firing rate at the ith input. • Excitatory input (wi > 0) will increase • Inhibitory input (wi < 0) will have the opposite effect.

30. Models of what? • We need data to constrain the models • Empirical data comes from various experimental techniques: • Physiology • Psychophysics • Various imaging • Etc.

31. Electrode setup • drill hole in cranium under anesthesia • install and seal “recording chamber” • - allow animal to wake up and heal • because there are no pain receptors • in brain, electrodes can then • be inserted & moved in chamber • with no discomfort to animal.

32. Receptive field

33. Example: yes/no task + time Example of contrast discrimination using yes/no paradigm. • subject fixates cross. • subject initiates trial by pressing space bar. • stimulus appears at random location, or may not appear at all. • subject presses “1” for “stimulus present” or “2” for “stimulus absent.” • if subject keeps giving correct answers, experimenter decreases contrast of stimulus (so that it becomes harder to see).

34. Staircase procedure • Staircase procedure is a method for adjusting stimulus to each observer such as to find the observer’s threshold. Stimulus is parametrized, and parameter(s) are adjusted during experiment depending on responses. • Typically: • - start with a stimulus that is very easy to see. • - 4 consecutive correct answers make stimulus more difficult to see by a fixed amount. • - 2 consecutive incorrect answers make stimulus easier to see by a fixed amount.

35. Example SPECT images

36. Reconstruction using coincidence

37. Example PET image

38. BOLD contrast The magnetic properties of blood change with the amount of oxygenation resulting in small signal changes with oxygenated with deoxygenated blood blood Bo Bo

39. Vascular System arteries arterioles capillaries venules veins (<0.1mm) (<0.1mm)

40. Oxygen consumpsion The exclusive source of metabolic energy of the brain is glycolysis: C6H12O6 + 6 O2 6 H2O + 6 CO2

41. BOLD Contrast stimulation neuronal activation metabolic changes hemodynamic changes local susceptibility changes MR-signal changes signal detection data processing functional image

42. Example of Blocked paradigm Gandhi et al., 1999

43. First BOLD-effect experiment • Kwong and colleagues at Mass. General Hospital (Boston). • Stimulus: flashing light.

44. Summary 2

45. Case study: Vision • Vision is the most widely studied brain function • Our goals: • analyze fundamental issues • Understand basic algorithms that may address those issues • Look at computer implementations • Look at evidence for biological implementations • Look at neural network implementations

46. Eye Anatomy

47. Visual Pathways

48. Retinal Sampling

49. Origin of Center-Surround • Neurons at every location receive inhibition from neurons at neighboring locations.

50. Origin of Orientation Selectivity • Feedforward model of Hubel & Wiesel: V1 cells receive inputs from LGN cells arranged along a given orientation.