Gabor Filter: A model of visual processing in primary visual cortex (V1)

1. Gabor Filter: A model of visual processing in primary visual cortex (V1) Presented by: CHEN Wei (Rosary) Supervisor: Dr. Richard So

2. Agenda • Visual Physiological Mechanism • Primary Visual Cortex (V1) Properties • Gabor Filter Model • Implications • Q & A

3. Anatomy of the Early Visual Pathways LGN optic nerve optic radiation light primary visual cortex retina

4. Visual Pathway • Light fall into the eye, form two images on the left and right retina • Each retina transforms the incoming light intensity distribution into spike patterns, transmitted by optic nerves to the Central nervous system (CNS) • CNS target: Lateral Geniculate Nuclei(LGN) of the thalamus • Proceeding from the LGN, optic radiation contracts the primary visual cortex (area 17 or V1, or Striate Cortex)

5. Primary Visual Cortex (V1) • A thin sheet of neurons • The largest cortical area in primates, containing some 200 million cells • Complex visual processing occurs here • Well known area in brain

6. What are cells in V1 doing? • Hubel and Wiesel’s outstanding research and other studies revealed: cells in the primary visual cortex have receptive fields which are restricted to small regions of space and highly structured. • These cells are highly orientation and spatial frequency selectively They are working as linear filters to decompose the visual image

7. Spatial Frequency Theory • Atomistic assumption: the representation of any image, no matter how complex, is an assemblage of many primitive spatial “atoms”------ spatially extended patterns called sinusoidal gratings

8. Sine Grating • Luminance profile is described as sine wave • Important parameters: • Amplitude contrast • Spatial frequency (cycle/degree) • Orientation • phase

9. Image decompose • Any 2D image can be decomposed into a unique set of sine gratings • According to Fourier’s Theorem, a complex pattern (like a face) is just the sum of a particular set of sine gratings

10. A Cell is a filter • Sachs, Nachmias (1971), H.R.Wilson (1979,1993) etc. • Each cell is trying to decompose the incoming information to particular component • Studies show the stimulus is processed by mechanisms selective for both spatial frequency and orientation. The response of each mechanism is produced by linear filtering by its receptive field

11. Spatial frequency selective V1 cells respond to narrow ranges of stimulus spatial frequency

12. Orientation Selective • Similarly, V1 cells respond to narrow ranges of stimulus Orientation • Preferred orientation: only excitation=big response • Orthogonal orientation: excitation and inhibition cancel out= no response

13. V1 functional architecture • Receptive field: linear, weighted sum of stimulus intensities. • This model is attractive because it allows for a complete description. Measure the linear weighting function for a given cell, you can predict that cell’s response to any visual stimulus

14. In cats, the spatial frequency dimension is hypothesized to be orthogonal to the orientation dimension in a Cartesian structure, whereas in monkeys it is hypothesized to be radially organized in a polar structure How do we model both spatial frequency and orientation Selectivity Simultaneously??

15. Gabor Filter Model • A mathematical model used to describe receptive field properties of the cells in V1 • The receptive field structure is called a Gabor filter (or wavelet) in constructed by multiplying a global sinusoidal grating by a bell-shaped Gaussian envelope. • Describe the 2D information

16. Elliptic Gaussian A 2D elliptic Gaussian centered on the origin of a Cartesian coordinate system: Where, is variance in the x direction is variance in the y direction

17. Elliptic Gaussian cont. The elliptic Gaussian can be centered at any desired spatial location traslating offset x0 and y0 and aligned in an arbitrary orientation by a rotation of angle A Where,

18. Sine Plane Wave A 2D sinusoidal plane wave can be written Where, is spatial frequencies in the x direction is spatial frequencies in the y direction

19. Sine Wave Plane cont. • Since only the real signals (cosine term) can be observed in a experiment and we can choose some other origin for the modulation term,

20. Observable form of the 2D Gabor filter in the space domain • The 2D Gabor filter can now be written as the product of an elliptic Gaussian and a sinusoidal plane wave K --- scale factor (amplitude)

21. sample output of Gabor filters with different scales and different orientations 360, scale 2 00, scale 1 720, scale 3 1080, scale 4 1440, scale 5

22. The evaluation of Gabor filter • Jones and Palmer (1987) concluded that the Gabor function provides a useful and reasonably accurate description of cells in cat’s V1 • In Jones & Palmer´s experiment, simple cells responses were measured with a micro electrode. • refer to Judson and Palmer, 1987, The two-Dimensional spatial structure of simple receptive fields in cat striate cortex, Journal of Neurophysiology

23. An example of the resulting responses • left: experimental data in the space domain • middle: adapted Gabor filter • right: difference of the adapted filter values to the experimental data

24. Conclusion of the evaluation • The differences between the data and the fits were statistically indistinguishable from random error • Simple receptive fields in cat primary visual cortex are linear filters having the function form of 2D Gabor filters

25. An Example: Output after Gabor Filter 22.50 450 67.50 00 900 112.50 1350 157.50

26. Implications • Early visual processing (V1) is commonly modeled with a population of visual filters selective for different orientations and spatial frequencies. • Besides V1 model,we are studying more models related to computational modeling to gain insight into biological brain visual function

27. Implications • Thus, we study biologically-plausible brain models, and compare the predictions resulting from model simulations to empirical measurements from living systems. • The target: We are trying to build a quantitative computational model to predict the severity of visually-induced motion sickness

28. Thank You! Your comments and suggestions are appreciated

29. Linear Filters • It is a computation which takes one sequence of numbers (the input signal) and produces a new sequence of numbers (the filtered output signal). • A filter is linear means simply that • Scaling: the amplitude of the output is proportional to the amplitude of the input • Superposition:when two signals are added together and fed to the filter, the filter output is the same as if one had put each signal through the filter separately and then added the outputs

30. Visual cortical pathways after V1 • V1 forms an “internal representation” • Neurons in higher cortical areas make use of the internal representation to infer what is out there in the world. • 30 or more secondary visual areas after V1 in the occipital lobe and in parts of the parietal and temporal lobes, for example, • MT: visual motion processing area • Handle where/what, motion processing involved in motion analysis

31. Data Fit algorithm • In the space domain, it was found 2D Gabor filters that fit the 2D spatial response profile of each simple cell in the least squared error sense (with a simplex algorithm), and it was shown that the residual error is devoid of spatial structure and statistically indistinguishable from random error.

32. Primary Visual Cortex • Hubel and Wiesel (1959) revealed: • Simple cells: their response to complex stimuli can be predicted from their response to individual spots of light. A simple cell’s receptive field can therefore be mapped just by determining it’s response to a small spot of light at each position on the retina • Complex cells:integrating the response of many simple cells • Hypercomplex cells:end-stopped simple or complex cells