Biologically Inspired Intelligent Systems

1. Biologically Inspired Intelligent Systems Lecture 06 Dr. Roger S. Gaborski

2. RECALL:Receptive Fields www.yorku.ca/ eye Roger S. Gaborski

3. Two Types of Retinal Ganglion Cell Receptive Fields On Center Off Surround Off Center On Surround Roger S. Gaborski

4. Roger S. Gaborski

5. Response of On Center to a Spot of Light • In darkness the ganglion cell fires at a ‘spontaneous’ rate • When RF is stimulated with a small diameter light spot the cell increases its firing rate – this continues to increase until the light reaches the edge of the on center region • When the spot is increased further and light strikes the inhibitory surround, the firing rate begins to decrease • It continues to decrease until the whole surround is covered with light Roger S. Gaborski

6. Simple Center Surround Receptive Field MODEL: output Ganglion Cell : Rod or Cone Positive Weight Negative Weight Ganglion Cell Roger S. Gaborski

7. Receptive Fields Different sizes, center on or off and overlap • - One photo-receptive cell (rod or cone) may be a member • of several receptive fields • Receptive fields are modeled by Difference of Gaussians • The output of the ganglion cells form the optic nerve Roger S. Gaborski

8. SUMMARIZE: Retina - Receptive Field Model • Light travels through layers of the retina cells and strikes the cones and rods in the receptive layer • Spatially local collections of rods or cones form receptive fields • The receptive field of a neuron can be defined as the area on the retina from which the activity of a neuron can be influenced by the light on the retina area Roger S. Gaborski

9. Visual Cortex • Groups of neurons process information about: • Form of objects • Contrast of objects • Location of objects • Movement of objects • Color of objects Roger S. Gaborski

10. Visual Cortex Cells Respond to: • Lines or edges with certain orientation or size • Angles or corners • Movement in one direction, but not another direction The Visual Cortex is divided into a number of regions, V1, V2, V3, V4 and MT (also called V5) Roger S. Gaborski

11. Directional Receptive Fields Vertical Receptive Field Overlapping, various orientations Roger S. Gaborski

12. Wide Variety of Directional RFs Roger S. Gaborski

13. Response of Simple V1 Cells http://www.biols.susx.ac.uk/home/George_Mather Light bar Stimulus on/off Cell response Cell responds best to a bright bar region surrounded by dark regions Roger S. Gaborski

14. Orientation Sensitivity You can quantify a neuron’s response in terms of its firing rate, the number of action potentials that occur per unit of time The response of the cell depends on the location and orientation Of the stimulus pattern Roger S. Gaborski

15. Different Types of Complex V1 Cells A B C D Types A and B are active as long as the stimulus is somewhere on the Receptive field Types C and D are sensitive to the direction of motion of the stimulus Roger S. Gaborski

16. Directional Receptive Fieldsare Modeled with Gabor FunctionsCosine Grating * 2D Gaussian Roger S. Gaborski

17. Edge Direction and Gradients Edge 90 Degrees 0 1.0 Gradient: 0 degrees Roger S. Gaborski

18. Cosine Grating and Slice Edge Direction 90 degrees 0 Degrees Cosine Grating Slice Indicated by Blue Bar Roger S. Gaborski

19. 45 Degrees Cosine Grating Edge Direction 135 degrees Roger S. Gaborski

20. GratingOrientation – Gradient Orientation functionGabor_cos = MakeGrating2( orient, numOfSamples, numOfCycles) %parameters: 0,10,2 sd = 12; %Gradient Orientation orient = (orient*pi/180); %create grading step = 1/numOfSamples; [x,y] = meshgrid( -pi:step:pi, -pi:step:pi); ramp = (cos (orient) * x) - (sin(orient)*y); figure, imagesc(ramp), colormap(gray) im = sin(ramp*numOfCycles); im_cos = cos(ramp*numOfCycles); figure, imagesc(im_cos), colormap(gray), title('Grating') Roger S. Gaborski

21. Gaussian and Gabor %Generate Gabor [grat_cos, grat_sin ] = MakeGrating2(45,10,2); filtSize = min(size(grat_cos)); x = linspace(-1,1,filtSize)*filtSize/2; sd = 12; y = (1/sqrt(2*pi*sd)).*exp(-.5*((x/sd).^2)); filt = (y'*y);filt=filt./max(filt(:)); %Gaussian figure, imagesc(filt), title('filt') colormap(hot) Gabor_sin = grat_sin .* filt; figure, imagesc(Gabor_sin), axis square, colorbar title('Gabor\_sin') Gabor_cos = grat_cos .* filt; figure, imagesc( Gabor_cos), axis square title('Gabor\_cos') Gabor_cos45_63 = Gabor_cos; Gabor_cos45_31 = imresize(Gabor_cos,[31 31], 'bicubic'); Gabor_cos45_15 = imresize(Gabor_cos,[15 15], 'bicubic'); Gabor_cos45_7 = imresize(Gabor_cos,[7 7], 'bicubic'); Roger S. Gaborski

22. 90 Degree Gradient Roger S. Gaborski

23. 90 Degree Gradient Roger S. Gaborski

24. 90 Degree Gradient will strongly respond to bars at 0 degrees Roger S. Gaborski

25. Multiply 2D Grating by 2D Gaussian Roger S. Gaborski

26. Roger S. Gaborski

27. Roger S. Gaborski

28. Gabor Models of Directional Receptive Fields: Gradient Directions 135 degrees 90 degrees 45 degrees 0 degrees Roger S. Gaborski

29. Receptive Fields measure in the cat Roger S. Gaborski

30. HW#4 Roger S. Gaborski

31. Roger S. Gaborski

32. Roger S. Gaborski

33. Roger S. Gaborski

34. Pattern One Image Roger S. Gaborski

35. Roger S. Gaborski

36. Roger S. Gaborski

37. Roger S. Gaborski

38. Roger S. Gaborski

39. GOAL: Biologically Inspired Vision System • Extract low level features using biologically inspired feature detectors (receptive fields) • Implement a Focus of Attention (FOA) mechanism based on low level features • Modify the low level FOA with high level information • Extract complex features in regions of FOA • Perform classification on objects in FOA area Roger S. Gaborski

40. Low Level Feature Extraction -1 • Form Contrast Image by convolving gray level input image with a set of Difference of Gaussian filters which model center on and center off circular receptive fields in retina • For Example: • Sizes 8x8, 16x16 and 32x32pixels • Small object will respond strongly to 8x8 DoG, large objects will respond strongly to 32x32 DoG Roger S. Gaborski

41. Vision Model of Retinal Processing Contrast Images imConv8 imConv16 imConv32 Retina Model 8x8, 16x16 and 32x32 circular receptive fields Gray Level Image R.S.Gaborski

42. Low Level Feature Extraction -2 • Convolve Contrast Image*images with Gabor modeled directional receptive fields • 0, 45, 90 and 135 degrees • Size: 7x7, 15x15 and 31x31 • Rectify resulting image (absolute value) • Know as S1 Cells • NOTE: FOR HW#4 USING GRAYSCALE IMAGE, NOT CONTRAST IMAGE Roger S. Gaborski

43. Vision Model of Visual Cortex Modeling – Simple Cells S1 WHERE n = 0, 45, 90 and 135 degrees n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 15x15 Gabor Contrast Images imConv8 imConv16 imConv32 (Retina Model) Retina Model 8x8, 16x16 and 32x32 circular receptive fields Gray Level Image R.S.Gaborski

44. Summary:Simple Cell Directional Images • Three contrast images (Retinal Model) • DoG8, DoG16 and DoG32 • Four Orientation Receptive Fields (Simple Visual Cortex Processing) • 0, 45, 90 and 135 degrees • Three sizes for each RF • 7x7, 15x15 and 31x31 3x4x3 = 36 Simple Cell images R.S.Gaborski

45. Color Channels - 1 • Four color planes • Create red-green, blue-yellow color channels • For each color plane, apply receptive fields at different scales, 7x7, 15x15, 31x31, 63x63… • Combine color receptive fields output R.S.Gaborski

46. Color Channels - 2 R-G Receptive Field* Color Image Red Green B-Y Receptive Field* Blue Yellow * At multiple scales R.S.Gaborski

47. R.S.Gaborski

48. R-G RF Size 36 R.S.Gaborski

49. R-G RF Size 25 (smaller RF) R.S.Gaborski

50. R-G RF Size 25Inhibition – 1 iteration R.S.Gaborski