Biologically Inspired Intelligent Systems

1. Biologically Inspired Intelligent Systems Lecture 07 Dr. Roger S. Gaborski Roger Gaborski

2. Exam 2 - Presentations Roger Gaborski

3. Feature Maps • Intensity contrast (Receptive fields - HW#4) • Orientation (HW#5 - Thursday) • 0, 45, 90 and 135 degrees • Experiment with different non-linear functions for output of neuron: absolute value, squared function, threshold (values less than zero set to zero), etc • Center ON surround OFF • Center OFF surround ON • Process images with different types of scenes: small details, close up of faces, landscapes, cityscapes, etc. • Research current literature on receptive fields – should model be modified? • Analyze results • 10 minute presentation Roger Gaborski

4. Vision Model WHERE n = 0, 45, 90 and 135 degrees n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 15x15 Gabor Contrast Images imConv8 imConv16 imConv32 Retina Model 8x8, 16x16 and 32x32 circular receptive fields Image

5. Input Image

6. DoG8 Contrast Image (On center) Orientation: 7x7 pixel 0 degree RF

7. DoG8 Contrast Image (On center) Orientation: 15x15 pixel 0 degree RF

8. Dog8 Contrast Image (On center)Orientation: 31x31 pixel 0 degree RF

9. DoG16 Contrast Image (On center) Orientation: 7x7 pixel 0 degree RF

10. DoG16 Contrast Image (On center) Orientation: 15x15 pixel 0 degree RF

11. DoG16 Contrast Image (On center) Orientation: 31x31 pixel 0 degree RF

12. DoG16 Contrast Image (On center) Orientation: 31x31 pixel 45 degree RF

13. Summary:Simple Cell Directional Images • Three contrast images • DoG8, DoG16 and DoG32 • Four Orientation Receptive Fields • 0, 45, 90 and 135 degrees • Three sizes for each RF • 7x7, 15x15 and 31x31 3x4x3 = 36 Simple Cell images

14. Roger Gaborski

15. Roger Gaborski

16. After image should be a properly colored flag Yellow leaves blue temporarily dominate Green leaves red temporarily dominate Roger Gaborski

17. Processing of Color Data • How is color coded in primates? • Three classes of cone photoreceptors • Cone photoreceptors function in bright light • Rods function in dim light resulting in a gray scale image • Arrangement of cone types seems to be random • Color opponent cells (cone signals brought together in opposition) • Red-green • Blue-yellow Roger Gaborski

18. Randomly distributed cones in fovea Roger Gaborski

19. Color Photoreceptors - cones Roger Gaborski

20. Cone Spectrum:Short, Medium and Long Wavelengths RODS PEAK AT ABOUT 500 NM Wavelength – nanometer (nm) Roger Gaborski

21. Color vision • We can distinguish more than a million colors • How can we distinguish a million colors with only three different color cones? • Combination of the three color receptors (R,G,B) Roger Gaborski

22. Assume we had only red cone type SAME OUTPUTVALUE Roger Gaborski

23. 510 and 630 nm light • The output from the red cones is the same for both light wavelengths • With three types of cones a comparison of the outputs of each cone type can be made Roger Gaborski

24. Assume we had only red cone type X X X EACH CONE TYPE HAS A RESPONSE X X x Roger Gaborski

25. Receptive field On Center / Off Surround RF Cones type A Cones type B - + - - Roger Gaborski

26. Double Opponent Red-Green RF White overlap Red in center Green in surround | | | | | | | | | | | | | | | | | | | | | Center: (+R-G), Surround: (+G-R) Receptive field Response: [(+R-G)+(+G-R)]>0 Roger Gaborski

27. Red-Green RF Green in center Red in center Green in center Green in surround Red in surround | | | ||||||||||||||||| NO RESPONSE Roger Gaborski

28. Blue-Yellow RF Same operation as Red- Green Also, Yellow center and Blue surround receptive fields Roger Gaborski

29. Opponent Color Theory (EwaldHering) • B-Y Receptive fields (Y=R+G) • R-G Receptive fields • Black-White receptive fields (luminance) Roger Gaborski

30. Recall • Only cones exist in the fovea (no rods) • High density results in visual fine detail • Without cones, in dim light, no details – world is blurry • In bright light, rods bleach out resulting in ‘over exposed’ images Roger Gaborski

31. A Dog’s View of the World Roger Gaborski

32. A Dog’s View of the World • The dog’s visual system is dominated by rods and only have two types of cones (short, 430nm and long wavelengths, 555nm) • Dip in sensitivity at 480nm (cyan) • Rods and cones are more evenly distributed without a concentration of cones at the fovea • Dogs are near sighted. Compared to our 20-20 vision, dogs typically have 20-75 vision • Dog’s eyes are set at an angle. This results in a wider field of vision, but smaller overlap of what their eyes see, reducing effectiveness of binocular vision (depth perception) Roger Gaborski

33. A dog’s view http://drsophiayin.com/blog/entry/can-dogs-see-color-and-how-do-we-know Roger Gaborski

34. http://drsophiayin.com/blog/entry/can-dogs-see-color-and-how-do-we-knowhttp://drsophiayin.com/blog/entry/can-dogs-see-color-and-how-do-we-know Roger Gaborski

35. Most marine mammals only have one type of cone • Deep ocean fish only have rods. • Low light conditions • Physically larger rods • Maximum sensitivity at a shorter wavelength matching the light available at great ocean depths Roger Gaborski

36. What if you only have one cone, which color do you see? Roger Gaborski

37. The term color doesn’t make much sense if you only have one type of cone – • See grayscale • Variation in brightness Roger Gaborski

38. Owl Monkey • Rods and one cone with sensitivity at 543 nm (middle-long wavelength) • Mostly active at night, therefore rods are more important • Dolphin, whale and seal • Rod and one long wave cone (red) Roger Gaborski

39. Feature Maps • Intensity contrast • Orientation • 0, 45, 90 and 135 degrees • Color Information • Red-Green opponent color • Blue-Yellow opponent color Roger Gaborski

40. Implementation of Color Difference RFs in Matlaband Processing of Color Images Roger Gaborski

41. Color Channels (approximation) • Create broadly tuned color channels (do not use rgb from imread): R = r-(g+b)/2 G = g- (r+b)/2 B = b- (r+g)/2 Y = r+g – 2(|r-g| + b) (negative values set to zero) • Maximum response to pure hue the channel is tuned to (if a pixel contains both r and g it will have a smaller R response than if it only had r) • Zero response to white or black inputs Roger Gaborski

42. Potential Simple Red-Green RF Model Create Dog64 Red_Kernel: Green_Kernel R_ctr=filter red data with Red_Kernel G_sur=filter green data with Green_Kernel SEE NEXT SLIDE Form difference:I_R* R_ctr –I_G* G_sur RED GREEN GREEN Roger Gaborski

43. Double Color Opponent GENERAL IDEA: Center: (+R-G), Surround: (+G-R) Receptive field Response: [(+R-G)+(+G-R)]>0 Using Center Function (see next slide) Rc = Ctr convolved with R Gc = Ctr convolved with G Using surround Function (see next slide) Rs = Sur convolved with R Gs = Sur convolved with G RF Response =( (+Rc-Gc) + (+Gs-Rs) ) > 0 Roger Gaborski

44. Note Colorbar Center Surround Roger Gaborski

45. Color Image Roger Gaborski

46. R = r-(g+b)_/2 Roger Gaborski

47. G Roger Gaborski

48. B Roger Gaborski

49. Y Roger Gaborski

50. Gray Scale – “Lack of Interest” Roger Gaborski