1 / 35

Mobile Robotics: 6. Vision 1

Dr. Brian Mac Namee ( www.comp.dit.ie/bmacnamee ) Mobile Robotics: 6. Vision 1 Acknowledgments These notes are based (heavily) on those provided by the authors to accompany “Introduction to Autonomous Mobile Robots” by Roland Siegwart and Illah R. Nourbakhsh

Samuel
Télécharger la présentation

Mobile Robotics: 6. Vision 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dr. Brian Mac Namee (www.comp.dit.ie/bmacnamee) Mobile Robotics:6. Vision 1

  2. Acknowledgments • These notes are based (heavily) on those provided by the authors to accompany “Introduction to Autonomous Mobile Robots” by Roland Siegwart and Illah R. Nourbakhsh More information about the book is available at:http://autonomousmobilerobots.epfl.ch/ The book can be bought at:The MIT Press and Amazon.com

  3. Today’s Lecture • Why is vision hard? • Brief historical overview • From early cameras to digital cameras • Low-level robot vision • Camera as sensor • Color representation • Object detection

  4. Vision In General • Vision is our most powerful sense providing us with an enormous amount of information about our environment and enables us to interact intelligently with the environment • It is therefore not surprising that an enormous amount of effort has occurred to give machines a sense of vision • Vision is also our most complicated sense • Whilst we can reconstruct views with high resolution on photographic paper, understanding how the brain processes the information from our eyes is still in its infancy

  5. Vision In General (cont…) • When an image is recorded through a camera, a 3-D scene is projected onto a 2-D plane • In order to try and recover some “useful information” from the scene, usually edge detectors are used to find the contours of the objects • From these edges or edge fragments, much research time has to been spent attempting to produce fool proof algorithms which can provide all the necessary information required to reconstruct the 3-D scene which produced the 2-D image • The interpretation of 3-D scenes from 2-D images is not a trivial task

  6. Vision Is Hard! (Segmentation)

  7. Vision Is Hard! (Classification)

  8. Vision Is Hard! (Perspectives)

  9. Vision Is Hard! (Brightness Adaptation) For more great illusion examples take a look at: http://web.mit.edu/persci/gaz/

  10. Vision Is Hard! (Illusions) • Our visual systems play lots of interesting tricks on us

  11. Vision Is Hard! (Illusions)

  12. Vision Is Hard! (Illusions) • Stare at the cross in the middle of the image and think circles

  13. Photograph of camera obscura interior: Portmerion Village, North Wales Camera Obscura • Mo Ti, Chinese philosopher, 5th Century B.C. • Described linear light paths, pinhole image formation • Leonardo da Vinci (1452-1519) • Demonstrated camera obscura (lens added later) Frisius (1544)

  14. Toward Photography • People sought a way to “fix” the images at the back of the camera obscura • Pursued decades of experimentation with light-sensitive salts, acids, etc. • First photograph produced when?

  15. First Photograph • Joseph Nicéphore Niépce “View from the Window at Le Gras”, c. 1826 • Aluminum plate coated with light-sensitive material Joseph Nicéphore Niépce Harry Ransom Center Kodak (reproduction) More information on the first photograph is available at: http://www.hrc.utexas.edu/exhibitions/permanent/wfp/

  16. First Digital Cameras • Photoelectric effect (Hertz 1887; Einstein 1905) • Charge-coupled devices as storage (late 1960’s) • Light sensing, pixel row readout (early 1970’s) • First electronic CCD still- image camera (1975): • Fairchild CCD element  • Resolution: 100 x 100 b&w • Image capture time: 23 sec.,mostly writing cassette tape • Total weight: 8½ pounds Kodak, c. 1975

  17. Modern Digital Cameras • Today, fifty Euro buys a camera with: • 640 x 480 pixel resolution at 30Hz • 1280 x 960 still image resolution • 24-bit RGB pixels (8 bits per channel) • Automatic gain control, color balancing • On-chip lossy compression algorithms • Uncompressed images if desired • Integrated microphone, USB interface • Limitations • Narrow dynamic range • Narrow FOV, with fixed spatial resolution • No motion / active vision capabilities

  18. Vision-Based Sensors: Hardware • CCD (light-sensitive, discharging capacitors of 5 to 25 micron) • CMOS (Complementary Metal Oxide Semiconductor) technology

  19. What Is A Digital Image? • A digital image is a 2-D representation of a 3-D scene as a finite set of digital values, called picture elements or pixels

  20. 1 pixel What Is A Digital Image? (cont…) • Pixel values typically represent gray levels, colours, heights, opacities etc • Rememberdigitization implies that a digital image is an approximation of a real scene

  21. v height width IO u Image coordinates (pixels) Digital Image Contents • Why are pixels represented as “RGB”? • Is world made of red, green, and blue “stuff”? • … Answer requires a digression (or two)about human vision, cameras as sensors

  22. Visible Light Spectrum Solar (ECI, Oxford) Incandescent (Wikipedia) Freedman & Kaufmann, Universe

  23. Illumination spectrum Reflection spectrum IJVS Image As Measurement • What does eye/camera actually observe? • … the product of illumination spectrum • with absorption or reflection spectrum! = (at each image point) X

  24. Eye Anatomy • Spectrum incident on light-sensitive retina (View of R eye from above) Rods and cones Incident spectral distribution After Isaka (2004)

  25. Blind-Spot Experiment • Draw an image similar to that below on a piece of paper (the dot and cross are about 6 inches apart) • Close your right eye and focus on the cross with your left eye • Hold the image about 20 inches away from your face and move it slowly towards you • The dot should disappear!

  26. 4 mm Cone Sensitivities • Three cone types (S, M, and L) are roughly blue, green, and red sensors, respectively • Their peak sensitivities occur at ~430nm, ~560nm, and ~610nm for the “average” human Rods & cones, ~1.35 mm fromcenter of fovea Rods & cones, ~8 mm fromcenter of fovea Cone sensitivities as a function of wavelength

  27. IJVS Color Perception • The cones form a spectral “basis” for visible light; incident spectral distribution differentially excites S,M,L cones, leading to color vision = (at each cone site) X X

  28. (Vaytek) Origin Of RGB CCD Sensors • So, in a concrete sense, CCD chips are designed as RGB sensors in order to emulate the human visual system

  29. (0,0,1) – pure blue (1,1,1) - white (0,0,0) - black (hidden) (0,1,0) – pure green (1,0,0) pure red RGB Colour Model • Think of R, G, B as color orthobasis

  30. HSV Colour Model • More robust against illumination changes • Still must confront noise, specularity etc.

  31. v height width IO u Image coordinates (pixels) Object Detection • Suppose we want to detect an object (e.g., a colored ball) in the field of view • We simply need to identify the pixels of some desired colour in the image … right?

  32. Noise! Real-World Images Occluded light source Specularhighlights Complexsurfacegeometry (self- shadowing) Mixedpixels

  33. Summary • Vision is our most useful sense, but is also the most difficult to replicate • Digital cameras have evolved to be powerful, cheap and accurate • Artificial vision systems tend to be modelled after the human eye • How do we do it?

  34. Questions? • ?

More Related