Computer Vision

1. Computer Vision 一目了然 一目 了然 眼睛 明白 Eye Comprehension CV = I P + A I

2. Chapter 1: Cameras ○ Human Eye Field Of View (FOV) Width × Height = 160 deg × 135 deg

3. Eyeball • Camera

4. CCD camera CCD (Charge-Coupled Device): rectangular grid of electron collection site laid over a silicon wafer CCD Image plane:

5. Retina Retina is composed of photoreceptors Two types of photoreceptors: rods and cones

6. Rods are sensitive to intensity, motion Cones are sensitive to color, structure Three types of cones:

7. Color camera Color image plane Color image plane: successive rows or columns are made sensitive to R, G or B light using a filter that blocks the complementary light. Bayer pattern: a filter pattern of 2 by 2 blocks, each of which is formed by 2 G, 1 R, and 1 B receptors.

8. ○ Visual Sensors Animal eyes: birds, bats, snakes, fishes, insects (fly, bee, locust, grasshopper, cricket, cicada) Cameras: fish-eye, panoramic, omni, PTZ Imaging Devices : telescopes, microscopes

9. ○ Imaging Surfaces Planar, Spherical, Cylindrical ○ Signals Single value (gray images) A few values (color images) Many values (hyper-spectral images)

10. 1.1. Pinhole Cameras and Model

11. 2 mm ○ Big pinhole - Averaging rays blurs image Small pinhole - Diffraction effect blurs image 0.6 mm 1 mm 0.35 mm 0.15 mm 0.07 mm

12. Diffraction (light wavelength > hole size) ＊In general, images acquired by pinhole cameras are relatively dark because a very small set of rays from a particular point hits the screen ＊ Pinholes Lenses Lenses: gather light, sharp focus

13. 1.1.1. Perspective Projection Perspective Projection Equations

14. Property: • the apparent size of objects depends on • their distances from the pinhole

15. (2) The projections of two parallel lines lying in a plane converge on a horizontal line formed by the intersection of the image plane with the plane parallel to and passing through the pinhole. (Assignment 1)

16. Assume a coordinate system x-y-z and

20. (B) Algebraic method (1) Define a) Camera coordinate system b) Image coordinate system (2) Prove by the projective projection equations and the limit theory How about if the image plane and the floor plane are not perpendicular to each other?

21. 1.1.2. Affine Projection ○ Three Models: (a) Weak-perspective projection – when the scene relief is small relative to the average distance (Zo) from the camera

22. (b) Orthographic Projection– when the scene is far away from the camera, i.e., remote scene (c) Para-Perspective Projection (see Ch. 2)

23. 1.2. Camera with Lenses ○ Reasons for equipping lenses: a) gather light, b) sharp focus Pinhole Cameras Modern Cameras

24. ○ Snell’s law • The laws of geometric optics (i) Light travels in straight lines in homogeneous media. (ii) Reflection: (iii) Refraction:

25. Proof: Find a path of light traveling from A to B with the minimal time (Fermat’s Principle)

26. 1.2.1. Paraxial Geometric Optics -- Consider light rays close to the optical axis are all small.

27. Approximation: Substituting into Snell’s law, Paraxial refraction equation :

28. Taylor expansions Approximation: Substituting into

29. Paraxial refraction equation :

30. 1.2.2. Thin Lenses (a) Rays passing through O are not refracted; (b) Rays parallel to the optical axis are focused on the focal point F’ (c) Rays passing through the focal point F are refracted to parallel the optical axis

32. 1.2.3. Thick Lenses (real lenses) -- There is a thickness between the two spherical interface surfaces. .

33. ○ Terminologies (a)Field of view (FOV): the scene space that projects onto the image plane of the camera FOV = , where (b) Depth of field or Depth of focus (DOF): the range of distances within which objects are in acceptable focus.

34. ○Types of aberrations (A) Spatial aberration -- The rays from P striking the interface farther from the optical axis are focused closer to the interface -- The image of P in the image plane forms a circle of confusion (COC)

35. -- Longitudinal spherical aberration (LSA): The distance between P’ and the intersection of the optical axis with a ray issued from P and refracted by the lens -- Transverse spherical aberration (TSA): The distance between P’ and the intersection of the ray with plane Result in shape distortion

36. (B) Chromatic aberration -- Due to both (i) the index of refraction of a medium and (ii) the focal length of the lens depend on the wavelength of the incident light rays

37. ○Compound lenses: for minimizing aberrations ○ Vignetting effect: Light beams emanating from object points located off-axis are partially blocked by lenses behind the aperture

38. 1.4.2. Sensor Models ○ The number of electrons recorded at the site (r, c) of a CCD array where : irradiance : reflectance T : time S(r,c) : spatial domain of the cell : quantum efficiency (the number of electrons generated per unit of incident light energy)

39. ○ Imaging process: Current (I) Voltage Signal Digit CCD camera frame amplifier electronics grabber ○ The model for digital signal

40. where : gain factor : (shot noise) resulting from quantum effects in the photo-conversion process : (dark current) originating from thermal effect : (bias) introduced by the CCD electronics R : (read-out noise) due to the CCD amplifier Q :(quantization noise) resulting from the digitization process

41. ○ Other defects: Blooming -- a light illuminating a site is so bright that the charge stored at that site overflows into adjacent ones Charge transfer efficiency – a source of uncertainty

42. 1.5. Note