Optics and Depth

1. Optics and Depth A Presentation on: By Srividya Varanasi Graduate Student Electrical Engineering Fall 2005 Computer Vision 6358

2. Contents 1.Optics a. Pin hole camera model b. Geometry of perspective projection c. Lens equation d. Image resolution e. Depth of field 2.Calculating Depth of Field a. Stereo imaging b. Stereo matching c. Shape from X 3.Range Imaging 4.Active vision Computer Vision 6358

3. What is optics? The branch of physics that deals with light and vision . How is optics related to Computer vision? Computer vision deals with images, hence we need to study the basics of image generation and propagation. Computer Vision 6358

4. Some definitions: Perspective Projection: The mapping of three dimensions onto two, is called perspective projection. Focal Length: The distance from the lens to the point of intersection of all parallel rays is the principal focal length f of the lens. Computer Vision 6358

5. Figure 1: Computer Vision 6358

6. Pin Hole camera Model: • To understand how vision might be modeled computationally and replicated on a computer, we need to understand the image acquisition process. The role of the camera in machine vision is analogous to that of the eye in biological systems. • Machine Vision relies on the pinhole camera model Computer Vision 6358

7. The pinhole camera is the simplest, and the ideal, model of camera function. • Pin hole model of a camera, models the geometry of perspective projection. • It has an infinitesimally small hole through which light enters before forming an inverted image on the camera surface facing the hole. To simplify things, we usually model a pinhole camera by placing the image plane between the focal point of the camera and the object, so that the image is not inverted. Computer Vision 6358

8. Drawbacks of the pin hole model: • It omits the effect of depth of field, i.e. only points within a certain depth range are in focus on the image plane. • Objects may not be seen clearly if they are too near or too far away. Computer Vision 6358

9. Geometry of Perspective Projection: Using principles of geometry it can be shown that if we denote the distance of the image plane to the centre of projection by f, then the image coordinates (xi,yi) are related to the object coordinates (xo,yo,zo) by Computer Vision 6358

10. Lens Equation: The lens equation relates the image distance ‘V’ and the Object distance ‘U’ to the focal length of the lens. It is given by: Computer Vision 6358

11. Image Resolution: The number of pixels in a digital photo is commonly referred to as its image resolution. It is a measurement of the quality of a video image. The greater the resolution, the better is the quality. Computer Vision 6358

12. Depth of Field • The amount of distance between the nearest and farthest objects that appear in acceptably sharp focus in a photograph. • Depth of field depends on the size of the aperture, the distance of the camera from the object, and the focal length of the lens. • The smaller the aperture, the greater the depth of field. If the camera focuses on a distant subject, the depth of field will be greater than if it was focused on a near subject. Computer Vision 6358

13. Image 1 Image 2 Computer Vision 6358

14. Different methods for calculating Depth Information Calculating the distance of various points in the scene relative to the position of the camera is one of the important tasks for a computer vision system. We shall see the various techniques for calculating the depth information. Computer Vision 6358

15. 1.STEREO IMAGING • It is a common method for extracting depth information from intensity images. • In this method a pair of images are acquired using two cameras displaced from each other by a known distance. • As a alternative method two or more images taken from a moving camera can also be used. Computer Vision 6358

16. Experimental setup for stereo imaging: The simplest model is two identical cameras separated only in the x direction by a base line distance ‘d’. Computer Vision 6358

17. Some Definitions: Epipolar Plane: The plane passing through the camera centers and the feature point in the scene. Epipolar line: The intersection of the epipolar plane with the image plane Computer Vision 6358

18. Contd…. Conjugate Pair: It is a pair of points in two different images that are the projections of the same point in the scene. Disparity: The displacement between the two features in the image plane. Computer Vision 6358

19. A scene point P is observed at points pland p r in the left and right image planes, respectively. Comparing similar triangles PMCl and plLCl, we get Computer Vision 6358

20. Similarly ,from the similar triangles PNCr and prRCr we get. Combing the two equations we get Thus depth at various scene points may be recovered by knowing the disparities of the corresponding image points. Computer Vision 6358

21. Cameras in Arbitrary position and orientation Computer Vision 6358

22. 2.Stereo Matching • The Stereo imaging technique assumes that we can identify conjugate pairs in the images. • Detecting conjugate pairs in stereo images ,is however a challenging problem known as the correspondence problem. Correspondence problem : For each point in the left image, find the corresponding point in the right image. To find conjugate pairs, it is required to find the similarity of the points. So this makes it necessary for the point to be matched to be distinct from its surroundings. Computer Vision 6358

23. The problem of finding matching pairs can be solved by identifying feature points. • Therefore in stereo matching, feature points are identified and their depths are calculated. • For other points depth is estimated using interpolation techniques. • The epipolar constraint significantly limits the search for conjugate pairs. Computer Vision 6358

24. What can form a set of matchable features? Edges and region features can be used as feature points. STEREO MATCHING Edge matching Region matching Computer Vision 6358

25. Edge Matching Basic ideas in Edge matching Algorithm: • Features are derived from left and right images by filtering the images ,and features are matched along epi polar lines • In this particular algorithm, the epipolar lines are along the image rows. • This algorithm uses edges detected by the first derivative of Gaussian, since these edges are more stable with respect to noise. Computer Vision 6358

26. Edge mapping Algorithm: 1. Filter each image in the stereo pair with Gaussian filters at four different filter widths, such that each filter is twice as wide as the next smaller filter. This can be done by repeated convolution with the smallest filter. 2. Compute the edge positions within the row. 3. Match edges at coarse resolutions by comparing their orientations and strengths. 4. Refine the disparity estimates by matching at finer scales Computer Vision 6358

27. Limitations of Edge Matching: • The value of computed depth is not meaningful along occluding edges where depth is not well defined. • Along these edges the value of the depth is anywhere from the foreground object’s occluding edge to the background scene point. Computer Vision 6358

28. Region Correlation Detection of interesting points in Regions: Features are identified in areas of image with high variance. 1. The Variance along different directions computed using all pixels in a window centered about a point are good measures of distinctness of point along different direction. The directional variances are given by: Computer Vision 6358

29. Contd… • To avoid multiple neighboring points as being detected as feature points, Feature points are chosen where the interest measure is a local maxima. • Once features are identifies in both the images, they can be matched using a number of different methods. • A Simple technique involves computing the correlation between a small window of pixels centered around the feature point in both the images. The feature with the highest correlation is considered as a match. Computer Vision 6358

30. More methods to obtain Depth information Computer Vision 6358

31. Shape from X • The Shape from X techniques estimate local surface orientation rather than absolute depth at each point. • If the actual depth at one point is known, then the depth at other points on the same object can be computed by integrating the local surface orientation. Hence these methods are called indirect methods. Some of the methods : a. Photometric stereo d. Shape from Focus b. Shape from Shading e. Shape from motion c. Shape from Texture Computer Vision 6358

32. a. Photometric Stereo • Three images of the same scene are obtained using light sources from three different directions. Both camera and the object have to be stationary during image acquisition • By knowing the surface reflectance properties of the objects in the scene, the local surface orientation at points illuminated by all three light sources can be computed. Computer Vision 6358

33. Shape from Shading: • This method exploits the changes in the image intensity (Shading) to recover surface shape information. • This is done by calculating the orientation of scene surface corresponding to each point in the image. Computer Vision 6358

34. Shape from Texture: • Image plane variations in the texture properties such as density, size and orientation are cues that are exploited in shape from texture algorithms. Shape from Focus: • Due to finite depth of field of optical systems, only objects which are at a proper distance appear focused in the image whereas those at other depths are blurred in proportion to their difference. Algorithms exploit this blurring effect. Computer Vision 6358

35. Range Imaging • Cameras which measure the distance to every single point within the viewing angle and record it as a two dimensional function are called range imaging systems,and the resulting images are range images. • Two common principles used in range imaging are 1. Triangulation 2. Radar • Structured lighting systems, which are extensively used in machine vision, make use of principle of triangulation. • Imaging radar systems use either acoustic or laser range finders to compute depth map. Computer Vision 6358

36. Structured lighting: • Structured light is the projection of a light pattern (plane, grid, or more complex shape) at a known angle onto an object. This technique can be very useful for imaging and acquiring dimensional information. • The most often used light pattern is generated by fanning out a light beam into a sheet-of-light. When a sheet-of-light intersects with an object, a bright line of light can be seen on the surface of the object. By viewing this line of light from an angle, the observed distortions in the line can be translated into height variations Computer Vision 6358

37. Imaging Radar • A second method for range imaging is imaging radar. • The time difference or the phase difference between the transmitted and received electromagnetic pulse can be used to calculate the depth information. Computer Vision 6358

38. Active Vision • Active vision is mainly an intelligent data acquisition process controlled by the measured and calculated parameters and errors from the scene. • Active vision systems can be applied to various fields like robotics, sonification systems for visually impaired etc Computer Vision 6358