3-D Computer Vision CSc 83029

1. 3-D Computer VisionCSc 83029 Photometric Stereo & Shape from Shading 3-D Computer Vision CSc83029 / Ioannis Stamos

2. Photometric Stereo & Shape from Shading • Technique for recovering 3-D shape information from image intensity (brightness) • We will discuss: • Reflectance maps. • Photometric stereo. • Shape from shading. 3-D Computer Vision CSc83029 / Ioannis Stamos

3. Radiometry and Reflectance Image irradiance Brightness falloff 1 / F-number of lens Scene radiance We assume (calibration is needed) that: 3-D Computer Vision CSc83029 / Ioannis Stamos

4. Lambertian Reflectance Model I(x,y) p θi v s n P A Lambertian sphere k : Source brightness ρ’: Surface albedo (reflectance) ρ : Effective albedo (absorbs source brightness) or: : REFLECTANCE MAP 3-D Computer Vision CSc83029 / Ioannis Stamos

5. Lambertian Reflectance Model I(x,y) p θi v s n P A Lambertian sphere : REFLECTANCE MAP Relates Image Irradiance E to surface orientation for given source direction and surface reflectance 3-D Computer Vision CSc83029 / Ioannis Stamos

6. Representation of surface normal n Unit vector ry Z Z=Z(x,y) rx y Appendix A.5 (Trucco) x 3-D Computer Vision CSc83029 / Ioannis Stamos

7. Gradient Space (p,q) Source z 1.0 q y p x Surface normal can be represented by a point (p,q) on a plane! Source direction can be represented by a point (ps,qs)! **We want to calculate (p,q) from intensity I(x,y) 3-D Computer Vision CSc83029 / Ioannis Stamos

8. Gradient Space (p,q) Source z 1.0 q y p x Surface normal Source direction Assumption: SOURCE DIR. IS CONSTANT FOR ENTIRE SCENE. 3-D Computer Vision CSc83029 / Ioannis Stamos

9. Reflectance Map (Lambertian) OR: Source z ISO-BRIGHTNESS CONTOUR q Constant θi p y x 3-D Computer Vision CSc83029 / Ioannis Stamos

10. Reflectance Map (Lambertian) ISO-BRIGHTNESS CONTOURS NOTE: R(p,q) is maximum when (p,q)=(ps,qs) 3-D Computer Vision CSc83029 / Ioannis Stamos

11. Reflectance Map (Lambertian) Examples. Where is the source with respect to the sphere? 3-D Computer Vision CSc83029 / Ioannis Stamos

12. Reflectance Map (Glossy Surfaces) 3-D Computer Vision CSc83029 / Ioannis Stamos

13. Shape from Shading ISO-BRIGHTNESS CONTOURS PROBLEM: Given 1) source direction 2) surface reflectance (ρ) 3) one intensity image I(x,y) Can we find unique surface orientation (p,q)?

14. Two reflectance maps? 3-D Computer Vision CSc83029 / Ioannis Stamos

15. Two reflectance maps? Intersections: 2 solutions for p and q. What if we don’t know the albedo? 3-D Computer Vision CSc83029 / Ioannis Stamos

16. Photometric Stereo Use multiple light sources to resolve ambiguity In surface orientation. Note: Scene does not move – Correspondence between points in different images is easy! Notation: Direction of source i: or Image intensity produced by source i: 3-D Computer Vision CSc83029 / Ioannis Stamos

17. Lambertian Surfaces (special case) Use THREE sources in directions Image Intensities measured at point (x,y): orientation albedo 3-D Computer Vision CSc83029 / Ioannis Stamos

18. Photometric Stereo: RESULT INPUT albedo orientation

19. From Surface Orientations to Shape Integrate needle map 3-D Computer Vision CSc83029 / Ioannis Stamos

20. Calibration Objects & Look-up Tables Calibration: Useful when Reflectance Map Equations are uknown. Use Calibration sphere of known size and same reflectance as scene objects. Each point on the sphere has a unique known orientation. 3-D Computer Vision CSc83029 / Ioannis Stamos

21. Calibration Objects & Look-up Tables Illuminate the sphere with one source at a time and obtain an image. Each surface point with orientation (p,q) produces three images (I1, I2, I3) Generate a LOOK-UP TABLE (I1, I2, I3) -> (p,q) I3 (p,q) ARRAY I1 I2 For an object of uknown shape but same reflectance, obtain 3 images using same sources. For each image point use LUT to map (I1, I2, I3) -> (p,q)

22. Photometric Stereo: Remarks • Reflectance & illumination must be known a-priori. • Local Method. • Major Assumption: No interreflections. Concave surfaces exhibit interreflections. 3-D Computer Vision CSc83029 / Ioannis Stamos

23. Shape from Shading • Can we recover shape from a Single Image? 3-D Computer Vision CSc83029 / Ioannis Stamos

24. Human Perception of Shape from Shading We assume light source is above us. Ramachandran 88 3-D Computer Vision CSc83029 / Ioannis Stamos

25. Human Perception of Shape from Shading We assume light source is above us. Ramachandran 88 3-D Computer Vision CSc83029 / Ioannis Stamos

26. Human Perception of Shape from Shading Surface boundaries have strong influence on perceived shape 3-D Computer Vision CSc83029 / Ioannis Stamos

27. Finding the Illumination Direction Assumption: The surface normals of the scene are distributed uniformly in 3-D space Illumination direction Mean intensity Mean square intensity Mean image gradient

28. Shape from Shading Smoothness constraint 3-D Computer Vision CSc83029 / Ioannis Stamos

29. Shape from Shading Calculus of Variations -> Discrete Case: Is the average of the 4 neighboring values Iterative solution 3-D Computer Vision CSc83029 / Ioannis Stamos

30. Shape from Shading We know the normal at the contour. This provides boundary conditions. OCCLUDING BOUNDARY 3-D Computer Vision CSc83029 / Ioannis Stamos