Download
shape from x n.
Skip this Video
Loading SlideShow in 5 Seconds..
Shape from X PowerPoint Presentation
Download Presentation
Shape from X

Shape from X

194 Vues Download Presentation
Télécharger la présentation

Shape from X

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. CS 636 Computer Vision Shape from X Nathan Jacobs Slides by Lazebnik

  2. Cues to 3D • Motion • Shading • Defocus • (moving) Shadows and Specularities • Texture

  3. Photometric Stereo Merle Norman Cosmetics, Los Angeles • Readings • R. Woodham, Photometric Method for Determining Surface Orientation from Multiple Images. Optical Engineering 19(1)139-144 (1980). (PDF) Slides by Seitz

  4. image intensity of P Diffuse reflection • Simplifying assumptions • I = Re: camera response function f is the identity function: • can always achieve this in practice by solving for f and applying f -1 to each pixel in the image • Ri = 1: light source intensity is 1 • can achieve this by dividing each pixel in the image by Ri

  5. Shape from shading • Suppose • You can directly measure angle between normal and light source • Not quite enough information to compute surface shape • But can be if you add some additional info, for example • assume a few of the normals are known (e.g., along silhouette) • constraints on neighboring normals—“integrability” • smoothness • Hard to get it to work well in practice • plus, how many real objects have constant albedo?

  6. L3 L2 L1 Photometric stereo N V Can write this as a matrix equation:

  7. Solving the equations

  8. Least squares solution: Solve for N, kd as before What’s the size of LLT? More than three lights • Get better results by using more lights

  9. Computing light source directions • Trick: place a chrome sphere in the scene • the location of the highlight tells you where the light source is

  10. V2 V1 N Depth from normals • Get a similar equation for V2 • Each normal gives us two linear constraints on z • compute z values by solving a matrix equation orthographic projection

  11. Results… Input (1 of 12) Normals Normals Shadedrendering Texturedrendering

  12. Results… from Athos Georghiades http://cvc.yale.edu/people/Athos.html

  13. Limitations • Big problems • doesn’t work for shiny things, semi-translucent things • shadows, inter-reflections • Smaller problems • camera and lights have to be distant • calibration requirements • measure light source directions, intensities • camera response function • Newer work addresses some of these issues • Some pointers for further reading: • Zickler, Belhumeur, and Kriegman, "Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction." IJCV, Vol. 49 No. 2/3, pp 215-227. • Hertzmann & Seitz, “Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs.” IEEE Trans. PAMI 2005

  14. Depth from (De)Focus

  15. Many Methods • Try many focal depths, pick the sharpest for each pixel • Treat circle of confusion like a local blur… use a deconvolution method. • (more on that)

  16. Image and Depth from a Conventional Camera with a Coded Aperture Anat Levin, Rob Fergus, Frédo Durand, William Freeman MIT CSAIL

  17. Output #1: Depth map Single input image:

  18. Output #1: Depth map Single input image: Output #2: All-focused image

  19. Lens and defocus Image of a point light source Lens’ aperture Camera sensor Lens Point spread function Focal plane

  20. Lens and defocus Image of a defocused point light source Lens’ aperture Camera sensor Object Lens Point spread function Focal plane

  21. Lens and defocus Image of a defocused point light source Lens’ aperture Camera sensor Object Lens Point spread function Focal plane

  22. Lens and defocus Image of a defocused point light source Lens’ aperture Camera sensor Object Lens Point spread function Focal plane

  23. Lens and defocus Image of a defocused point light source Lens’ aperture Camera sensor Object Lens Point spread function Focal plane

  24. Out of focus In focus Depth and defocus Depth from defocus: Infer depth by analyzing local scale of defocus blur ill posed

  25. Challenges • Hard to discriminate a smooth scene from defocus blur • Hard to undo defocus blur ? Out of focus Input Ringing with conventional deblurring algorithm

  26. Defocus as local convolution Calibrated blur kernels at different depths Input defocused image

  27. Defocus as local convolution Local sub-window Calibrated blur kernels at depth Sharp sub-window Input defocused image Depth k=1: Depth k=2: Depth k=3:

  28. Overview Try deconvolving local input windows with different scaled filters: ? Larger scale ? Correct scale ? Smaller scale Somehow: select best scale.

  29. ? Larger scale ? Correct scale ? Smaller scale Challenges • Hard to deconvolve even when kernel is known • Hard to deconvolve even when kernel is known Input Ringing with the traditional Richardson-Lucy deconvolution algorithm • Hard to identify correct scale:

  30. Deconvolution is ill posed ? =

  31. Deconvolution is ill posed Solution 1: = ? Solution 2: = ?

  32. Idea 1: Natural images prior What makes images special? Natural Unnatural Image gradient Natural images have sparse gradients put a penalty on gradients

  33. Deconvolution with prior Convolution error Derivatives prior 2 _ + ? Low Equal convolution error 2 _ + ? High

  34. “spread” gradients “localizes” gradients Richardson-Lucy Gaussian prior Sparse prior Comparing deconvolution algorithms (Non blind) deconvolution code available online: http://groups.csail.mit.edu/graphics/CodedAperture/ Input

  35. Comparing deconvolution algorithms (Non blind) deconvolution code available online: http://groups.csail.mit.edu/graphics/CodedAperture/ Input “spread” gradients “localizes” gradients Richardson-Lucy Gaussian prior Sparse prior

  36. Recall: Overview Try deconvolving local input windows with different scaled filters: ? Larger scale ? Correct scale ? Smaller scale Somehow: select best scale. Challenge: smaller scale not so different than correct

  37. Depth from Diffusion Changyin Zhou Oliver Cossairt Shree Nayar Columbia University Supported by ONR

  38. Optical Diffuser

  39. Optical Diffuser ~ 10 micron Micrograph of a Holographic Diffuser (RPC Photonics) [Gray, 1978] [Chang et al., 2006] [Garcia-Guerrero et al. 2007]

  40. Diffuser to preview the image (B&H) Diffusers for illumination (B&H) Diffusers to soften the image Diffusers as Accessories

  41. Object Object Diffuser Camera Diffuser Camera Diffusion Encodes Depth The amount of diffusion varies with depth.

  42. Geometry of Diffusion: A Pinhole Camera Object P Miss Pinhole Q Sensor

  43. θ Geometry of Diffusion: A Pinhole Camera Object P Pinhole Sensor Diffuser

  44. A θ B θ Geometry of Diffusion: A Pinhole Camera Object P Pinhole Sensor Diffuser

  45. Geometry of Diffusion: A Pinhole Camera Diffusion Law: Object A P θ B θ Pinhole O 2r V Z U Sensor Diffuser Object

  46. Geometry of Diffusion: A Pinhole Camera Diffusion Size and Depth: Object A P θ B θ Pinhole O 2r V Z U Sensor Diffuser Object

  47. Geometry of Diffusion: A Pinhole Camera Diffuser as a proxy object Diffusion Size and Depth: Z P Pinhole O 2r V U Sensor Diffuser Object

  48. Captured Image Diffusion PSF Latent clear image Diffusion Size Diffusion as Convolution: A Pinhole Camera Assume field angle and depth are constant for small image patches, we have:

  49. Experiments Canon 20D + 50mm Lens Five playing cards, 0.29mm thick each Luminit Diffuser (20o)

  50. Experiments Captured WITHOUT a Diffuser Captured WITH a Diffuser