1 / 51

Computational Photography Light Field Rendering

Computational Photography Light Field Rendering. Jinxiang Chai. Image-based Modeling: Challenging Scenes. Why will they produce poor results? lack of discernible features occlusions difficult to capture high-level structure illumination changes specular surfaces. Some Solutions.

kolya
Télécharger la présentation

Computational Photography Light Field Rendering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Computational PhotographyLight Field Rendering Jinxiang Chai

  2. Image-based Modeling: Challenging Scenes • Why will they produce poor results? • lack of discernible features • occlusions • difficult to capture high-level structure • illumination changes • specular surfaces

  3. Some Solutions • Use priors to constrain the modeling space • Aid modeling process with minimal user interaction • Combine image-based modeling with other modeling approaches

  4. Videos • Morphable face (click here) • Image-based tree modeling (click here) • Video trace (click here) • 3D modeling by ortho-images (Click here)

  5. Spectrum of IBMR Model Panoroma Image-based rendering Image based modeling Images + Depth Geometry+ Images Camera + geometry Imagesuser input range scans Images Light field Geometry+ Materials Kinematics Dynamics Etc.

  6. Outline • Light field rendering [Levoy and Hanranhan SIG96] • 3D light field (concentric mosaics) [Shum and He Sig99]

  7. Plenoptic Function Can reconstruct every possible view, at every moment, from every position, at every wavelength Contains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality! An image is a 2D sample of plenoptic function! P(x,y,z,θ,φ,λ,t)

  8. Ray • Let’s not worry about time and color: • 5D • 3D position • 2D direction P(x,y,z,q,f)

  9. How can we use this? Static Lighting No Change in Radiance Static object Camera

  10. How can we use this? Static Lighting No Change in Radiance Static object Camera

  11. Ray Reuse • Infinite line • Assume light is constant (vacuum) • 4D • 2D direction • 2D position • non-dispersive medium Slide by Rick Szeliski and Michael Cohen

  12. Only need plenoptic surface

  13. Synthesizing novel views Assume we capture every ray in 3D space!

  14. Synthesizing novel views

  15. Light field / Lumigraph • Outside convex space • 4D Empty Stuff

  16. Light Field • How to represent rays? • How to capture rays? • How to use captured rays for rendering

  17. Light Field • How to represent rays? • How to capture rays? • How to use captured rays for rendering

  18. Light field - Organization • 2D position • 2D direction s q

  19. Light field - Organization 2D position 2D position 2 plane parameterization u s

  20. Light field - Organization s,t u,v s,t u,v 2D position 2D position 2 plane parameterization t v u s

  21. Light field - Organization Hold u,v constant Let s,t vary What do we get? u,v s,t

  22. Lumigraph - Organization Hold s,t constant Let u,v vary An image u,v s,t

  23. Lightfield / Lumigraph

  24. Light field/lumigraph - Capture • Idea 1 • Move camera carefully over u,v plane • Gantry • see Light field paper u,v s,t

  25. Stanford multi-camera array • 640 × 480 pixels ×30 fps × 128 cameras • synchronized timing • continuous streaming • flexible arrangement

  26. Light field/lumigraph - rendering • For each output pixel • determine s,t,u,v • either • use closest discrete RGB • interpolate near values s u

  27. s u Light field/lumigraph - rendering • Nearest • closest s • closest u • draw it • Blend 16 nearest • quadrilinear interpolation

  28. s u Ray interpolation Nearest neighbor Quadrilinear interpolation Linear interpolation in S-T

  29. Camera Plane Light Field/Lumigraph Rendering Light Field Capture Rendering Image Plane

  30. Light fields • Advantages: • No geometry needed • Simpler computation vs. traditional CG • Cost independent of scene complexity • Cost independent of material properties and other optical effects • Disadvantages: • Static geometry • Fixed lighting • High storage cost

  31. 3D plenoptic function • Image is 2D • Light field/lumigraph is 4D • What happens to 3D? • - 3D light field subset • - Concentric mosaic [Shum and He]

  32. 3D light field • One row of s,t plane • i.e., hold t constant s,t u,v

  33. 3D light field • One row of s,t plane • i.e., hold t constant • thus s,u,v • a “row of images” s u,v

  34. Concentric mosaics [Shum and He] Polar coordinate system: - hold r constant - thus (θ,u,v)

  35. Concentric mosaics Why concentric mosaic? - easy to capture - relatively small in storage size

  36. Concentric mosaics From above How to captured images?

  37. Concentric mosaics From above How to render a new image?

  38. Concentric mosaics From above How to render a new image? - for each ray, retrieval the closest captured rays

  39. Concentric mosaics From above How to render a new image? - for each ray, retrieval the closest captured rays

  40. Concentric mosaics From above How to render a new image? - for each ray, retrieval the closest captured rays

  41. Concentric mosaics From above object How to retrieval the closest rays?

  42. Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?

  43. Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?

  44. Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?

  45. Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?

  46. Concentric mosaics From above object (s,t) interpolation plane How to synthesize the color of rays?

  47. Concentric mosaics From above object (s,t) interpolation plane How to synthesize the color of rays? - bilinear interpolation

  48. Concentric mosaics From above

  49. Concentric mosaics From above

  50. Concentric mosaics • What are limitations?

More Related