1 / 113

Image Based Rendering

This article explores the advancements in image-based rendering (IBR) techniques, highlighting tools that have been acquiring and displaying high-quality photographs for nearly a century. We delve into the work of Sergei Mikhailovich Prokudin-Gorskii, whose early 20th-century photographs, such as "A Settler's Family" and "Tea Factory in Chakva," showcase the evolution of imaging technology. The piece discusses the plenoptic function, lightfields, and lumigraph methodologies, along with construction and compression techniques critical for modern rendering.

gent
Télécharger la présentation

Image Based Rendering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Based Rendering an overview

  2. Photographs • We have tools that acquire and tools that display photographs at a convincing quality level

  3. Photographs • We have tools that acquire and tools that display photographs at a convincing quality level, for almost 100 years now

  4. Sergei Mikhailovich Prokudin-Gorskii. A Settler's Family, ca. 1907-1915.

  5. Sergei Mikhailovich Prokudin-Gorskii. Tea Factory in Chakva. Chinese Foreman Lau-Dzhen-Dzhau. ca. 1907-1915.

  6. Sergei Mikhailovich Prokudin-Gorskii. The Emir of Bukhara, 1911.

  7. RGB in early 1900’s

  8. Plenoptic function • Defines all the rays • through any point in space (x, y, z) • with any orientation (θ, φ) • over all wavelenghts (λ) • at any given moment in time (t)

  9. IBR summary Representation of plenoptic function implicit explicit texture mapping panoramas view morphing geometric model 3D image warping ray databases

  10. Lightfield – Lumigraph approach[Levoy96, Gortler96] • Take all photographs you will ever need to display • Model becomes database of rays • Rendering becomes database querying

  11. Overview • Introduction • Lightfield – Lumigraph • definition • construction • compression

  12. Overview • Introduction • Lightfield – Lumigraph • definition • construction • compression

  13. From 7D to 4D • Static scene, t constant • λ approximated with RGB • consider only convex hull of objects, so the origin of the ray does not matter

  14. 4D Lightfield / Lumigraph

  15. Discreet 4D Lightfield

  16. Lightfield: set of images with COPs on regular grid

  17. or Lightfield: set of images of a point seen at various angles

  18. Depth correction of rays

  19. Overview • Introduction • Lightfield – Lumigraph • definition • construction • compression

  20. Overview • Introduction • Lightfield – Lumigraph • definition • construction • compression

  21. Construction from dense set of photographs

  22. Construction from sparse set of photographs camera positions acquisition stage blue screening space carving

  23. Filling in gaps using pull-push algorithm • Pull phase • low res levels are created • gaps are shrunk • Push phase • gaps at high res levels are filled using low res levels

  24. Overview • Introduction • Lightfield – Lumigraph • definition • construction • compression

  25. Overview • Introduction • Lightfield – Lumigraph • definition • construction • compression

  26. Compression • Large size uncompressed: 1.125GB • 32x32 (s, t) x 256x256 (u, v) x 6 faces x 3 B • Compression • jpeg + mpeg (200:1 to 6MB) • or vector quantization + entropy encoding

  27. Vector Quantization (VQ) • Principle • codebook made of codewords • replace actual word with closest codeword • Implementation • training on representative set of words to derive best codebook • compression: replacing word with index to closest codeword • decompression: retrieve indexed codeword from codebook

  28. Lightfield compression using VQ

  29. View morphing

  30. Motivation – rendering from images • Given • left image • right image • Create intermediate images • simulates camera movement [Seitz96]

  31. Previous work • Panoramas ([Chen95], etc) • user can look in any direction at few given locations • Image-morphing ([Wolberg90], [Beier92], etc) • linearly interpolated intermediate positions of features • input: two images and correspondences • output: metamorphosis of one image into other as sequence of intermediate images

  32. Previous work limitations • Panoramas ([Chen95], etc.) • no camera translations allowed • Image morphing ([Wolberg90], [Beier92], etc.) • not shape-preserving • image morphing is also a morph of the object • to simulate rendering with morphing, the object should be rigid when camera moves

  33. Overview • Introduction • Image morphing • View morphing • image pre-warping • image morphing • image post-warping

  34. Overview • Introduction • Image morphing • View morphing • image pre-warping • image morphing • image post-warping

  35. Image morphing • Correspondences

  36. Image morphing • Correspondences

  37. Image morphing • Correspondences

  38. Image morphing • Correspondences

  39. Image morphing • Correspondences • Linear interpolation P0 Pk Pn frame 0 frame k frame n

  40. Image morphing • Image morphing • not shape preserving

  41. Early IBR research Soft watch at moment of first explosion – Salvador Dali 1954

  42. Overview • Introduction • Image morphing • View morphing • image pre-warping • image morphing • image post-warping

More Related