CS 563 Advanced Topics in Computer Graphics Introduction To IBR

1. CS 563 Advanced Topics in Computer GraphicsIntroduction To IBR By Cliff Lindsay Slide Show ’99 Siggraph[6]

2. What Is IBR? • IBR: • Multidisciplinary field that includes computer vision and graphics • Techniques that replace and/or augment polygon models • Primary data pre-rendered images and photographs as input The Rendering Spectrum [3]

3. Where Did IBR Come From? • Photo-realism • Modeling ability has been stifled by rendering advancements • Availability of inexpensive digital image acquisition hardware • Recent graphics accelerators trends • Necessity to render object that can’t be rendered using polygons

4. Tenants and Common Techniques of IBR • Rendering time is decoupled from scene complexity • Images are used as input • Exploits coherence • Pre-calculation of scene data/images

5. Computer Vision  Computer Graphics = IBR. Combining CG and Computer Vision • Vision Technology Lacks robust Algorithms • The Graphics Industry Needs better modelling Siggraph ’99 Course on Image-based Rendering[6]

6. How Does IBR Compare to Traditional Rendering? • Image Warping Vs. Matrix Transformations • Perspective Division Vs. Projective Normalization • What the !@#$ Is A Splat Kernel? [1] Image from Leonard McMillan

7. Approaches • 2D Approaches: • Texture Mapping • Sprite, Billboards, and Impostors • Image Layering • 3D approaches: • LDI (2.5D) • View Interpolation & Morphing • Mosaics • 4D approaches: • The Lumigraph • Light Fields

8. Texturing Mapping • Texture Space (u,v)  3D Object Space (x0, y0, z0)  Screen Space (x, y) [5] • Texture mapping has close ties to Image Warping • Wide Industry Support (hardware and Software) • Filtering

9. Sprites, Billboards, and Impostors (Oh my!) • Sprites: • Pure 2D image • No warping, or projection (like mouse cursor) • Billboards: • Sprite applied to a polygon • Alpha channel usually employed • Uses texture mapping for acceleration • Impostors: • Billboards created on the fly. • Can represent complex models • Error metric associated w/ changed views

10. Billboards • Billboards: • Oriented toward viewer • Matrix transformations (classical pipeline) • Special effects (lens flares, laser/light bursts, etc) • Hard to render objects (clouds, fire, smoke)

11. Impostors • Impostor Techniques: • Error Angle • Off Screen Rendering • Polygon Texturing • Texture resolution need not exceed screen resolution • texres = screenres * objsize/(2 * distance * tan(fov/2))

12. Billboard Example

13. Billboard Example

14. Lumigraph/Light Fields 0 Plenoptic function • An image is a collection of radiance values a long a ray. • Radiance value for all possible rays = Plenoptic function • 4D (for our purpose) [7] [7]

15. Lumigraph/Light Fields 1 • Represent an object by it’s extents • Each point on a cube has multiple rays eminating. • Each wall has 2 planes (12 planes make a cube)

16. Lumigraph/Light Fields 2 • You parameterize a ray using the 2 planes • L(s, t, u, v) = radiance for a ray • Ray – plane intersection make it easy and fast

17. Lumigraph/Light Fields 3 • Sample of the objects on the plane are not continuos • Gaps are Created [10]

18. Lumigraph/Light Fields 4 • Continuos luminance is a linear sum • B – basis function for which we can calculate at grid points • If we use a constant value, the coefficient take on the values of the grid points [10]

19. Lumigraph/Light Fields 5 [10]

20. Lumigraph/Light Fields 6 Example Rendering [10]

21. View Interpolation View Morphing - more to come next presentation! Reference Image 1 Reference Image 2 Corresponding Pixels Morph maps Based on diagrams from Watt[8]

22. View Morphing View Morphing - more to come next presentation! View Morphing[9]

23. View Morphing View Morphing - more to come next presentation! 3 1 1 2 2 View Morphing[9]

24. Recent Developments & The Future of IBR • Surface Light fields • High Dynamic Range Radiance Maps • View-dependent texture-mapping (VDTM) • IBO (Image Based Objects)

25. Conclusion • Rendering time is decoupled from scene complexity • Images are used as input • Pre-calculation of scene data/images

26. Additional Resources • http://citeseer.nj.nec.com/cs - NEC Digital Library • http://www.siggraph.org • http://www.debevec.org/ (View Morphing, High Dynamic Range Radiance Maps, Projective Texture-Mapping) • http://www-2.cs.cmu.edu/%7Eph/869/www/misc.html (a cool site with a bunch of IBR links) • http://www.peter-oel.de/ibmr-focus/ (Another cool site)

27. References [1] McMillan, Leonard, “An Image-Based Approach to Three-Dimensional Computer Graphics ”, , http://graphics.laces.mitt.edu/~mcmillan/IBRwork/defense23.html, date unknown, Cited slide #6. [2]McMillan, Leonard, Gortler, Steven, “Applications of Computer Vision to Computer Graphics”, ACM Siggraph, Vol. 33 no. 4, Nov. 99 [3] Akenine-Moller, Tomas, Haines, Eric, “Real-Time Rendering, 2nd Edition”, A K Peters, 2002 [4] Watt, Alan, “3D Computer Games”, Addison-Wesley Pub Co, Volume 1, 2nd edition, 1999 [5] Heckbert, Paul S., “Survey of Texture Mapping,” IEEE Computer Graphics & Applications, Cited slide #10, November 1986, [6] Cohen, Michael, “Course on Image-based, Modeling, Rendering, and Lighting”, Siggraph ‘99 [7] Mcmillian, Leonard, Bishop, Gary, “Plenoptic Modeling: An Image-Based Rendering System”, Proceedings of SIGGRAPH 95, (Los Angeles, CA August 6-11, 1995), pp. 39-46 [8] Watt, Alan, “3D Computer Graphics”, Addison-Wesley Pub Co, 3nd edition (), 2000 [9] Chen, S.E., Williams L., “View Interpolation for Image Synthesis”, ACM Siggraph ’95 [10]Gortler, S, Cohen, M, Girzesczuk, R, Szeliski, R, “The Lumigraph”, ACM Siggraph, 1996