Rendering with Concentric Mosaics

1. Rendering with Concentric Mosaics Heung – Yeung Shum and Li – Wei He Presentation By: Jonathan A. Bockelman

2. Agenda • A general description of concentric mosaics • Rendering concentric mosaics • Capturing concentric mosaics • Some examples • Issues that still need to be resolved and future plans • A brief demo

3. Boo! Yay! Rendering Made Easy... Sort of • Problems with traditional rendering schemes • The appeal of image-based modeling and rendering • The plenoptic function

4. History of Plenoptic Functions

5. What is a Concentric Mosaic? • “A manifold mosaic” • A 3D plenoptic: radius, rotation angle, and vertical elevation • A 3D image built from a series of 360° slit images

6. Rendering a Novel View • Any point within the outermost circle can be the viewpoint • Rays tangent to the camera paths are used • Bilinear interpolation between neighboring mosaics can also be used

7. The Problem of Non-Planar Rays • Rays off the plane need to be approximated • Objects assumed to have an infinite depth • Vertical distortion is created

8. The Need for Depth Correction • Depth correction can fix the vertical distortion • 3 types of depth correction exist

9. Full Perspective Correction • Individual corrections are made for each pixel • Exact depths of objects are necessary • Hole-filling problems are a complication • Excellent results are seen in synthetic scenes

10. Weak Perspective Correction • Corrections are made for each vertical line • Estimated depths are calculated • Vertical distortions can occur

11. Constant Depth Approximation • A constant depth is used • Users can control the assumed depth • Vertical distortions are produced if the wrong depth is given

12. Consequences of a 3D function • Vertical parallax is not captured • Much smaller data sets are required • Users can move in a circular region

13. Synthetic Mosaics • 3D Studio Max can be used • Images are cut into slits • Depth values for each pixel can be found • Sampling is a bit tricky

14. How NOT to Do Real World Scenes • A series of single-slit cameras on a rotating beam • A single camera that can slide along a beam

15. The Lone Camera • A single off-centered camera sits on a rotary table • Regular images are taken • Multiple concentric mosaics can be recreated from one image

16. Ideal Solution • A single camera can produce distortion • A few tangential cameras along a beam can correct the problem

17. How the Pros Do It • An single ordinary digital video camera is used with a rotary table • The camera faces radially outward • 1351 frames are captured in 90 seconds • The system is incredibly simple and efficient

18. The Lobby Scene 3 concentric mosaics from a lobby scene

19. Occlusion is captured. Occlusion

20. Horizontal parallax is simulated quite well. Horizontal Parallax

21. Spectacular lighting effects are easy to create. Lighting and Glare

22. Constant Depth Correction Revisited • Aspect ratios are maintained at the chosen depth • Objects at other depths are distorted

23. Point vs. Bilinear Sampling • Point sampling is twice as fast, but image quality is lower • Bilinear sampling is slower, but images are much smoother

24. Compression • Since adjacent frames are very similar, a majority of the data can be compressed. • Vector quantization and entropy coding allow the 415Mb original video to be shrunk to 16Mb. • MPEG4 compression can reduce the data size to 640k, but blocky artifacts are created.

25. Why Use Concentric Mosaics? • Quick and easy image capture • Parallax and specular highlights are preserved • Much smaller data sets than Lumigraphs • No messy geometry and lighting • User interaction is automatically incorporated

26. Future Endeavors • Correcting vertical distortion • Increasing the region of motion • Improving compression ratios

27. One Last Example

28. Demo

29. Mathematical Madness