Синтез изображений по изображениям. Рельефные текстуры

1. Синтез изображений по изображениям.Рельефные текстуры

2. Сегодня на лекции • Введение в Синтез Изображений по Изображениям(Image-Based Rendering) • Простейшие методы IBR • Рельефные текстуры(relief textures)

3. Traditional Rendering • For photorealism • Modeling is hard • Rendering is slow User input texture maps, survey data Modeling Geometry Textures Light sources Rendering Images

4. Image-Based Rendering • Основные идеи: • Использование изображений (фотографий) в качестве исходных данных • Использование методов обработки изображений для визуализации.

5. Image-Based Rendering • For photorealism • Fast modeling • Complexity independent rendering Images user input range scanners Modeling Images & depth maps Rendering Images

6. Traditional vs. Image-based • Imagebased computer graphics has three main advantages • Photorealism of produced images • The speed • Simple modeling

7. Simplest IBR methods. Texture • Texture - is the simplest of IBR methods.

8. Simplest IBR methods. Sprites • Texture + simple planar geometry = Sprite • Sprites are taken from certain camera position (sprite camera)

9. Simplest IBR methods. Sprites • No geometry information and... • Sprites are looking good from view-points close to the sprite camera, but awful from others

10. 3D model rendering distortions • What are we missing? • The effect of 3-dimensionality on the screen is a combination of two effects: perspective distortion and parallax • Sprites are capable of producing perspective distortions but they are unable to produce correct parallax effect

11. Sprites with Depth • Combine depth and color: • Color texel now is a 3D sample.

12. Warping • Using the samples z-values, image can be transformed (or warped), to enchance the image descriptive power (realism)

13. Planar pinhole camera

14. Orthographic camera

15. 3D point position from a pinhole camera • One image is not enough to determine location of a point in 3D.

16. From two cameras • If we have two cameras, Camera1 and Camera2 with different parameters, capturing the same scene from different locations, a point can be expressed as

17. Warping example

18. Schematic view Warping x’ = warp(x) Depth map Final view Parallax, perspective projection, translation

19. Relief textures • Most of modern 3D accelerators can ultimately fast render textured triangles • => We can use this capability to speed up and simplify Image-Based Rendering • Such an algorithm is called Relief Textures

20. Schematic view Software Hardware Pre-warping Texture mapping Relief textures Warped textures Final view Parallax Perspective projection, translation

21. New warping function • x’ = warp(x) = g(h(x), Poly), where • g(y, Poly) is texture-mapping function and usually done in hardware • h(x) is pre-warpring function

22. New warping function (2) • Prewarping function h(x) after some optimizations looks like following: • u2 = (u1+mu[d])*nu[d]; • v2 = (v1+mv[d])*nv[d]; • Extremely simple, isn’t it?

23. Two stage warping example

24. Two stage warping example (2)

25. Filling holes • This sprite with depth was warped to the new viewpoint • Look how many empty spaces on the women face and hair at the picture • Let’s call them holes

26. Two classes of holes • All the holes fall into two classes by its nature • Resampling problem • Missing information

27. Resampling Before warping After warping

28. Resampling methods • Methods to fill the holes • Inverse warping • Meshing • Splatting • Interpolation

29. Splatting • Draw a little cloud (splat) instead of a pixel in desired image. This cloud has to be opaque in its center becoming more and more transparent to its sides • Features • Relatively small computation cost • Not all the holes are filled

30. Interpolation • Use the fact that u and v are independent from each other after pre-warping (two-pass algorithm) and linearly interpolate depth and color in the intermediate and final images

31. Interpolation (2)

32. Пример Пример 1 Пример 2