1. Datorseende TexPoint fonts used in EMF: AA

2. World model Computer graphics World model Computer vision What is computer vision? • Image Understanding (AI, behavior) • Computer emulation of human vision • A sensor modality for robotics • Inverse of Computer Graphics

3. shape estimation • modeling • shape • light • motion • optics • images • IP motion estimation rendering recognition • modeling • shape • light • motion • optics • images • IP surface design 2D modeling animation Computer Vision user-interfaces Computer Graphics Intersection of vision and graphics

4. Image-based rendering • What is image-based rendering? • The synthesis of new views of a scene from pre-recorded pictures • Why? • Many applications

5. Example: Panoramic mosaics • + + … + =

6. Image-based rendering • How? General pipeline:

7. Image-based rendering • Three approaches: • 3D model construction from image sequences • Transfer-based image synthesis • Light field

8. Approach 1:3D model construction from image sequences • Techniques that first recover a three dimensional scene model from a sequence of pictures, then render it with classical computer graphics tools • Scene modelling from: • Registered images • Unregistered images

9. Scene modellingfrom registered images • All images are registered in the same global coordinate system • What kinds of reconstruction? • Volumetric reconstruction • Surface reconstruction • Depth maps • …

10. Surfaces and their outlines Occluding contour Camera centre Image contour

11. Surfaces and their outlines Shadow boundary The viewing cone

12. Volumetric reconstruction • It is impossible to uniquely reconstruct an object from its image contours. Why? • Two main constraints imposed on a solid shape by its image contours: • The shape should lie in the intersection of all viewing cones • The cones should be tangent to its surface • Techniques: • Voxel carving • Polyhedral approximation • Smooth surface fitting

13. Smooth surfaces from image contours • Example by Ponce: Spline parametrization which minimizes the energy:

15. Virtualized RealityTM • Capture synchronized video from a full hemisphere of views. • Perform new view generation

16. Virtualized RealityTM • Spatio-Temporal View InterpolationS. Vedula, S. Baker, and T. KanadeEurographics Workshop on Rendering, June, 2002.

17. Virtualized RealityTM • Build 3D model and compute 3D scene flow, interpolate view and time.

18. FILM!

19. Scene modellingfrom unregistered images • Not necessary to reconstruct all images into one global coordinate system • A priori model of the scene

20. Image-based modeling

21. Façade • Select building blocks • Align them in each image • Solve for camera poseand block parameters(using constraints)

22. View-dependent texture mapping • Determine visible cameras for each surface element • Blend textures (images) depending on distance between original camera and novel viewpoint

23. FILM!

24. Model-based reconstruction from one image J-E Solem, F. Kahl, 2005

26. Approach 2: Transfer-based image synthesis This example is based on computing consistent homographies between all planes (B. Johansson, 2003)

27. View Morphing • Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’96]

28. Affine view synthesis • På tavlan!

30. Approach 3: The light field

31. What is light? • Electromagnetic radiation (EMR) moving along rays in space • R(l) is EMR, measured in units of power (watts) • l is wavelength • Light field • We can describe all of the light in the scene by specifying the radiation (or “radiance” along all light rays) arriving at every point in space and from every direction

32. Ray • Constant radiance • time is fixed • 5D • 3D position • 2D direction

33. Line • Infinite line • 4D • 2D direction • 2D position • non-dispersive medium

34. Image • What is an image? • All rays through a point • Panorama

35. Panoramic Mosaics • Convert panoramic image sequence into a cylindrical image • + + … + =

36. Image • Image plane • 2D • position in plane

37. Object • Light leaving towards “eye” • 2D • just dual of image

38. Object • All light leaving object

39. Object • 4D • 2D position (on surface) • 2D direction

40. Object • All images

41. The light field • Summary: • Capture as many images as possible • Store them in a smart way • Discretize rays to synthesize new images

42. Complex Light Field acquisition • Digital Michelangelo Project • Marc Levoy, Stanford University • Lightfield (“night”) assembled by Jon Shade

43. Surface Light Fields • [Wood et al, SIGGRAPH 2000]

44. Sammanfattning • Vysyntes och bildbaserad modellering • Nära relationer till datorgrafik • Tre strategier: • Först 3D modell, sedan använd datorgrafik • Transfer-baserad vysyntes • Light field