Dynamic View Morphing

1. Dynamic View Morphing • performs view interpolation of dynamic scenes

2. Expanded Theory • orthography • methods for finding camera-to-camera transformation • virtual camera not restricted to line connecting original cameras • “weak rectification” is sufficient for physical correctness • appearance of straight-line motion without camera-to-camera transformation

3. A A B A motion from time=0 to time=1, as seen through A

4. A A A B B B For Orthographic Projection physically correct straight-line motion (because motion vectors aligned) constant-velocity motion (because motion vectors identical)

5. For Perspective Projection • IF first make image planes parallel to: • motion of object, and • each other • THEN orthographic results apply • condition above is “weak rectification”

6. A B time = 0 time = 1 camera views related by fundamental matrix F

7. A B time = 1 time = 0 camera views still related by same fundamental matrix F

8. A B time = 0 time = 1

9. A B each object W has its own fundamental matrix FW

10. T B B A A Camera-to-camera transformation • denoted TAB • once known, view interpolations portray “constant velocity” motion • potential for model building

11. Finding TAB • can be determined from fundamental matrices for two distinct objects • can be determined from four conjugate directions • can be approximated from two conjugate directions

12. A B each object W has its own fundamental matrix FW

13. Environment Map • “environment map” or “panoramic mosaic” or “plenoptic function”: all the light that reaches a given point in space at an instant in time

14. Environment Map Morphing • view morphing for environment maps A time=0.0 ??? time=0.4 B time=1.0

15. a b c 0 1 0 0 0 1 that is, make TBA = Environment Map Morphing • (STEP 1) find fundamental matrix • (STEP 2) “strongly rectify” the views then notice that, for any point in space, camera A and camera B will give the same y and z coordinates

16. Environment Map Morphing • (STEP 3) project environment map onto “image cylinder” (a.k.a “pipe”) • (STEP 4) interpolate conjugate points and morph this is the cylinder y2 + z2 = 1

17. y2 + z2 = 1 “image cylinder” z = 1 “image plane”

18. Interpolating Augmented Views A B

19. Benefits • placing synthetic object over real object • segmentation • point correspondences • camera-to-camera transformation • added realism: moving parts, shadows, transparency, don’t morph synthetic object • can also use real object views instead of a synthetic object

20. Benefits • automation • by matching edges, computer can place model automatically • all previous benefits become automated • scenario visualization • combine synthetic objects with real scenes to create new scenarios

21. cylinder y2 + z2 = 1

22. A B = TBA x after applying TBA A and B

23. Outline • layering; static scenes, improvement • orthography • generalization of math for view morphing • making objects appear to follow line • Tab and how to find

24. Layering

25. Underlying Mathematics • “weak” rectification: image planes parallel • virtual movement not restricted to line

26. Orthography • long-distance photography • no prewarps needed! (physical correctness) • straight-line motion by aligning directions

27. Preconditions/Output

28. Appearance of Straight-line Motion

30. Orthographic Projection physically correct straight-line motion constant-velocity motion A B

32. T B B A A T B B A B A A = x TBA A B A A B

33. t = 1 t = 0 B took this view A took this view after applying TBA A and B

34. [ [

35. A B physically correct straight-line motion constant-velocity motion