Multi-perspective Panoramas

1. Multi-perspective Panoramas Slides from a talk by Lihi Zelnik-Manor at ICCV’07 3DRR workshop

2. Pictures capture memories

3. Panoramas Registration: Brown & Lowe, ICCV’05 Blending: Burt & Adelson, Trans. Graphics,1983 Visualization: Kopf et al., SIGGRAPH, 2007

4. Bad panorama? Output of Brown & Lowe software

5. No geometrically consistent solution

6. Scientists solution to panoramas:Single center of projection No 3D!!! Registration: Brown & Lowe, ICCV’05 Blending: Burt & Adelson, Trans. Graphics,1983 Visualization: Kopf et al., SIGGRAPH, 2007

7. From sphere to plane Distortions are unavoidable

8. Distorted panoramas Actual appearance Output of Brown & Lowe software

9. Objectives • Better looking panoramas • Let the camera move: • Any view • Natural photographing

10. Stand on the shoulders of giants Cartographers Artists

11. Cartographic projections

12. φ θ Common panorama projections Perspective Stereographic Cylindircal

13. Global Projections Perspective Stereographic Cylindircal

14. Sharp discontinuity perspective perspective Learn from the artists Multiple view points De Chirico “Mystery and Melancholy of a Street”, 1914

15. Renaissance painters solution “School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

16. Personalized projections “School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

17. Multiple planes of projection Sharp discontinuities can often be well hidden

18. Single view Our multi-view result

19. Single view Our multi-view result

20. Single view Our multi-view result

21. Applying personalized projections Input images Foreground Background panorama

22. Single view Our multi-view result

23. Single view Our multi-view result

24. Objectives - revisited • Better looking panoramas • Let the camera move: • Any view • Natural photographing Multiple views can live together

25. Multi-view compositions 3D!! David Hockney, Place Furstenberg, (1985)

26. Why multi-view? Multiple viewpoints Single viewpoint David Hockney, Place Furstenberg, 1985 Melissa Slemin, Place Furstenberg, 2003

27. Multi-view panoramas Single view Multiview Zomet et al. (PAMI’03) Requires video input

28. Long Imaging Agarwala et al. (SIGGRAPH 2006)

29. Smooth Multi-View Google maps

30. What’s wrong in the picture? Google maps

31. Non-smooth Google maps

32. The Chair David Hockney (1985)

33. Joiners are popular Flickr statistics (Aug’07): 4,985 photos matching joiners. 4,007 photos matching Hockney. 41 groups about Hockney Thousands of members

34. Main goals: Automate joinersGeneralize panoramas to general image collections

35. Objectives • For Artists:Reduce manual labor Fully automatic Manual: ~40min.

36. Objectives • For Artists:Reduce manual labor • For non-artists:Generate pleasing-to-the-eye joiners

37. Objectives • For Artists:Reduce manual labor • For non-artists:Generate pleasing-to-the-eye joiners • For data exploration:Organize images spatially

38. What’s going on here?

39. A cacti garden

40. Principles

41. Principles • Convey topology Correct Incorrect

42. Principles • Convey topology • A 2D layering of images Blending: blurry Graph-cut: cuts hood Desired joiner

43. Principles • Convey topology • A 2D layering of images • Don’t distort images translate rotate scale

44. Principles • Convey topology • A 2D layering of images • Don’t distort images • Minimize inconsistencies Bad Good

45. Algorithm

46. Step 1: Feature matching Brown & Lowe, ICCV’03

47. Step 2: Align Large inconsistencies Brown & Lowe, ICCV’03

48. Step 3: Order Reduced inconsistencies

49. Ordering images Try all orders: only for small datasets

50. Ordering images Try all orders: only for small datasets complexity: (m+n)m = # imagesn = # overlaps = # acyclic orders