1 / 32

Using Photographs to Enhance Videos of a Static Scene

Using Photographs to Enhance Videos of a Static Scene. Pravin Bhat 1 , C. Lawrence Zitnick 2 , Noah Snavely 1 , Aseem Agarwala 3 , Maneesh Agrawala 4 , Michael Cohen 1,2 , Brian Curless 1 , Sing Bing Kang 2. University of Washington 1 , Microsoft Research Redmond 2

farhani
Télécharger la présentation

Using Photographs to Enhance Videos of a Static Scene

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Photographs to Enhance Videos of a Static Scene Pravin Bhat1, C. Lawrence Zitnick2, Noah Snavely1, Aseem Agarwala3, Maneesh Agrawala4, Michael Cohen1,2, Brian Curless1, Sing Bing Kang2 University of Washington1, Microsoft Research Redmond2 University of California3, Adobe Systems4 EGSR 2007

  2. An overview of Spacetime Fusion

  3. Motivation • Low quality video Input Video

  4. Motivation • Low quality video • Reconstructed video Input Video Reconstructed Video

  5. Motivation • Low quality video • Reconstructed video • Reconstructed from photos • Good spatial reconstruction • Bad temporal reconstruction Input Video Reconstructed Video

  6. Motivation • Spacetime Fusion result Input Video Spacetime Fusion Result

  7. Motivation • Spacetime Fusion result • Spatial properties of reconstruction • Temporal properties of input video Input Video Spacetime Fusion Result

  8. Spacetime Fusion • Define a 3D gradient field

  9. Spacetime Fusion • Define a 3D gradient field • Spatial gradients from reconstruction • Temporal gradients from input video

  10. Spacetime Fusion • Define a 3D gradient field • Spatial gradients from reconstruction • Temporal gradients from input video • Key Idea • Temporal gradients defined betweenmotion compensated temporal neighbors

  11. Spacetime Fusion • Define a 3D gradient field • Spatial gradients from reconstruction • Temporal gradients from input video • Key Idea • Temporal gradients defined betweenmotion compensated temporal neighbors Video frame: t - 1 Video frame: t

  12. Spacetime Fusion • Define a 3D gradient field • Spatial gradients from reconstruction • Temporal gradients from input video • Key Idea • Temporal gradients defined betweenmotion compensated temporal neighbors Gt Video frame: t - 1 Video frame: t • Gt(x, y, t) = V(x, y, t) - V(x, y, t - 1)

  13. Spacetime Fusion • Define a 3D gradient field • Spatial gradients from reconstruction • Temporal gradients from input video • Key Idea • Temporal gradients defined betweenmotion compensated temporal neighbors Gt Video frame: t - 1 Video frame: t • Gt(x, y, t) = V(x, y, t) - V(x - u, y - v, t - 1)

  14. Spacetime Fusion • Define a 3D gradient field • Spatial gradients from reconstruction • Temporal gradients from input video • Key Idea • Temporal gradients defined betweenmotion compensated temporal neighbors • Increases compatibility betweentemporal gradients and spatial gradients

  15. Spacetime Fusion • Define a 3D gradient field • Spatial gradients from reconstruction • Temporal gradients from input video • Key Idea • Temporal gradients defined betweenmotion compensated temporal neighbors • Increases compatibility betweentemporal gradients and spatial gradients • Integrate the 3D gradient field

  16. Spacetime Fusion • Integrating the gradient field Solve linear system: Av = b

  17. Spacetime Fusion • Integrating the gradient field Solve linear system: Av = b Constraints: vx, y, t – vx-1, y, t = Gx(x, y, t) vx, y, t – vx, y-1, t = Gy(x, y, t) vx, y, t – vx-u, y-v, t = Gt(x, y, t)

  18. Applications

  19. Enhanced Exposure

  20. Edit Propagation Input Video

  21. Edit Propagation User Edits

  22. Edit Propagation User Edits

  23. Edit Propagation User Edits

  24. Edit Propagation User Edits

  25. Edit Propagation User Edits

  26. Edit Propagation User Edits

  27. Edit Propagation Edited Video

  28. Super-Resolution

  29. Conclusion • Spacetime fusion

  30. Conclusion • Spacetime fusion • Combines spatial and temporal gradients from two different sources

  31. Conclusion • Spacetime fusion • Combines spatial and temporal gradients from two different sources • Requires motion vectors for temporal source • stereo (static scenes) • flow (dynamic scenes)

  32. Conclusion • Spacetime fusion • Combines spatial and temporal gradients from two different sources • Requires motion vectors for temporal source • stereo (static scenes) • flow (dynamic scenes) • Major applications • Enforcing temporal coherence • Transferring lighting information

More Related