1. Video Textures Arno Schödl Richard Szeliski David Salesin Irfan Essa Microsoft Research, Georgia Tech

2. Still Photos

3. Video Clips

4. Video Textures

5. Problem Statement video clip video texture

6. Approach How do we find good transitions?

7. Finding Good Transitions Compute L2 distance Di, j between all frames vs. frame i frame j Similar frames make good transitions

8. Markov Chain Representation Similar frames make good transitions

9. Transition Costs Transition from i to j if successor of i is similar to j Cost function: Cij = Di+1, j

10. Transition Probabilities Probability for transition Pij inversely related to cost: high  low 

11. Preserving Dynamics

12. Preserving Dynamics

13. Preserving Dynamics Cost for transition ij Cij = wkDi+k+1, j+k

14. Preserving Dynamics – Effect Cost for transition ij Cij = wkDi+k+1, j+k

15. Deadends No good transition at the end of sequence

16. Future Cost • Propagate future transition costs backward • Iteratively compute new cost Fij = Cij +  minkFjk

17. Future Cost • Propagate future transition costs backward • Iteratively compute new cost Fij = Cij +  minkFjk

18. Future Cost • Propagate future transition costs backward • Iteratively compute new cost Fij = Cij +  minkFjk

19. Future Cost • Propagate future transition costs backward • Iteratively compute new cost Fij = Cij +  minkFjk

20. Future Cost • Propagate future transition costs backward • Iteratively compute new cost Fij = Cij +  minkFjk • Q-learning

21. Future Cost – Effect

22. Finding Good Loops • Alternative to random transitions • Precompute a good set of loops up front (using dynamic programming)

23. Visual Discontinuities • Problem: Visible “Jumps”

24. Crossfading • Solution: Crossfade from one sequence to the other.

25. Morphing • Interpolation task:

26. Morphing • Interpolation task: • Compute correspondencebetween pixels of all frames

27. Morphing • Interpolation task: • Compute correspondence between pixels of all frames • Interpolate pixel position andcolor in morphed frame • based on [Shum 2000]

28. Results – Crossfading / Morphing

29. Results – Crossfading / Morphing Jump Cut Crossfade Morph

30. Crossfading

31. Frequent Jump & Crossfading

32. Video Portrait Useful for web pages

33. Video Portrait – 3D Combine with IBR techniques

34. Region-based Analysis • Divide video up into regions • Generate a video texture for each region

35. Automatic Region Analysis

36. User-controlled Video Textures User selects target frame range slow variable fast

37. Time Warping Lengthen / shorten video without affecting speed shorter original longer

38. Summary • Video clips  video textures • define Markov process • preserve dynamics • avoid dead-ends • disguise visual discontinuities

39. Summary • Extensions • regions • external constraints • video-based animation

40. Discussion • Some things are relatively easy

41. Discussion • Some are hard