Introduction to Data-driven Animation

1. Introduction to Data-driven Animation Jinxiang Chai Computer Science and Engineering Texas A&M University

2. Outline Data-driven animation - Motion graphs - Motion interpolations - Statistical motion synthesis

3. Motivations for Data-driven Approaches Motion capture data are easy to capture But we cannot capture all kinds of motion variations - different subjects - different styles - different emotions Key idea: reuse prerecorded motion data to achieve new goals!

4. Data-driven Animation Motion processing Goal: convert motion data into a usable form. Motion data Motion User specifications Motion model • Can we automate this? • Must preserve realism and provide control

5. Outline Data-driven animation - Motion graphs - Motion interpolations - Statistical motion synthesis

6. Motion Graphs: Key Ideas Given: lots of prerecorded motion clips Concatenate them to create new motions!

7. Motion Graphs: Key Ideas Given: lots of prerecorded motion clips Concatenate them to create new motions!

8. Maze

9. Motion Concatenation Motion capture region Virtual environment Sketched path Obstacles

10. Motion Concatenation Motion capture region Virtual environment

11. Unstructured Input Data A number of motion clips • Each clip contains many frames • Each frame represents a pose

12. Unstructured Input Data Connecting transition • Between similar frames

13. Graph Construction

14. Building Motion Graphs So how can we find transition points between motion clips?

15. Building Motion Graphs Motion 1 Frames Motion 2 Frames - Every pair of frames has a distance. - Transitions are local minima below a threshold.

16. Finding Similar Frames • Need derivatives (velocity, acceleration, etc.) • Compare motion in joint angle space or 3d point space? • Must account for coordinate invariance • Different camera ≠ different motion!

17. Distance Metric For more detail, refer to [Kovar and Gleicher, Lee et al]

18. Finding Transition Points Transition thresholds control quality vs. flexibility tradeoff. Threshold = 0 cm Threshold = 8 cm Threshold = 16 cm

19. Structures of Motion Graphs Motion data structure: a graph of frames/poses Avoid dead-ends:finding strongly connected components Contact states:avoid transition to dissimilar contact state

20. Interacting with Motion Graphs So given a motion graph, how can we generate an animation sequence?

21. Interacting with Motion Graphs So given a motion graph, how can we generate an animation sequence? - Random graph walk: Any sequence of edges is a motion!

22. Using Motion Graphs How can we control synthesized motions (e.g., moving from point A to point B, speeds, walking directions)? - Graph search: Find graph walks that minimize a cost function.

23. Path Synthesis Goal: extract motion that follows a path. User’s path ( ) Motion’s path ( ) Minimize

24. Motion Control Goal: extract motion (M) that satisfied constraints (C) specified by the user Minimize

25. Results See videos [click here]!

26. Discussion Pros: + Fully automatic: work on unstructured data + High-quality animation: motion concatenations + Easy to control: graph search Cons - Poor generalization: cannot produce new poses - Control accuracy: cannot generalize new poses - Not compact: needs to retain original mocap data - Scalability

27. Outline Data-driven animation - Motion graphs - Motion interpolations - Statistical motion synthesis

28. Motion Interpolations: Key Ideas Given: lots of prerecorded motion clips Interpolating motions to achieve new goals!

29. Motion Interpolations: Key Ideas In research: more than decades [e.g., Rose et al. 98] In games for a long time - Interpolating motions needs build correspondences between motion examples - Thus, motion interpolations require structurally similar motion examples!

30. Motion Decomposition w(t) t t Canonical timeline Time warping functions

31. Motion Decomposition w(t) Reference motion t t Canonical timeline Time warping functions

32. Motion Decomposition w(t) Contact Transitions t t Canonical timeline Time warping functions

33. Motion Decomposition w(t) Motion 1 t t Canonical timeline Time warping functions

34. Motion Decomposition w(t) t t Canonical timeline Time warping functions

35. Motion Decomposition w(t) t t Canonical timeline Time warping functions

36. Motion Decomposition w(t) Using dynamic time warping! t t Canonical timeline Time warping functions

37. Motion Decomposition w(t) t t Canonical timeline Time warping functions

38. Motion Decomposition w(t) Motion 2 t t Canonical timeline Time warping functions

39. Motion Decomposition w(t) t t Canonical timeline Time warping functions

40. Motion Decomposition w(t) t t Canonical timeline Time warping functions

41. Motion Decomposition w(t) t t Canonical timeline Time warping functions

42. Motion Decomposition w(t) t t Canonical timeline Time warping functions

43. Motion Decomposition w(t) t t Canonical timeline Time warping functions

44. Motion Representation Contact Transitions Registered motions Time warping functions

45. Motion Annotation Preprocessed motions mi

46. Motion Annotation Preprocessed motions mi For each motion mi, we annotate the motion with control parameters si - such as walking speed, direction, step size, kicking directions and positions, etc.

47. Motion Annotation Motion space: m Preprocessed motions mi Control parameter space: s

48. Motion Interpolations and Control Motion space: m Preprocessed motions mi How can we generate an animation that achieves the goals specified c* by the user? Control parameter space: s

49. Motion Interpolations and Control Motion space: m Preprocessed motions mi How can we generate an animation that achieves the goals specified c* by the user? Scattered data interpolation Control parameter space: s

50. Scatter Data Interpolations Motion space: m Preprocessed motions mi How can we generate an animation that achieves the goals specified c* by the user? Scattered data interpolation Control parameter space: s