video motion capture n.
Skip this Video
Loading SlideShow in 5 Seconds..
Video Motion Capture PowerPoint Presentation
Download Presentation
Video Motion Capture

Video Motion Capture

1 Vues Download Presentation
Télécharger la présentation

Video Motion Capture

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Video Motion Capture Christoph Bregler Jitendra Malik UC Berkley 1997 Phillip Saltzman

  2. Overview • Challenges • Review • Method • Results • Conclusions Phillip Saltzman

  3. Challenges • High Accuracy • Frequent Inter-part Occlusion • Low Contrast Phillip Saltzman

  4. Review Phillip Saltzman

  5. Review • Motion capture on synthetic images • O’Rouke and Balder, 1980 • 1 DOF marker free tracking • Hogg, 1983. Rohr, 1993 • Higher DOF full body tracking • Gravrila and Davis, 1995 Phillip Saltzman

  6. Review About the previous work • All in controlled environments with high contrast and clear edge bounries • Most use skintight suits or markers • Camera calibration needed Phillip Saltzman

  7. Method Phillip Saltzman

  8. Method Basic Assumptions • From frame to frame, all intensity pixel intensity changes are local: • u is motion model and is written as a matrix equation: Phillip Saltzman

  9. Method Finding Gradients • Gradient form of the first equation: • Find a least squares solution to f • Warp image I(t+1) based on f • Find new gradients • Repeat to minimize Phillip Saltzman

  10. Method Motion as twists • Standard pose matrix to move from object space to camera space (3D) • Scaled orthographic projection moves to image space • Requires knowing something about the 3D model of the image. Approximated as ellipsoids. Phillip Saltzman

  11. Method Motion as twists • Any motion can be represented as a rotation about an axis, and a translation about that axis • For example, to make this motion: Phillip Saltzman

  12. Method Motion as twists You make this motion: Phillip Saltzman

  13. Method Motion as twists • Twists can be represented as small vector or matrix • Can be made to a pose by • Encode the motion of a pixel between two frames Phillip Saltzman

  14. Method Motion as twists • Linear algebra manipulation allows using the twist vector to write a motion equation for each pixel • Those equations are put in a vector and used to find a global f parameter for that object Phillip Saltzman

  15. Method Kinematic chains • Body parts represented as multiple connected objects • Each object can be found by the top pose and an angle and twist for each object down the chain • More linear algebra is used to find a f for each body part Phillip Saltzman

  16. Method Multiple cameras • Adds accuracy because change of fully occluded parts drop with each view • Normal motion equation is: • H is system of equations for each pixel • f is global parameter vector for each object • z is initial position of the pixel Phillip Saltzman

  17. Method Multiple cameras • Adding synchronized cameras: • H becomes a matrix where each column represents a view • The f vector gets a term Wfor each view that represents the pose seen from that view • z becomes a vector with an initial position for each view. Phillip Saltzman

  18. Method Support maps • Limits pixel search to area defined by map for speed • Value for each pixel in range [0,1], where 1 means pixel is in the region • Method for finding starts as an elliptical guess, but refining it is not described Phillip Saltzman

  19. Method Algorithm review Input: Image I(t), I(t+1), pose and IK angles Output: Pose and IK angles for I(t+1) Find 3D points for each pixel in image Compute support map for each segment Set poses and IK angles for I(t+1) = I(t) Iterate: Compute gradients Estimate f Update poses and IK angles Warp image based on the pose and support map Phillip Saltzman

  20. Method Initialization • Algorithm depends on known positions for the first frame • For multiple views, each first frame must be initialized • User clicks joint positions, and 3D estimations and joint angles are computed • Values like symmetry can be enforced Phillip Saltzman

  21. Results Phillip Saltzman

  22. Results In Lab Movie • Single angle • 53 frames with decent results • Upper leg hard to track, so IK chain compensates with lower leg and torso Phillip Saltzman

  23. Results Oblique Lab Movie • Oblique angle • Tracking over 45 frames • Algorithm could track change in scale due to perspective changes Phillip Saltzman

  24. Results Digital Muybridge • Oldest known “movie” • High noise and low contrast • Low framerate • Multiple views Phillip Saltzman

  25. Conclusions Future Work/Shortcomings • May break with large movements • Fixed camera only • Did not show tracking of back limbs • No timing data • Few results Phillip Saltzman