Estimating Human Body Configurations using Shape Context Matching

1. Estimating Human Body Configurations using Shape Context Matching Greg Mori and Jitendra Malik

2. Problem

3. Approach: Exemplar-based Matching • Set of stored exemplars with hand-labeled keypoints • Obtain sample points • Deformable matching to exemplars: • Shape context matching to get correspondences • Kinematic chain deformation model • Estimate 3D body configuration

4. Comparing Pointsets

5. Shape Context Count the number of points inside each bin, e.g.: Count = 4 ... Count = 10 • Compact representation of distribution of points relative to each point

6. Shape Context

7. Comparing Shape Contexts Compute matching costs using Chi Squared distance: Recover correspondences by solving linear assignment problem with costs Cij [Jonker & Volgenant 1987]

8. Deformable Matching • Kinematic chain-based deformation model • Use iterations of correspondence and deformation • Keypoints on exemplars are deformed to locations on query image

10. Problem

11. Estimate 3D Body Configuration [Taylor ’00] • Known: • Relative lengths of body segments • (x,y) Image locations of keypoints • “closer endpoint” labels for each segment • Scaled orthographic camera model • Solve for 3D locations of keypoints up to some scale factor • Scale factor can be estimated automatically

12. Solving for Foreshortening

13. Choosing Scale

14. Results

16. Multiple Exemplars • Parts-based approach • Use a combination of keypoints/whole limbs from different exemplars • Reduces the number of exemplars needed • Compute a matching cost for each limb from every exemplar • Compute pairwise “consistency” costs for neighbouring limbs • Use dynamic programming to find best K configurations

17. Parts-based Approach

18. Tracking by Repeated Finding