Using linking features in learning Non-parametric part models

1. AmmarKamalHattab ENGN2560 Final Project Proposal March 11, 2013 Using linking features in learning Non-parametric part models

2. Project Goal Nk Torso • Project Goal: implement Linking Features Algorithm to detect a set of parts of a deformable object. • Examples: • detect human parts: head, torso,upper/lower limbs • detect facial landmarks: eyes, nose, mouth outlines, etc. • detect animal parts • … tll trl bll brl

3. Linking Features Method • Step 1 Preprocessing : • Finding bounding box • Computing image features (SIFT descriptors on a regular grid) Preprocessing Input Image SIFTs

4. Linking Features Method • Step 2 Individual Parts: • SIFTs are grouped to generate part candidates for each part. • Each part candidate features vote for three orientations and one length estimate • Involves Generalized Hough Transform (GHT) • Using Training Images to help computeApproximate Nearest Neighbors featuresefficiently Preprocessing Individual parts Input Image SIFTs part candidates

5. Linking Features Method • Step 3: Pairwise Connectivity • Find ‘linking features’ for each pair part candidates using Training (to find nearest neighbors) • Find the correct choice of part candidates: • Score of a pair of part candidates is measured by the cumulative score of the linking features between them • This results in combined set of estimated connectivity parameters Preprocessing Individual parts Input Image Pairwise Connectivity connectivity parameters part candidates

6. Linking Features Method • Step 4: Finding the most likely global parts configuration • The most likely joint configuration of parts is inferred from the model. • the most consistent subset of candidate parts is computed by a greedy approximate MAP inference over the estimated model Preprocessing Individual parts Input Image Pairwise Connectivity Global Parts Configuration

7. Project Plan • Stage 1: Implement Preprocessing (and finding SIFTs) • Stage 2: Implement Training (to help compute Approximate Nearest Neighbors Features) • Stage 3: Implement Finding parts candidates (Involves Generalized Hough Transform (GHT) and clustering) • Stage 4: Finding Linking Features using Training Images • Stage 5: Finding the most likely global parts configuration using greedy approach Preprocessing Individual parts Input Image Pairwise Connectivity Global Parts Configuration Parts

8. Results Example

9. References • Leonid Karlinsky, Shimon Ullman: Using Linking Features in Learning Non-parametric Part Models. ECCV (3) 2012: 326-339 • Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. IJCV (2005)

10. Training Phase • The training phase of the method receives a set of images with annotated parts (e.g. stick annotation in upper body experiments). • Enlarge them and find each part descriptor • Then building efficient data structures for similar descriptor search (Approximate Nearest Neighbors ANN) and memorizing a set of parameters for each feature, e.g. relative offset from different parts. • These data structures are later used in order to compute the model probabilities (features probabilities) for test images using Kernel Density Estimation (KDE) Training Images A test feature KDE Fi GHT=