1. Object Tracking Zhenhua Guo 17-Dec-08

2. Goal • Estimate the trajectory of an object in the image plane as it moves around a scene. • position of the object • Additional goals • area of the object • orientation of the object

3. Assumptions • Rigid/semirigid object • Color of the arget object is not similar to background color • Small speed • Motion has patterns

4. Features • Color • Space • RGB • HSV (seems better) • Representation • Multi-dimension histogram

5. Bootstrap • User selects region of interest • Offline training is not needed

6. Techniques • Color-based • Template matching • Kanade-Lucas-Tomasi • Camshift • Motion model based • Kalman filter • Particle filter

7. Template Matching • Move the template over the search image to find the most similar region. • template window size • fixed, the initial region selected by user • Step • If the template is moved one pixel at a time, too slow • If the template is moved too far at a time, inaccurate • In my case, step is 10 pixels in both horizontal and vertical directions. • Histogram comparison • Intersection • It can partly solve the problems: • a variety of viewpoints • partial occlusion • varying image resolution

8. KLT • Uses spatial intensity gradient information to direct the search for the position that yeilds the best match. • Coarse-fine strategy • Use a low-resolution smoothed version of the image to obtain an approximate match • Applying the algorithm to higher resolution images will refine the match

9. Camshift • Every pixel has a probability • Seek mode in the probability image from initial region • Deduce size and orientation

10. Kalman Filter • System model St = A*St-1 + M Ot = B*St + N (S is state and O is observation) • Every state is a quadruple (x, y , delta_x, delta_y) • Every observation is a pair (x, y) • Formulation xt+delta_xt→ xt+1 yt+delta_yt→ yt+1 delta_xt→ delta_xt+1 delta_yt→ delta_yt+1 • Initial transition matrix A is 1 0 1 00 1 0 10 0 1 00 0 0 1 • Initial measurement matrix B is1 0 1 0 0 1 0 1

11. Particle Filter • Using sampling to maintain multiple hypotheses • Weight of each sample  possibility • How to decide number of samples? • trial and error • in my case, the number is 10.

12. Combination • Weighted sum of some algorithms selected based on contextual information • For template matching, there are two thresholds: lowconfidence and highconfidence. • Algorithm sketch • matching result < lowconfidence • Kalman filter and particle filter are used • matching result > highconfidence • if result in last frame < highconfidence • weighted sum • if result in last frame > highconfidence • small speed assumption • matching result falls in between • if result in last frame > highconfidence • small speed assumption • else • weighted sum

13. Problems • Illumination change

14. Thanks !