Computer Vision: Chamfer System

1. Computer Vision:Chamfer System Dr. Edgar Seemann seemann@pedestrian-detection.com

2. Silhouette Matching Dr. Edgar Seemann

3. Goal Requirements for an alignment algorithm • Align known object shapes with image • High detection rate • Few false positives • Robustness • Computationally inexpensive Chamfer Matching [Gavrila & Philomin ICCV’99] Object shapes Real-world image of object Dr. Edgar Seemann

4. Shape 2 Shape 1 Distance transform 8 6 5 4 4 5 6 8 6 5 3 2 2 3 5 6 5 3 2 0 0 2 3 5 3 2 0 2 2 0 2 3 2 0 2 2 2 2 0 2 2 0 0 0 0 0 0 2 3 2 2 2 2 2 2 3 5 4 4 4 4 4 4 5 Distance Transform • Used to compare/align two (typically binary) shapes • Compute for each pixel the distance to the next edge pixel • Here the eculidean distances areapproximated by the 2-3 distance Distance = ? Dr. Edgar Seemann

5. Distance transform 8 6 5 4 4 5 6 8 6 5 3 2 2 3 5 6 5 3 2 0 0 2 3 5 3 2 0 2 2 0 2 3 2 0 2 2 2 2 0 2 2 0 0 0 0 0 0 2 3 2 2 2 2 2 2 3 5 4 4 4 4 4 4 5 Distance Transform • Overlay second shape over distance transform • Accumulate distances along shape 2 • Find best matching position by an exhaustive search • Distance is not symmetric • Distance has to be normalized w.r.t. the length of the shapes Distance = 14 Dr. Edgar Seemann

6. Chamfer Matching Distance transform of a real-world image • Compute distance transform (DT) • For each possible object location • Position known object shape over DT • Accumulate distances along the contour Distance measure Binary image Distance transform 8 6 5 4 4 5 6 8 6 5 3 2 2 3 5 6 5 3 2 0 0 2 3 5 3 2 0 2 2 0 2 3 2 0 2 2 2 2 0 2 2 0 0 0 0 0 0 2 3 2 2 2 2 2 2 3 5 4 4 4 4 4 4 5 Chamfer Matching Dr. Edgar Seemann

7. 3 2 3 2 0 2 0 3 2 3 Efficient implementation • The distance transform can be efficiently computed by two scans over the complete image • Forward-Scan • Starts in the upper-left corner and moves from left to right, top to bottom • Uses the following mask • Backward-Scan • Starts in the lower-right corner and moves from right to left, bottom to top • Uses the following mask Dr. Edgar Seemann

8. d 2+2 s d 2+3 3+5 s 2 c 2+0 3 3 5 2 3 ? ? ? 2 2 0 ? ? ? ? ? ? ? ? Forward scan • We can choose different values for the filter mask • The local distances, d, s and c, in the mask are added to the pixel values of the distance map and the new value of the zero pixel is the minimum of the five sums • Example: Dr. Edgar Seemann

9. Advantages and Disadvantages • Fast • Distance transform has to be computed only once • Comparison for each shape location is cheap • Good performance on uncluttered images (with few background structures) • Bad performance for cluttered images • Needs a huge number of people silhouettes • But computation effort increases with the number of silhouettes Dr. Edgar Seemann

10. Template Hierachy • To reduce the number of silhouettes to consider, silhouettes can be organized in a template hierarchy • For this, the shapes are clustered by similarity Dr. Edgar Seemann

11. Search in the hierarchy • Matching the shapes, then corresponds to a traversal of the template hierarchy • How can we prune search branches to speed up matching? • Thresholds depend on: • Edge detector (likelihood of gaps) • Silhouette sizes • Hierarchy level • Allowed shape variation • Thresholds are set statisticallyduring training Dr. Edgar Seemann

12. Example Detections Dr. Edgar Seemann

13. Video Dr. Edgar Seemann

14. Coarse-To-Fine Search • Goal: Reduce search effort by discarding unlikely regions with minimal computation • Idea: • Subsample image and searchfirst at a coarse scale • Only consider regions with alow distance when searchingfor a match on finer scales • Again, we have to findreasonable thresholds Level 1 Level 2 Level 3 Dr. Edgar Seemann

15. Protector System (Daimler) Dr. Edgar Seemann

16. Distance = small Adding edge orientation • So far edge orientation has been completely ignored • Idea: Consider edge orientation for each pixel Dr. Edgar Seemann

17. Edge orientation - The math • Given two shapes S, C, we can express the chamfer distance in the following manner • The orientation correspondence between two points is then measured by • The combined distance measure: Dr. Edgar Seemann

18. Statistical Relevance • Adding statistical relevance of silhouette regions further improves the results [Dimitrijevic06] Dr. Edgar Seemann

19. Spatio-Temporal templates • Use multiple successive frames to build a spatio-temporal template (T={T1,…,TN}) • Allow spatial variations of dx, dy (due to motion or camera movements) Dr. Edgar Seemann

20. Example: single-frame vs. 3 frames Dr. Edgar Seemann

21. Quantitative Results • Red: spatio-temporal templates + statistical relevance Dr. Edgar Seemann

22. Video • Restrict detection to a single articulation (when legs are in a v-shaped position) • Spatio-Temporal templates: • Allows more reliable detection of motion direction • Avoids confusions and some false positive detections Dr. Edgar Seemann

23. Alternatives: Earth Mover’s Distance • Originally developed to compare histograms • Idea: Find the minimal ‘flow’ to transform one histogram to another • Example: Dr. Edgar Seemann

24. Chamfer: EMD: EMD Matching • Detect edges in image • For each possible object location • Optimize correspondences between known shape and edge image Example detection Earth mover’s distance (EMD) Distance measure basis Dr. Edgar Seemann

25. EMD – The math • Variant of the transportation problem (possible solutions: Stepping Stone Algorithm, Transportation-simplex method) • Constraints • EMD-Distance Dr. Edgar Seemann

26. Advantages and Disadvantages • Optimizes matching between silhouette and edge structure in image • Enforces one-to-one matchings (unlike chamfer) • Allows partial matches • Can deal with arbitrary features • High computational complexity • Approximation is possible [Graumann, Darrel CVPR’94] Dr. Edgar Seemann