A Robust Pedestrian Detection Approach Based on Shapelet Feature and Haar Detector Ensembles

1. A Robust Pedestrian Detection Approach Based on Shapelet Feature and Haar Detector Ensembles Wentao Yao, Zhidong Deng TSINGHUA SCIENCE AND TECHNOLOGY ISSNl l1007-0214l l04/12l lpp40-50 Volume 17, Number 1, February 2012

2. OUTLINE • Introduction • Proposed Algorithm • Improved Full Body Shapelet Pedestrian Detector • Ensemble Approach for Pedestrian Part Detectors • Experimental Results • Conclusions and Future Work

3. OUTLINE • Introduction • Proposed Algorithm • Improved Full Body Shapelet Pedestrian Detector • Ensemble Approach for Pedestrian Part Detectors • Experimental Results • Conclusions and Future Work

4. Introduction • Example-based methods for pedestrian detection use one or more detectors trained on a large number of samples that contain both positives and negatives, with the detectors then applied to the images during the detection stage. • Full body detectors can achieve a relatively high detection rate, but they do not deal well with various poses and pedestrian occlusions. • Part-based detection methods have become more popular.

5. OUTLINE • Introduction • Proposed Algorithm • Improved Full Body Shapelet Pedestrian Detector • Ensemble Approach for Pedestrian Part Detectors • Experimental Results • Conclusions and Future Work

6. Proposed Algorithm • This paper describes a two-stage detection approach that combines both full body and part-based detectors. • The first stage uses a full body detector based on shapeletfeatures to generate pedestrian candidates. • The second stage uses part detectors based on Haar-like wavelet features to verify the full body candidates.

7. OUTLINE • Introduction • Proposed Algorithm • Improved Full Body Shapelet Pedestrian Detector • Ensemble Approach for Pedestrian Part Detectors • Experimental Results • Conclusions and Future Work

8. Improved Full Body Shapelet Pedestrian Detector • The absolute gradient intensities along four are used here as the low-level features with a smoothing filter to compensate for small spatial shifts in the detection window. • Use adaboost algorithm • AdaBoost is an algorithm for constructing a “strong” classifier as linear combination of “weak” classifier. • Final classification based on weighted vote of weak classifiers. [17] Sabzmeydani P, Mori G. Detecting pedestrians by learning shapeletfeatures. In: IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, Minnesota, USA, 2007.

9. Shapelet Feature is a weighted combination of low-level features. • The shapelet feature of sub-window is defined as • is the weak classifier in sub-window selected by the adaboost algorithm in the t-thiteration. • is the weight for the weak classifier.

10. After calculating the shapelet feature for each sub-window , the adaboost algorithm is used to train a final strong classifier: • is a weak classifier corresponding to a shapeletfeature in a sub-window. • λis a normalized threshold ranging from 0 to 1.

11. OUTLINE • Introduction • Proposed Algorithm • Improved Full Body Shapelet Pedestrian Detector • Ensemble Approach for Pedestrian Part Detectors • Experimental Results • Conclusions and Future Work

12. Ensemble Approach for Pedestrian Part Detectors • Human parts segmentation and part detectors training • Part detector ensemble • Genetic algorithm search for the optimal labeling state

13. Human Parts Segmentation and Part Detectors Training • The part detectors are trained using Haar-like wavelet . • The human body is divided into five body parts as the head, torso, leg, left arm, and right arm with each corresponding to a part detector. [15] Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: IEEE Conference on Computer Vision and Pattern Recognition. Kauai, HI, USA, 2001: 511-518.

14. Part Detector Ensemble • The detector approach provides a simple, yet powerful way to integrate the part detector results. • The detector ensemble labels a pedestrian candidate as true if at least one substructure is valid. [22] Dai S Y, Yang M, Wu Y, et al. Detector ensemble. In: IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, Minnesota, USA, 2007.

15. The constraint among the parts is a decision function defined on the distance function . • θ represents the inner angles between the parts. • S is the part scale. • L is the distance between the parts. • . • x is a sample. • and are the mean and covariance of Gaussian distribution i in substructure j.

16. The decision function of substructure j is the thresholded comparison between the distance function and a pre-defined threshold ϒ • The detector ensemble substructures were chosen according to the covering set. [22] Dai S Y, Yang M, Wu Y, et al. Detector ensemble. In: IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, Minnesota, USA, 2007.

17. is the false positive rate for substructure . • An ensemble with more substructure detectors tends to have a higher detection rate, as well as a larger number of false positives. • A (t,m) covering set is a set of m element subsets, such that for any t elements, there must exists at least one subset in this covering set, whose elements are all included by those t elements.

18. Genetic Algorithm Search For the Optimal Labeling State • Detect objects using the detector ensemble. • For each substructure, randomly choose the part labeling state and calculate the distance function as in Eq. (3). • If the distance function of the optimal labeling state is smaller than the threshold, as in Eq. (5), then this substructure is recognized as valid. If at least one substructure is valid, then the pedestrian candidate is labeled as positive. • Some pedestrian detection systems use a mean shift method to do the [26] Comaniciu D. Nonparametric information fusion for motion estimation. In: IEEE Conference on Computer Vision and Pattern Recognition. Madison, WI, USA, 2003: 59-66. [27] Guo H M, Guo P, Lu H Q. A fast mean shift procedure with new iteration strategy and re-sampling. In: IEEE International Conference on Systems, Man and Cybernetics. Taipei, Taiwan, 2006: 2385-2389.

19. OUTLINE • Introduction • Proposed Algorithm • Improved Full Body Shapelet Pedestrian Detector • Ensemble Approach for Pedestrian Part Detectors • Experimental Results • Conclusions and Future Work

20. Experimental Results • Miss Rate (MR)、False Positives Per Window (FPPW) • TP, TN, FP, and FN stand for the number of true positives, true negatives, false positives, and false negatives.

21. Tests of the Full Body ShapeletDetector (2,4,6) indicates that the sliding step sizes for sub-windows with 5×5, 10×10, and 15×15 pixels were 2, 4, and 6 pixels along both the x and y directions.

22. Pedestrian Detection System Test Results

24. S stands for shapelet, H for Haar, and FB for fullbody.

26. OUTLINE • Introduction • Proposed Algorithm • Improved Full Body Shapelet Pedestrian Detector • Ensemble Approach for Pedestrian Part Detectors • Experimental Results • Conclusions and Future Work

27. Conclusions and Future Work • The two-stage detection approach eliminates most non-pedestrian samples in the first stage. • Five part detectors and a detector ensemble are trained to verify the pedestrian candidates generated by the first stage. • After the verification process, neighboring detection results are partitioned into equivalency classes and merged together. • More features, such as color and texture, can be used to further improve the system performance.