Window-based models for generic object detection

1. Window-based models for generic object detection Mei-Chen Yeh 04/24/2012

2. Object Detection • Find the location of an object if it appear in an image • Does the object appear? • Where is it?

4. Viola-Jones face detector P. Viola and M. J. Jones. Robust Real-Time Face Detection. IJCV 2004.

5. Viola-Jones Face Detector: Results

6. Viola-Jones Face Detector: Results

7. Viola-Jones Face Detector: Results

8. Viola-Jones Face Detector: Results Paul Viola, ICCV tutorial

9. A successful application

11. Consumer application: iPhoto 2009 http://www.apple.com/ilife/iphoto/ Slide credit: Lana Lazebnik

12. Consumer application: iPhoto 2009 • Things iPhoto thinks are faces Slide credit: Lana Lazebnik

13. Consumer application: iPhoto 2009 • Can be trained to recognize pets! http://www.maclife.com/article/news/iphotos_faces_recognizes_cats Slide credit: Lana Lazebnik

14. Challenges scale illumination view point variation Michelangelo 1475-1564 occlusion Magritte, 1957 Slide credit: Fei-Fei Li

15. Challenges deformation background clutter Xu, Beihong 1943 Klimt, 1913 Slide credit: Fei-Fei Li

16. Basic framework • Build/train object model • Choose a representation • Learn or fit parameters of model / classifier • Generate candidates in new image • Score the candidates

17. Basic framework • Build/train object model • Choose a representation • Learn or fit parameters of model / classifier • Generate candidates in new image • Score the candidates

18. Window-based modelsBuilding an object model Given the representation, train a binary classifier Face/non-face Classifier Yes, face. No, not a face.

19. Basic framework • Build/train object model • Choose a representation • Learn or fit parameters of model / classifier • Generate candidates in new image • Score the candidates

20. Window-based modelsGenerate and score candidates • Scans the detector at multiple locations and scales face/non-face Classifier

21. Car/non-car Classifier Window-based object detection • Training: • Obtain training data • Define features • Define classifier • Given new image: • Slide window • Score by classifier Training examples Feature extraction

22. Viola-Jones detection approach • Viola and Jones’ face detection algorithm • The first object detection framework to provide competitive object detection rates in real-time • Implemented in OpenCV • Components • Features • Haar-features • Integral image • Learning • Boosting algorithm • Cascade method

23. Haar-features (1) • The difference between pixels’ sum of the white and black areas

24. Haar-features (2) • Capture the face symmetry

26. Haar-features (3) Type A Four types of haar features Can be extracted at any location with any scale! A 24x24 detection window

27. 24 24 Haar-features (4) • Too many features! • location, scale, type • 180,000+ possible features associated with each 24 x 24 window • Not all of them are useful! • Speed-up strategy • Fast calculation of haar-features • Selection of good features AdaBoost

28. 3 5 8 12 15 • 5 8 11 16 22 28 • 9 14 18 24 31 39 • 1 2 3 4 3 • 2 1 2 2 3 • 2 1 1 1 2 Image Integral image Integral image (1) Sum of pixel values in the blue area Example: Time complexity?

29. A, B: 2 rectangles => C: 3 rectangles => D: 4 rectangles => Integral image (2) a = sum(1) b = sum(1+2) c = sum(1+3) d = sum(1+2+3+4) 1 2 a b 3 4 c d Sum(4) = ? d + a – b – c Four-point calculation! 6-point 8-point 9-point

30. Feature selection • A very small number of features can be combined to from an effective classifier! • Example: The 1st and 2nd features selected by AdaBoost

31. Feature selection • A weak classifier h f1 f2 f1 > θ (a threshold) => Face! f2 ≤ θ (a threshold) => Not a Face! 1 if fi > θ h = 0 otherwise

32. Feature selection • Idea: Combining several weak classifiers to generate a strong classifier …… α1 α3 α2 αT α1h1+ α2h2 + α3h3 + … + αThT Tthresold >< ~performance of the weak classifier on the training set weak classifier (feature, threshold) h1 = 1 or 0

33. Feature selection • Training Dataset • 4916 face images • non-face images cropped from 9500 images non-face images positive samples negative samples

34. AdaBoost • Each training sample may have different importance! • Focuses more on previously misclassified samples • Initially, all samples are assigned equal weights • Weights may change at each boosting round • misclassified samples => increase their weights • correctly classified samples => decrease their weights

35. Boosting illustration: 2D case Weak Classifier 1 Slide credit: Paul Viola

36. Boosting illustration Weights Increased

37. Boosting illustration Weak Classifier 2

38. Boosting illustration Weights Increased

39. Boosting illustration Weak Classifier 3

40. Boosting illustration Final classifier is a combination of weak classifiers

41. Viola-Jones detector: AdaBoost • Want to select the single rectangle feature and threshold that best separates positive (faces) and negative (non-faces) training examples, in terms of weighted error. Resulting weak classifier: For next round, reweight the examples according to errors, choose another filter/threshold combo. … Outputs of a possible rectangle feature on faces and non-faces. Kristen Grauman

42. Initial weights for each data point AdaBoost -∞ ∞ decreased decreased increased fi ~ error rate error↘ α↗ misclassified

43. AdaBoost ……

44. Learning the classifier • Initialize equal weights to training samples • For T rounds • normalize the weights • select the best weak classifier in terms of the weighted error • update the weights(raise weights to misclassified samples) • Linearly combine these T weak classifiers to form a strong classifier

45. AdaBoost Algorithm Start with uniform weights on training examples {x1,…xn} For T rounds Evaluate weighted error for each feature, pick best. Re-weight the examples: Incorrectly classified -> more weight Correctly classified -> less weight Final classifier is combination of the weak ones, weighted according to error they had. Freund & Schapire 1995

46. Feature Selection: Results First two features selected

47. Boosting: pros and cons • Advantages of boosting • Integrates classification with feature selection • Complexity of training is linear in the number of training examples • Flexibility in the choice of weak learners, boosting scheme • Testing is fast • Easy to implement • Disadvantages • Needs many training examples • Often found not to work as well as an alternative discriminative classifier, support vector machine (SVM) • especially for many-class problems Slide credit: Lana Lazebnik

48. Viola-Jones detection approach • Viola and Jones’ face detection algorithm • The first object detection framework to provide competitive object detection rates in real-time • Implemented in OpenCV • Components • Features • Haar-features • Integral image • Learning • Boosting algorithm • Cascade method

49. Even if the filters are fast to compute, each new image has a lot of possible windows to search. • How to make the detection more efficient?

50. 1 2 3 Cascade method Strong Classifier = (α1h1+ α2h2) + (…)+ (…+ αThT) Tthresold >< Most windows contain no face! Rejects negative windows in an early stage!