Enhancing Exemplar SVMs using Part Level Transfer Regularization

1. Enhancing Exemplar SVMs using Part Level Transfer Regularization

2. Problem Definition:Image Retrieval

3. Problem Definition: Image Retrieval query

4. Problem Definition: Image Retrieval query Retrieved Images Retrievingsame category in a similar pose Image Database Example:bicycle facing left Retrieved Images query

5. A Candidate Solution: Exemplar SVM (E-SVM) [Malisiewicz’11] [Shrivastava’11] Training a SVM with a single positive and many negative samples Linear SVMs over HoG features [Dalal &Triggs’05], [Felzenszwalb’08] Exemplar SVM

6. A Candidate Solution:Exemplar SVM (E-SVM) Training a SVM with a single positive and many negative samples Retrieval via sliding window search on the image database Linear SVMs over HoG features Image Database [Dalal &Triggs’05], [Felzenszwalb’08] Exemplar SVM

7. A Candidate Solution:Exemplar SVM (E-SVM) Training a SVM with a single positive and many negative samples Retrieval via sliding window search on the image database Linear SVMs over HoG features Image Database Retrieved Images [Dalal &Triggs’05], [Felzenszwalb’08] Exemplar SVM

8. Framework:Enhanced Exemplar SVM (EE-SVM) positive sample Train E-SVM over HoG features Previously Trained Classifiers Exemplar SVM Part-Level Transfer negative samples Enhanced E-SVM

9. Benefit:Enhanced Exemplar SVM (EE-SVM) Exemplar SVM Subwindow Retrieval Retrieved Subwindows Image Database Query Image Retrieved Subwindows Subwindow Retrieval Enhanced E-SVM

10. Overview • Transfer Learning in Computer Vision • Classification & Detection • Enhanced Exemplar SVM • Feature Augmentation vs Transfer • Results & Discussion

11. Transfer Learning in Computer Vision Learning new classes by building upon previously learned classes. • Image Classification • Adaptive SVMs, • Transfer from Multiple Models, • Adaptive Multiple Kernel Learning • Object Detection • Rigid Transfer • Flexible Transfer [Yang et al. ICDM’07] [Tommasiet al. BMVC’09] [Tommasiet al. CVPR’10] [Luoet al. ICCV’11] [Duanet al. CVPR’10] • [Stark et al. ICCV’09] • [Aytar and Zisserman ICCV’11] • [Gao et al. ECCV’12]

12. Transfer Learning for Detection • Rigid Transfer [Aytar and Zisserman ICCV’11] • Transfer between fixed sizedtemplates • Good performance, especially for smaller number of training samples. • Hard to find visually similar detectors with same aspect ratio and size. • Flexible Transfer • Transfer between different sized templates. • Transferring shape features [Stark et al. ICCV’09] • Deformable Transfer [Aytar and Zisserman ICCV’11] • Transfer via Structured Priors [Gao et al. ECCV’12] Fixed Sized Transfer Flexible Transfer

13. Overview • Transfer Learning in Computer Vision • Classification & Detection • Enhanced Exemplar SVM • Feature Augmentation vs Transfer • Results & Discussion

14. Framework:Enhanced Exemplar SVM (EE-SVM) Part-Level Transfer Train E-SVM Exemplar SVM Enhanced E-SVM Query Previously Trained Classifiers

15. Framework:Part-Level Transfer Regularization ui Exemplar SVM

16. Parameters:Part-Level Transfer Regularization close to E-SVM close to construction from ui’s ui

17. Framework:Matching Classifier Patches Exemplar SVM Previously Learned Classifiers ui

18. Why is it beneficial?Part-Level Transfer Regularization • Part level transfer is beneficial because… • parts can be relocated (deformation), • the possibility of finding a good match for transfer increases when we look at smaller classifier patches. • Advantages of transferring parts from well trained classifiers: • Better background suppression and discriminativity due to well trained source classifiers. • Better handling of local variations since source classifiers are trained on many positive samples. • No additional cost on runtime

19. Where is it beneficial?Part-Level Transfer Regularization • Unusual Poses • Composition of Objects [Visual Phrases - Sadeghi CVPR’11]

20. PASCAL 2007:Results - Left Facing Horse query Enhanced E-SVM E-SVM

21. PASCAL 2007:Results - Left Facing Bicycle query Enhanced E-SVM E-SVM

22. PASCAL 2007:Visual Phrase – Riding Horse query Enhanced E-SVM E-SVM

23. ImageNet:Unusual Pose - Bicycle query Enhanced E-SVM E-SVM

24. Overview • Transfer Learning in Computer Vision • Classification & Detection • Enhanced Exemplar SVM • Feature Augmentation vs Transfer • Results & Discussion

25. Implementation:Transfer vs. Feature Augmentation . . . . Transfer Regularization is equivalent to learning . . . “normal” SVM with augmented features. 0.2 0.7 0.1

26. Implications:Transfer vs. Feature Augmentation • This equivalence is not specific to Exemplar SVMs. • Transfer regularization can be implemented as feature augmentation. • Transfer regularization can be efficiently solved using standard SVM packages.

27. Overview • Transfer Learning in Computer Vision • Classification & Detection • Enhanced Exemplar SVM • Feature Augmentation vs Transfer • Results & Discussion

28. PASCAL 2007:Quantitative Results

29. ImageNet:Quantitative Results • Three queries are evaluated for each of the five classes. • Precisions at top 5, 10, 50 and 100 are reported.

30. Handling Occlusions Query E-SVM EE-SVM

31. Handling Truncation Query E-SVM EE-SVM

32. Conclusions • Boosted the performance of E-SVM which incurs no additional cost on runtime. • Presented the equivalence between Transfer regularization and feature augmentation. • Showed the benefit for unusual poses and visual phrases. • Handling truncation and occlusion.