1. Relative Attributes ICCV 2011 Best Paper Devi Parikh and Kristen Grauman

2. Outline • Introduction • Learning Relative Attributes • Relative Zero-shot Learning • Automatic Relative Image Description • Experiments

3. Introduction Smiling ??? Not smiling ??? Natural Not natural

4. Learning Relative Attributes Image features Learn a scoring function Learned parameters

5. Learning Relative Attributes Ranking SVM 2 1 4 3 6 5 Based on [Joachims 2002] Image Relative Attribute Score

6. Relative Zero-shot Learning • N : total categories • N = S + U • S : ‘seen’ categories • training images are provided • “lions are larger than dogs, as large astigers, but less large than elephants” • U : ‘unseen’ categories • training images are not provided

7. Relative Zero-shot Learning Age: Scarlett Hugh Clive Miley Jared Jared Miley Smiling:

8. Relative Zero-shot Learning • For attribute m •  •  • If attribute m is not used to describe  to be the mean of all training image

9. Relative Zero-shot Learning Can predict new classes based on their relationships to existing classes – without training images Age: Scarlett Hugh Clive S Miley Jared Clive Jared Miley Smiling: Smiling Miley J H Age

10. Automatic Relative Image Description Novel image Density Conventional binary description: not dense Dense: Not dense:

11. Automatic Relative Image Description Novel image Density less dense than more dense than

12. Experiments

13. Datasets Public Figures Face (PubFig) [Kumar 2009] 8 classes 800 images 11 attributes: white, chubby, etc. Outdoor Scene Recognition (OSR) [Oliva 2001] 8 classes 2700 images 6 attributes: open, natural, etc.

14. Ranker vs. Classifier – – – + + +

15. Baselines • Zero-shot learning • Binary attributes: Direct Attribute Prediction [Lampert 2009] • Relative attributes via classifier scores • Automatic image-description • Binary attributes – 1 – 2 – 3 + 4 + 5 + 6

16. Relative Zero-shot Learning Rel. att.(ranker) Rel. att. (classifier) Binary attributes An attribute is more discriminative when used relatively

17. Relative zero-shot learning Classifier score Binary attributes Proposed

18. Automatic Relative Image Description Binary (existing): Not natural Not open Has perspective Relative (ours): More natural than insidecity Less natural than highway More open than street Less open than coast Has more perspective than highway Has less perspective than insidecity

19. Automatic Relative Image Description Binary (existing): Not natural Not open Has perspective Relative (ours): More natural than tallbuilding Less natural than forest More open than tallbuilding Less open than coast Has more perspective than tallbuilding

20. Automatic Relative Image Description Binary (existing): Not Young BushyEyebrows RoundFace Relative (ours): More Young than CliveOwen Less Young than ScarlettJohansson More BushyEyebrows thanZacEfron Less BushyEyebrows than AlexRodriguez More RoundFace than CliveOwen Less RoundFace than ZacEfron (Viggo)

21. Thank You