Label Embedding Trees for Large Multi-class Tasks

1. Label Embedding Trees for Large Multi-class Tasks SamyBengio Jason Weston David Grangier Presented by Zhengming Xing

2. Outline • Introduction • Label Trees • Label Embeddings • Experiment result

3. Introduction Large scale problem: the number of example Feature dimension Number of class Main idea: propose a fast and memory saving multi-class classifier for large dataset based on trees structure method

4. Introduction Label Tree: Indexed nodes: Edges: Label Predictors: Label sets: The root contain all classes, and each child label set is a subset of its parent K is the number of classes Disjoint tree: any two nodes at the same depth cannot share any labels.

5. Introduction Classifying an example:

6. Label Trees Tree loss I is the indicator function is the depth in the tree of the final prediction for x

7. Label tree Learning with fixed label tree: N,E,L chosen in advance Goal: minimize the tree loss over the variables F Given training data Relaxation 1 Replace indicator function with hinge loss and Relaxation 2

8. Label tree Learning label tree structure for disjoint tree Basic idea: group together labels into the same label set that are likely to be confused at test time. Treat A as the affinity matrix and apply the steps similar to spectral clustering

9. Label embeddings define is a k-dimensional vector with a 1 in the yth position and 0 otherwise solve Problem : how to learn W, V

10. Label embeddings Method 1: Learn V The same two steps of algorithm 2 minimize Learn W minimize

11. Label embedding Method 2: join learn W and V minimize Combine all the methods discussed above minimize

12. Experiment Dataset

13. Experiment

14. Experiment