1. An Efficient Online Algorithm for Hierarchical Phoneme Classification Joseph Keshet joint work with Ofer Dekel and Yoram Singer The Hebrew University, Israel MLMI ‘04 Martigny, Switzerland

2. Motivation Phonetic transcription of DECEMBER Gross errors d ix CH eh m bcl b er Minor errors d AE s eh m bcl b er d ix s eh NASAL bcl b er Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

3. Hierarchical Classification • Goal: spoken phoneme recognition PHONEMES Sononorants Silences Nasals Obstruents Liquids n m ng l Vowels y w Affricates r Plosives jh Fricatives ch Front Center Back f b v g oy aa iy sh d ow ao ih s k uh er ey th p uw aw eh dh t ay ae zh z Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

4. Metric Over Phonetic Tree • A given hierarchy induces a metric over the set of phonemes tree distance Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

5. b a Metric Over Phonetic Tree • A given hierarchy induces a metric over the set of phonemes tree distance Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

6. b a Metric Over Phonemes • Metric semantics:γ(a,b) is the severity of predicting phoneme group “b” instead of correct phoneme “a” • Our high-level goal: Tolerate minor errors … • Sibling errors • Under-confident predictions - predicting a parent …but, avoid major errors Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

7. W0 W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 Hierarchical Classifier • Assume and • Associate a prototypewith each phoneme • Score of phonemeas • Classification rule: Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

8. w0 w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 Hierarchical Classifier • Goal: maintain “close” to • Define • Goal: maintain small Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

9. Online Learning For • Receive an acoustic vector • Predict a phoneme • Receive correct phoneme • Suffer tree-based penalty • Apply update rule to obtain Goal: Suffer a small cumulative tree error Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

10. Tree Loss • Difficult to minimize directly • Instead upper bound bywherealso known as the hinge loss Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

11. Local update – only nodesalong the path from to are updated OnlineUpdate w0 w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

12. Loss BoundTheorem • sequence of examples • satisfies • Then where and Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

13. Extension: Kernels • Since • Note that • Therefore Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

14. Experiments • Synthetic data: • Symmetric tree of depth 4, fan out 3, 121 labels • Prototypes: orthogonal set in with Gaussian noise • 100 train instances and 50 test instances per label • Phoneme recognition: • Subset of the TIMIT corpus • 55 phonemes and phoneme groups • MFCC+∆+∆∆ front-end, concatenation of 5 frames • RBF kernel • 2000 train vectors and 500 test vector per phoneme Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

15. Greedy approach: solve a multiclass problem at nodes with at least 2 children C C C Experiments • Multiclass - Ignore the hierarchy C Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

16. Results Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

17. Results Difference between the tree error rates of the tree algorithm and the multiclass (MC) algorithm gross errors Tree err-MC err Tree err-MC err minor errors Syntheticdata Phonemes Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

18. Tree vs. Multiclass Online Learning • Similarity between the prototypes in Multiclass and Tree training Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University

19. Thanks! Large Margin Hierarchical Classification Joseph Keshet, The Hebrew University