Application of LVQ to novelty detection using outlier training data

1. Application of LVQ to novelty detection using outlier training data Hyoung-joo Lee, Sungzoon Cho, Pattern Recognition Letters, 2006. (article in press) . Presenter : Wei-Shen Tai Advisor : Professor Chung-Chian Hsu 2006/7/5

2. Outline • Introduction • Learning vector quantization for novelty detection • Codebook update for an LVQ for novelty detection • Determining local thresholds • Parameters for the proposed approach • Experimental results • Conclusion and discussion • Comments

3. Motivation • Novelty detection • A model learns the characteristics of normal patterns in training data and detects outliers or novel patterns. • Original LVQ problem • Cannot deal with a highly imbalanced dataset, • Codebook update is modified • codebooks should be located close to normal patterns and far away from novel patterns.

4. Objective • Local thresholds to determine • Effectively exclude novel patterns outside boundaries of the normal class. • LVQ for novelty detection (ND) • generate more accurate and tighterboundaries than other approaches that use only the normal class of patterns.

5. Training algorithm and classification

6. Results on an artificial dataset • No training at all since all codebooks were assigned to the normal class while training was prematurely stopped due to the class imbalance .(a) SOM-G, (b) SOM-L, (c) LVQ-ND and (d) LVQ. • Effects of the modified LVQ update. (a) True boundaries, (b) SOM, (c) LVQ-ND and (d) LVQ.

7. Results on real-world datasets When applied to the Ratsch’s benchmark datasets and the pump vibration dataset, It performed better than other widely-used novelty detectors.

8. Codebook update rule • Initial codebooks • Generated by training a SOM. Note that only the normal patterns are used in this process. • A modified error function (yi = +1, -1) • Codebooks can be written as • if xi does not belong to Voronoi region Sk that wk represents, wkremains unchanged. If xi does belong to Sk, wk moves toward xi if xi is normal, or moves away from xi otherwise.

9. Determining local threshold • Voronoi region Sk • A hypersphere with a center at wk and a minimal radius can be obtained so that it surrounds as many normal patterns and as few novel patterns as possible. • Find the radius • an ‘‘optimization’’ problem • a large radius can surround many normal patterns, but may increase false acceptance. • a small radius can exclude many novel patterns, but may increase false rejection.

10. Parameter setting • The number of codebooks, K, • Minimize the misclassification error • C1, C2 • While larger normal regions are defined with a larger C1, tighter boundaries are obtained with a larger C2. • Suppose  k, Ok = ; and x1; . . . ; x|Tk| Tk. If FRk denotes the FRR (false rejection rate) in Voronoi region Sk, the following holds (|Tk|-uk means normal pattern outside the hypersphere)

11. Average AUROCs (%) with respect to |O|/|T| • (a) Banana, (b) Breast-cancer, (c) Diabetes, (d) German, (e) Heart and (f) Titanic.

12. Conclusions • Utilizing information on the novel class • LVQ-ND and SVDD, outperformed their counterparts at least slightly, it can improve novelty detection performance. • Well determined thresholds • A codebook-based method with well determined thresholds can be good enough for novelty detection tasks. (in SOM-L) • The number of novel patterns gradually increases. • As |O|/|T| increases, however, the LVQ-ND excels other models.

13. Comments • Is it feasible for classification of two more classes? • Focus on outlier processing, but those functions seems cannot be utilized in the experiments. • If it do so, the effectiveness of LVQ-ND merely was applied in binary classification so far. • Further experiments for multiple classes • It is essential for demonstrating those effectiveness of the proposed method or functions.