Dissertation Proposal

1. Dissertation Proposal Negative selection algorithms: from the thymus to V-detector Zhou Ji, advised by Prof. Dasgupta

2. Outline • Background of the area • Major contributions of current work • Description of the algorithm • Demonstration of the software • Experimental results • Work to do next

3. Background • AIS (Artificial Immune Systems) – only about 10 years’ history • Negative selection (development of T cells) • Immune network theory (how B cells and antibodies interact with each other) • Clonal selection (how a pool of B cells, especially, memory cells are developed) • New inspirations from immunology: danger theory, germinal center, etc. • Negative selection algorithms • The earliest and most widely used AIS. background

4. Biological metaphor of negative selection How T cells mature in the thymus: • The cell are diversified. • Those that recognize self are eliminated. • The rest are used to recognize nonself. background

5. The idea of negative selection algorithms (NSA) The concept of feature space and detectors • The problem to deal with: anomaly detection (or one-class classification) • Detector set • random generation: maintain diversity • censoring: eliminating those that match self samples background

6. Outline of a typical NSA Anomaly detection: (classification of incoming data items) Generation of detector set background

7. Family of NSA Types of works about NSA • Applications: solving real world problems by using a typical version or adapting for specific applications • Improving NSA of new detector scheme and generation method and analyzing existing methods. Works are data representation specific, mostly binary representation. • Establishment of framework for binary representation to include various matching rules; discussion on uniqueness and usefulness of NSA; introduction of new concepts. What defines a negative selection algorithm? • Representation in negative space • One-class learning • Usage of detector set background

8. Major issues in NSA • Number of detectors • Affecting the efficiency of generation and detection • Detector coverage • Affecting the accuracy detection • Generation mechanisms • Affecting the efficiency of generation and the quality of resulted detectors • Matching rules – generalization • How to interpret the training data • depending on the feature space and representation scheme • Issues that are not NSA specific • Difficulty of one-class classification • Curse of dimensionality background

9. V-detector: work done for the proposed dissertation to deal with the issues in NSA • V-detector is a new negative selection algorithm. • It embraces a series of related works to develop a more efficient and more reliable algorithm. • It has its unique process to generate detectors and determine coverage. contribution

10. V-detector’s major features • Variable-sized detectors • Statistical confidence in detector coverage • Boundary-aware algorithm • Extensibility contribution

11. Variable sized detectors in V-detector method are “maximized detector” • Unanswered question: what is the self space? V-detector: maximized size traditional detectors: constant size contribution

12. Why is the idea of “variable sized detectors” novel? • The rational of constant size: a uniform matching threshold • Detectors of variable size exist in some negative selection algorithms as a different mechanism • Allowing multiple or evolving size to optimize the coverage – limited by the concern of overlap • Variable size as part of random property of detectors/candidates • V-detector uses variable sized detectors to maximize the coverage with limited number of detectors • Size is decided on by the training data • Large nonself region is covered easily • Small detectors cover ‘holes’ • Overlap is not an issue in V-detector contribution

13. Statistical estimate of detector coverage • Exiting works: estimate necessary number of detectors – no direct relationship between the estimate and the actual detector set obtained. • Novelty of V-detector: • Evaluate the coverage of the actual detector set • Statistical inference is used as an integrated components of the detector generation algorithm, not to estimate coverage of finished detector set. contribution

14. Basic idea leading to the new estimation mechanism • Random points are taken as detector candidates. The probability that a random point falls on covered region (some exiting detectors) reflects the portion that is covered -- similar to the idea of Monte Carlo integral. • Proportion of covered nonself space = probability of a sample point to be a covered point. (the points on self region not counted) • When more nonself space has been covered, it becomes less likely that a sample point to be an uncovered one. In other words, we need try more random point to find a uncovered one - one that can be used to make a detector. contribution

15. Statistics involved • Central limit theory: sample statistic follows normal distribution • Using sample statistic to population parameter • In our application, use proportion of covered random points to estimate the actual proportion of covered area • Point estimate versus confidence interval • Estimate with confidence interval versus hypothesis testing • Proportion that is close to 100% will make the assumption of central limit theory invalid – not normal distribution. • Purpose of terminating the detector generation 0 1 proportion contribution

16. Hypothesis testing • Identifying null hypothesis/alternative hypothesis. • Type I error: falsely reject null hypothesis • Type II error: falsely accept null hypothesis • The null hypothesis is the statement that we’d rather take as true if there is not strong enough evidence showing otherwise. In other words, we consider type I error more costly. • In term of coverage estimate, we consider falsely inadequate coverage is more costly. So the null hypothesis is: the current coverage is below the target coverage. • Choose significant level: maximum probability we are willing to accept in making Type I Error. • Collect sample and compute its statistic, in this case, the proportion. • Calculate z score from proportion an compare with za • If z is larger, we can reject null hypothesis and claim adequate coverage with confidence

17. Boundary-aware algorithm versus point-wise interpretation • A new concept in negative selection algorithm • Previous works of NSA • Matching threshold is used as mechanism to control the extent of generalization • However, each self sample is used individually. The continuous area represented by a group of sample is not captured. (point-wise interpretation) Desired interpretation: The area represented by The group of points More specificity Relatively more aggressive to detect anomaly More generalization The real boundary is Extended.

18. Boundary–aware: using the training points as a collection • Boundary-aware algorithm • A ‘clustering’ mechanism though represented in negative space • The training data are used as a collection instead individually. • Positive selection cannot do the same thing contribution

19. V-detector is more than a real-valued negative selection algorithm • V-detector can be implemented for any data representation and distance measure. • Usually negative selection algorithms were designed with specific data representation and distance measure. • The features we just introduced are not limited by representation scheme or generation mechanism. (as long as we have a distance measure and a threshold to decide matching) contribution

20. V-detector algorithm with confidence in detector coverage contribution

21. V-detector algorithm with confidence in detector coverage contribution

22. V-detector algorithm with confidence in detector coverage contribution

23. V-detector’s contributions • Efficiency: • fewer detectors • fast generation • Coverage confidence • Extensibility, simplicity contribution

24. Experiments • A large pool of synthetic data (2-D real space) are experimented to understand V-detector’s behavior • More detail analysis of the influence of various parameters is planned as ‘work to do’ • Real world data • Confirm it works well enough to detect real world “anomaly” • Compare with methods dealing with similar problems • Demonstration • How actual training data and detector look like • Basic UI and visualization of V-detector implementation

25. Parameters to evaluate its performance • Detection rate • False alarm rate • Number of detectors

26. Control parameters and algorithm variations • Self radius – key parameter • Target coverage • Significant level (of hypothesis testing) • Boundary-aware versus point-wise • Hypothesis testing versus naïve estimate • Reuse random points versus minimum detector set (to be implemented) contribution

27. Data’s influence on performance • Specific shape • Intuitively, “corners” will affect the results. • Number of training points • Major influence contribution

28. Synthetic data (‘intersection’ and pentagram): compare naïve estimate and hypothesis testing ‘intersection’ shape pentagram

29. Synthetic data : results for different shapes of self region

30. Synthetic data (ring): compare boundary-aware and point-wise False alarm rate Detection rate

31. Synthetic data (cross-shaped self): balance of errors

32. Real world data • Biomedical data • Pollution data • Ball bearing – preprocessed time series data • Others: Iris data, gene data, India Telugu contribution

33. Results of biomedical data

34. Results of air pollution data Detection rate and false alarm rate Number of detectors

35. Ball bearing data Example of raw data (new bearings, first 1000 points) • raw data: time series of acceleration measurements • Preprocessing (from time domain to representation space for detection) • FFT (Fast Fourier Transform) with Hanning windowing: window size 30 • Statistical moments: up to 5th order

36. Ball bearing experiments with two different preprocessing techniques contribution

37. Results of Iris data

38. Work to do next • Extension to different data representation • Searching for real world applications • Compare with other methods, e.g. SVM • Analysis on the influence of control parameters and algorithm variations to do

39. Publications • Dasgupta, Ji, Gonzalez, Artificial immune system (AIS) research in the last five years, CEC 2003 • Ji, Dasgupta, Augmented negative selection algorithm with variable-coverage detectors, CEC 2004 • Ji, Dasgupta, Real-valued negative selection algorithm with variable-sized detectors, GECCO 2004 • Ji, Dasgupta, Estimating the detector coverage in a negative selection algorithm, GECCO 2005 • Ji, A boundary-aware negative selection algorithm, ASC 2005 • Ji, Dasgupta, Revisiting negative selection algorithms, submitted to the Evolutionary Computation Journal • Ji, Dasgupta, An efficient negative selection algorithm of “probably adequate” coverage, submitted to SMC

40. Questions and comments? Thank you!

41. What is matching rule? • When a sample and a detector are considered matching. • Matching rule plays an important role in negative selection algorithm. It largely depends on the data representation.

42. Match or not match? In real-valued representation, detector can be visualized as hyper-sphere. Candidate 1: thrown-away; candidate 2: made a detector.

43. Experiments and Results • Synthetic Data • 2D. Training data are randomly chosen from the normal region. • Fisher’s Iris Data • One of the three types is considered as “normal”. • Biomedical Data • Abnormal data are the medical measures of disease carrier patients. • Air Pollution Data • Abnormal data are made by artificially altering the normal air measurements • Ball bearings: • Measurement: time series data with preprocessing - 30D and 5D

44. Synthetic data - Cross-shaped self spaceShape of self region and example detector coverage (a) Actual self space (b) self radius = 0.05 (c) self radius = 0.1

45. Synthetic data - Cross-shaped self spaceResults Detection rate and false alarm rate Number of detectors

46. Synthetic data - Ring-shaped self spaceShape of self region and example detector coverage (a) Actual self space (b) self radius = 0.05 (c) self radius = 0.1

47. Synthetic data - Ring-shaped self spaceResults Detection rate and false alarm rate Number of detectors

48. Iris dataComparison with other methods: number of detectors

49. Iris dataVirginica as normal, 50% points used to train Detection rate and false alarm rate Number of detectors

50. Biomedical data • Blood measure for a group of 209 patients • Each patient has four different types of measurement • 75 patients are carriers of a rare genetic disorder. Others are normal.