Kohonen Maps and Counterpropagation Artificial Neural Networks Toolbox for MATLAB

1. Davide Ballabio Milano Chemometrics and QSAR Research Group Università Milano - Bicocca Kohonen Maps and Counterpropagation Artificial Neural Networks Toolbox for MATLAB

2. Kohonen and CPANN Toolbox for MATLAB • Kohonen Maps (or Self Organizing Maps) and Counterpropagation Artificial Neural Networks (CPANNs) are two of the most popular Neural Networks (algorithms that simulate the human learning). • can handle both supervised and unsupervised problems • Kohonen and CP-ANN toolbox is a collection of MATLAB modules freely available via internet: http://michem.disat.unimib.it/chm

3. Kohonen maps • Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems SOM are made of neurons and need time (iterations) to learn to describe the data. The time (number of iterations) and the size (number of neurons) must be defined by the user

4. x1 x2 x3 x4 Kohonen maps • Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems TOP MAP A Kohonen layers p Kohonen layers (p = number of variables)

5. X Kohonen maps • Kohonen Maps (or Self Organizing Maps) can handle unsupervised problems Toroidal geometry

6. CPANN • Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems) • CPANNs are an evolution of Kohonen Maps

7. CPANN • Class unfolding: transform the class vector in a matrix of zeros and ones Class (n x 1) Multi-y (n x G) 1 1 2 2 3 3 .. G G 1 0 0 … … … 0 1 0 0 … … … 0 0 1 0 … … … 0 0 1 0 … … … 0 0 0 1 … … … 0 0 0 1 … … … 0 0 0 0 … … … 1 0 0 0 … … … 1 unfolding

8. x1 x2 x3 x4 A y1 y2 y3 CPANN • Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems) TOP MAP Kohonen layers Output layers

9. CPANN • Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems): For each sample, for each epoch: 1) Find the winning neuron

10. CPANN • Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems): For each sample, for each epoch: 2) Update the Kohonen weights learning rate topological distance

11. CPANN • Counterpropagation Artificial neural Networks can handle supervised problems (in this case classification problems): For each sample, for each epoch: 3) Update the output weights

12. Kohonen Maps and CPANN • In order to better understand how variables can characterize the data, we can do Principal Component Analysis on the Kohonen weights: variables 1. Eigenvalues: decide how many component we can retain PCA W Weights matrix N x p neurons 2. Eigenvectors (loadings) analyse the variables 3. Scores analyse the neurons relate neurons (and samples placed in neurons) and variables in a global way

13. Kohonen Maps and CPANN toolbox • Features of the toolbox: • user defined data scaling + automatic range scaling • modules for fitting and validating models • main available settings: • size and epochs (required) • boundary condition • learning rates can be modified • missing values are handled • visualize top map with Graphical User Interface • visualize PCA on weights with Graphical User Interface • The toolbox is based on the algorithm explained in: • Zupan J, Novic M, Ruisánchez I. Chemometrics and Intelligent Laboratory Systems (1997) 38 1-23.

14. Kohonen Maps and CPANN toolbox • Example of application on the ITAOILS dataset: • 572 olive oil samples, • each sample is described by the percentage composition of 8 fatty acids (variables) • samples belong to 9 different Italian areas of productions (classes) • the final aim of the classification model is the geographical origin determination of the samples. • Data reference: • M. Forina, C. Armanino, S. Lanteri, E. Tiscornia, in: Food Research and Data Analysis, Classification of olive oils from their fatty acid composition, Applied Science Publishers, London, 1983.

15. Kohonen Maps and CPANN toolbox • Principal Component Analysis for data structure evaluation

16. Davide Ballabio Milano Chemometrics and QSAR Research Group Department of Environmental Sciences Università Milano – Bicocca You can download the Kohonen and CP-ANN toolbox 1.0 here: http://michem.disat.unimib.it/chm Version 2.0 will be available soon… Paper in preparation, to be submitted to Chemometrics and Intelligent Laboratory Systems Thanks for your attention !