1 / 26

Self-Organization in Neural Networks: Biological Inspiration and Applications

This lecture by Pushpak Bhattacharyya from IIT Bombay delves into the principles of self-organization in neural networks, inspired by biological systems, particularly the human brain. It covers the structure and functions of the brain's higher regions, such as the cerebrum and cerebellum, and discusses concepts like Maslow’s hierarchy and the dichotomy of left and right brain functions. The lecture explores the Kohonen Self-Organizing Maps (SOM), detailing competitive learning, weight adjustment, and neighborhood functions crucial for clustering and feature extraction. It provides insight into advanced neural network architectures like the NeoCognitron.

tarik-baldwin
Télécharger la présentation

Self-Organization in Neural Networks: Biological Inspiration and Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS623: Introduction to Computing with Neural Nets(lecture-20) Pushpak Bhattacharyya Computer Science and Engineering Department IIT Bombay

  2. Self Organization Biological Motivation Brain

  3. Higher brain Brain Cerebellum Cerebrum 3- Layers: Cerebrum Cerebellum Higher brain

  4. Maslow’s hierarchy Search for Meaning Contributing to humanity Achievement,recognition Food,rest survival

  5. Higher brain ( responsible for higher needs) 3- Layers: Cerebrum (crucial for survival) Cerebrum Cerebellum Higher brain

  6. Mapping of Brain Back of brain( vision) Lot of resilience: Visual and auditory areas can do each other’s job Side areas For auditory information processing

  7. Left Brain and Right Brain Dichotomy Left Brain Right Brain

  8. Words – left Brain Music Tune – Right Brain Left Brain – Logic, Reasoning, Verbal ability Right Brain – Emotion, Creativity Maps in the brain. Limbs are mapped to brain

  9. Character Recognition: A, A, A, , , O/p grid . . . . I/p neuron

  10. Kohonen Net Self Organization or Kohonen network fires a group of neurons instead of a single one. The group “some how” produces a “picture” of the cluster. Fundamentally SOM is competitive learning. But weight changes are incorporated on a neighborhood. Find the winner neuron, apply weight change for the winner and its “neighbors”.

  11. Winner Neurons on the contour are the “neighborhood” neurons.

  12. Neighborhood: function of n Weight change rule for SOM W(n+1) = W(n) + η(n) (I(n) – W(n)) P+δ(n) P+δ(n) P+δ(n) Learning rate: function of n δ(n) is a decreasing function of n η(n) learning rate is also a decreasing function of n 0 < η(n) < η(n –1 ) <=1

  13. Winner δ(n) Convergence for kohonen not proved except for uni-dimension Pictorially . . . .

  14. A … P neurons o/p layer Wp … … . n neurons Clusters: A : A : : B : C :

  15. Clustering Algos 1. Competitive learning 2. K – means clustering 3. Counter Propagation

  16. …… 26 neurons K – means clustering K o/p neurons are required from the knowledge of K clusters being present. Full connection …… n neurons

  17. Steps 1. Initialize the weights randomly. 2. Ik is the vector presented at kth iteration. 3. Find W* such that |w* - Ik| < |wj - Ik| for all j 4. make W*(new) = W* (old) + η(Ik - w* ). 5 K  K +1 ; if go to 3. 6. Go to 2 until the error is below a threshold.

  18. Two part assignment Supervised Hidden layer

  19. Cluster Discovery By SOM/Kohenen Net 4 I/p neurons A4 A1 A2 A3

  20. NeoCognitron(Fukusima et. al.)

  21. Hierarchical feature extraction based

  22. Corresponding Netowork

  23. S-Layer • Each S-layer in the neocognitron is intended for extraction of features from corresponding stage of hierarchy. • Particular S-layers are formed by distinct number of S-planes. Number of these S-planes depends on the number of extracted features.

  24. V-Layer • Each V-layer in the neocognitron is intended for obtaining of informations about average activity in previous C-layer (or input layer). • Particular V-layers are always formed by only one V-plane.

  25. C-Layer • Each C-layer in the neocognitron is intended for ensuring of tolerance of shifts of features extracted in previous S-layer. • Particular C-layers are formed by distinct number of C-planes. Their number depends on the number of features extracted in the previous S-layer.

More Related