120 likes | 293 Vues
Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2010. Lecture 3 September 27, 2010. Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall10/ann/. Plan. Multilayer Perceptron Architecture Signal Flow Learning rule - Backpropagation
E N D
Artificial Neural NetworksECE.09.454/ECE.09.560Fall 2010 Lecture 3September 27, 2010 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall10/ann/
Plan • Multilayer Perceptron • Architecture • Signal Flow • Learning rule - Backpropagation • Matlab MLP Demo • Finish Lab Project 1 • Start Lab Project 2
Hidden Layers Input Layer j j Output Layer 1 x1 j j j y1 1 Outputs x2 Inputs j j j y2 1 x3 wlk j j wji wkj Multilayer Perceptron (MLP): Architecture
1 j(t) 0.5 0 -1 1 t MLP: Characteristics • Neurons possess sigmoidal (logistic) activation functions • Contains one or more “hidden layers” • Trained using the “backpropagation” algorithm • MLP with 1-hidden layer is a “universal approximator”
Artificial Neural Network • Massively parallel distributed processor made up of simple processing units, which can store and retrieve experiential knowledge • The network “learns” from the data presented to it • The “knowledge” is stored in the interconnection weights Adapted from Haykin
Function signal Error signal Forward propagation Backward propagation MLP: Signal Flow j j j • Computations at each node, j • Neuron output, yj • Gradient vector, dE/dwji
k j i Right Left Backpropagation Notation At a node j, dj(n) yj(n) vj(n) wji(n) yi(n) ej(n) j(.) -1
k j i Right Left Backprop. (contd) Notation If node j is a hidden node, dk(n) vk(n) yj(n) yk(n) vj(n) wji(n) wkj(n) yi(n) ek(n) j(.) -1 j(.)
MLP Training k j i Right Left • Forward Pass • Fix wji(n) • Compute yj(n) • Backward Pass • Calculate dj(n) • Update weights wji(n+1) y x k j i Right Left
MLP’s in Matlab http://engineering.rowan.edu/~shreek/fall10/ann/demos/mlp.m
Lab Projects 1 and 2 • http://engineering.rowan.edu/~shreek/fall10/ann/lab1.html • http://engineering.rowan.edu/~shreek/fall10/ann/lab2.html