1 / 68

Exploring Perceptrons and Optimization Theories in Neural Networks

This chapter delves into Rosenblatt's perceptron model, focusing on the proofs of key theorems and methodologies in neural network optimization. It discusses the stochastic approximation method and sigmoid approximations of indicator functions, as well as potential functions and radial basis functions (RBFs). Key optimization theories, including Fermat's theorem, Lagrange multipliers, and the Kuhn-Tucker conditions, are examined in both deterministic and stochastic settings. The text emphasizes the learning process and the application of these theories in estimating coefficients and recognizing patterns.

jeri
Télécharger la présentation

Exploring Perceptrons and Optimization Theories in Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 9Perceptrons and their generalizations

  2. Rosenblatt’s perceptron • Proofs of the theorem • Method of stochastic approximation and sigmoid approximation of indicator functions • Method of potential functions and Radial basis functions • Three theorem of optimization theory • Neural Networks

  3. Perceptrons (Rosenblatt, 1950s)

  4. Recurrent Procedure

  5. Proofs of the theorems

  6. Method of stochastic approximation and sigmoid approximation of indicator functions

  7. Method of Stochastic Approximation

  8. Sigmoid Approximation of Indicator Functions

  9. Basic Frame for learning process • Use the sigmoid approximation at the stage of estimating the coefficients • Use the indicator functions at the stage of recognition.

  10. Method of potential functions and Radial Basis Functions

  11. Potential function • On-line • Only one element of the training data • RBFs (mid-1980s) • Off-line

  12. Method of potential functions in asymptotic learning theory • Separable condition • Deterministic setting of the PR • Non-separable condition • Stochastic setting of the PR problem

  13. Deterministic Setting

  14. Stochastic Setting

  15. RBF Method

  16. Three Theorems of optimization theory • Fermat’s theorem (1629) • Entire space, without constraints • Lagrange multipliers rule (1788) • Conditional optimization problem • Kuhn-Tucker theorem (1951) • Convex optimizaiton

  17. To find the stationary points of functions • It is necessary to solve a system of n equations with n unknown values.

  18. Lagrange Multiplier Rules (1788)

More Related