Download
introduction to radial basis function n.
Skip this Video
Loading SlideShow in 5 Seconds..
Introduction to Radial Basis Function PowerPoint Presentation
Download Presentation
Introduction to Radial Basis Function

Introduction to Radial Basis Function

339 Vues Download Presentation
Télécharger la présentation

Introduction to Radial Basis Function

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Introduction to Radial Basis Function Mark J. L. Orr

  2. Radial Basis Function Networks • Linear model

  3. Radial functions • Gassian RBF:c : center, r : radius • monotonically decreases with distance from center • Multiquadric RBF • monotonically increases with distance from center

  4. Gaussian RBF multiqradric RBF

  5. Least Squares • model • training data : {(x1, y1), (x2, y2), …, (xp, yp)} • minimize the sum-squared-error

  6. Example • Sample points (noisy) from the curve y = x : {(1, 1.1), (2, 1.8), (3, 3.1)} • linear model : f(x) = w1h1(x) + w2h2(x),where h1(x) = 1, h2(x) = x • estimate the coefficient w1, w2

  7. f(x) = x

  8. New model : f(x) = w1h1(x) + w2h2(x) + w3h3(x)where h1(x) = 1, h2(x) = x, h3(x) = x2

  9. absorb all the noise : overfit • If the model is too flexible, it will fit the noise • If it is too inflexible, it will miss the target

  10. The optimal weight vector • model • sum-squared-error • cost function : weight penalty term is added

  11. Example • Sample points (noisy) from the curve y = x : {(1, 1.1), (2, 1.8), (3, 3.1)} • linear model : f(x) = w1h1(x) + w2h2(x),where h1(x) = 1, h2(x) = x • estimate the coefficient w1, w2

  12. The projection matrix • At the optimal weight:the value of cost function C = yTPythe sum-squared-error S = yTP2y

  13. Model selection criteria • estimates of how well the trained model will perform on future input • standard tool : cross validation • error variance

  14. Cross validation • leave-one-out (LOO) cross-validation • generalized cross-validation

  15. Ridge regression • mean-squared-error

  16. Global ridge regression • Use GCV • re-estimation formula • initialize  • re-estimate , until convergence

  17. Local ridge regression • research problem

  18. Example

  19. Selection the RBF • forward selection • starts with an empty subset • added one basis function at a time • most reduces the sum-squared-error • until some chosen criterion stops • backward elimination • starts with the full subset • removed one basis function at a time • least increases the sum-squared-error • until the chosen criterion stops decreasing