SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Kernel Methods: Non-linear Mapping and Property Analysis

burgess avatar
burgess
9 diapositives · 220 Views · Dec 20, 2012

Kernel methods use non-linear mappings in finite, infinite, and Hilbert spaces. The kernel trick simplifies computations using Gram matrices in algorithms such as SVM and kernel PCA. Key topics include kernel selection, regularization to prevent overfitting, and optimizing kernels through cross-validation and Bayesian methods.

#kernel methods #nonlinear mapping #gram matrix #hilbert space #kernel trick #regularization #svm #feature selection
Kernel Methods: Non-linear Mapping and Property Analysis - slide 1
01 / 9Kernel Methods: Non-linear Mapping and Property Analysis
1 / 17

Télécharger la présentation

Kernel Methods: Non-linear Mapping and Property Analysis
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Kernel-class Jan. 13 2005

  2. Recap: Feature Spaces non-linear mapping to F 1. high-D space 2. infinite-D countable space : 3. function space (Hilbert space) example:

  3. Recap: Kernel Trick Note: In the dual representation we used the Gram matrix to express the solution. Kernel Trick: Replace : kernel If we use algorithms that only depend on the Gram-matrix, G, then we never have to know (compute) the actual features This is the crucial point of kernel methods

  4. Recap: Properties of a Kernel Definition:A finitely positive semi-definite function is a symmetric function of its arguments for which matrices formed by restriction on any finite subset of points is positive semi-definite. Theorem:A function can be written as where is a feature map iff k(x,y) satisfies the semi-definiteness property. Relevance: We can now check if k(x,y) is a proper kernel using only properties of k(x,y) itself, i.e. without the need to know the feature map!

  5. Reproducing Kernel Hilbert Spaces The proof of the above theorem proceeds by constructing a very special feature map (note that more feature maps may give rise to a kernel) i.e. we map to a function space. definition function space: reproducing property:

  6. Modularity Kernel methods consist of two modules: 1) The choice of kernel (this is non-trivial) 2) The algorithm which takes kernels as input Modularity: Any kernel can be used with any kernel-algorithm. some kernel algorithms: - support vector machine - Fisher discriminant analysis - kernel regression - kernel PCA - kernel CCA some kernels:

  7. Niceties and Challenges • Niceties: • Kernel algorithms are typically constrained convex optimization • problems  solved with either spectral methods or convex optimization tools. • Efficient algorithms do exist in most cases. • The similarity to linear methods facilitates analysis. There are strong • generalization bounds on test error. • Challenges: • You need to choose the appropriate kernel • Kernel learning is prone to over-fitting • All information must go through the kernel-bottleneck.

  8. Regularization • regularization is very important! • regularization parameters typically determined by out of sample. • measures (cross-validation, leave-one-out). Example: Gaussian Kernel: if c is very small: G=I (all data are dissimilar): over-fitting if c is very large: G=1 (all data are very similar): under-fitting In RKHS view we compute overlap between 2 Gaussians with width “c”. Demo Trevor Hastie.

  9. cone k1 k2 Learning Kernels • All information is tunneled through the Gram-matrix information • bottleneck. • The real art is to pick an appropriate kernel for the data domain. • Warning: Since kernels can overfit, we need to regularize. Solution: We need to learn the kernel. Here is some ways to combine kernels to improve them: any positive polynomial parameters can be set by i) cross-validation, ii) Bayesian methods, iii) test-error bound minimization.