SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Non-Parametric Learning

reese-roberson avatar
reese-roberson
17 diapositives · 398 Views · Nov 16, 2014

Non-Parametric Learning. Prof. A.L. Yuille Stat 231. Fall 2004. Chp 4.1 – 4.3. Parametric versus Non-Parametric. Previous lectures on MLE learning assumed a functional form for the probability distribution. We now consider an alternative non-parametric method based on window function.

Non-Parametric Learning - slide 1
01 / 17Non-Parametric Learning
1 / 17

Télécharger la présentation

Non-Parametric Learning
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Non-Parametric Learning Prof. A.L. Yuille Stat 231. Fall 2004. Chp 4.1 – 4.3.

  2. Parametric versus Non-Parametric • Previous lectures on MLE learning assumed a functional form for the probability distribution. • We now consider an alternative non-parametric method based on window function.

  3. Non-Parametric • It is hard to develop probability models for some data. • Example: estimate the distribution of annual rainfall in the U.S.A. Want to model p(x,y) – probability that a raindrop hits a position (x,y). • Problems: (i) multi-modal density is difficult for parametric models, (ii) difficult/impossible to collect enough data at each point (x,y).

  4. Intuition • Assume that the probability density is locally smooth. • Goal: estimate the class density model p(x) from data • Method 1: Windows based on points x in space.

  5. Windows • For each point x, form a window centred at x with volume Count the number of samples that fall in the window. • Probability density is estimated as:

  6. Non-Parametric • Goal: to design a sequence of windows so that at each point x • (f(x) is the true density). • Conditions for window design: • increasing spatial resolution. (ii) many samples at each point (iii)

  7. Two Design Methods • Parzen Window: Fix window size: • K-NN: Fix no. samples in window:

  8. Parzen Window • Parzen window uses a window function • Example: • (i) Unit hypercube: and 0 otherwise. • (ii) Gaussian in d-dimensions.

  9. Parzen Windows • No. of samples in the hypercube is • Volume • The estimate of the distribution is: • More generally, the window interpolates the data.

  10. Parzen Window Example • Estimate a density with five modes using Gaussian windows at scales h=1,0.5, 0.2.

  11. Convergence Proof. • We will show that the Parzen window estimator converges to the true density at each point x with increasing number of samples.

  12. Proof Strategy. • Parzen distribution is a random variable which depends on the samples used to estimate it. • We have to take the expectation of the distribution with respect to the samples. • We show that the expected value of the Parzen distribution will be the true distribution. And the expected variance of the Parzen distribution will tend to 0 as no. samples gets large.

  13. Convergence of the Mean • Result follows.

  14. Convergence of Variance • Variance:

  15. Example of Parzen Window • Underlying density is Gaussian. Window volume decreases as

  16. Example of Parzen Window • Underlying Density is bi-modal.

  17. Parzen Window and Interpolation. • In practice, we do not have an infinite number of samples. • The choice of window shape is important. This effectively interpolates the data. • If the window shape fits the local structure of the density, then Parzen windows are effective.