Information Criterion for Model Selection

Information Criterion for Model Selection Romain Hugues

Problem description DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Important parameters • We have N m-vector data a sampled from a m’-dim manifold A. • We want to estimate a d-dim manifold S. • S is parameterized by a n-vector u constrained to be in a n’-dim manifold U DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Model can be described by: • d: dimension • r (=m’-d) : codimension • n’ :degrees of freedom DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Minimization and expected residual Max. Lik. Solution of problem by minimizing J: New Notation for residual with respect to model: Residuals for future data a* : Expected residual of Model S: DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Mahalanobis projection of Data: DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Optimally fitted Manifold: DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Evaluation of expected residual: WE NEED TO ESTIMATE I(S) DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Geometric Information Criterion AIC(S) is an unbiased estimator of I(S): Extracting noise level ε from covariance: Normalized residual : Normalized AIC : DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Model Selection S1 ”better” than S2 if AIC0 (S1) <AIC0(S2) If model S1 is CORRECT DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Model Comparison S1 ”better” than S2 if AIC0 (S1) <AIC0(S2) DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

What should be done in practice? • Collect Data. • Estimate Manifolds and true positions for each model. • Compute Residuals for each model. • If a model is always “correct”, estimate noise level from residuals of this model • Compare two models: DESCRIPTION THEORETICAL BACKGROUND IN PRACTICE

Situations when this is useful ? • ??

Information Criterion for Model Selection