1 / 24

차원감소방법의 개요

차원감소방법의 개요. 박 정 희 충남대학교 컴퓨터공학과. Outline. dimension reduction dimension reduction methods Linear dimension reduction Nonlinear dimension reduction Graph-based dimension reduction Applications. Dimension reduction. features. dimenion reduction ( 차원감소 ). feature extraction ( 특징추출 )

rristau
Télécharger la présentation

차원감소방법의 개요

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 차원감소방법의 개요 박 정 희 충남대학교 컴퓨터공학과

  2. Outline • dimension reduction • dimension reduction methods • Linear dimension reduction • Nonlinear dimension reduction • Graph-based dimension reduction • Applications

  3. Dimension reduction features dimenion reduction (차원감소) • feature extraction • (특징추출) • feature selection • (특징선택) objects • dimension: the number of features

  4. Dimension reduction • Reduce the dimensionality of high dimensional data • Identify new meaningful underlying features • The computational overhead of the subsequent processing stages is reduced • Reduce noise effects • Visualization of the data

  5. Dimension reduction methods • PCA(principal component analysis) • LDA(Linear discriminant analysis) • LLE(Locally linear embedding) • Isomap • LPP(Locality preserving projection) • UDP(Unsupervised discriminant projection) • Kernel based nonlinear dimension reduction • …….

  6. Linear dimension reduction W: a matrix of size mxp Rm Rp a Wta • How to find p column vectors of W? - use a training data set - what is objective criteria? • Traditional linear dimension reduction methods - PCA(principal component analysis) - LDA(Linear discriminant analysis) (p << m) =

  7. Principal component analysis • Minimize information loss by dimension reduction • Capture as much of variability as possible

  8. PCA (or Karhunen-Loéve transform) • given data set {a1,┉,an } , centroid • total scatter matrix • Find a projection vector w to maximize • Find the eigenvector w corresponding to the • largest eigenvalue of St : • eigenvectors w1, … ,wpcorresponding to the p largest • eigenvalues of St -> reduction to p-dimensional space

  9. Linear Discriminant analysis(LDA) • PCA seeks directions which are efficient for representing data • LDA seeks direction which are useful for discriminating between data in different classes

  10. LDA • given data set {a1,┉,an} with class label information , global centroid class centroid • Within-class • scatter matrix • Between-class • scatter matrix • Maximize between-class scatter and • minimize within-class scatter • Solve the generalized eigenvalue problem

  11. Nonlinear dimension reduction • It is difficult to represent nonlinearly structured data by linear dimension reduction

  12. Locally linear embedding(LLE) • Expectation: each data point and its neighbors lie on a locally linear patch of the manifold • [reference] Nonlinear dimensionality reduction by locally linear embedding, A.T. Roweis and L.K. Saul, Science, vol.290, pp.2323-2326, 2000

  13. LLE • Algorithm • Find the nearest neighbors of each data point • Express each data point xi as a linear combination of its neighbors wij = 0 if xj is not a near neighbor of xi 3. Find the coordinates yi of each point xi in lower-dimensional space by using the weights wij found in step 2 minimize

  14. Isomap • Preserve intrinsic geometry of the data, as captured in the geodesic manifold distances between all pairs of data points • [reference] A Global Geometric Framework for Nonlinear Dimensionality Reduction, J. B. Tenenbaum, V. de Silva and J. C. Langford, Science, vol.290, pp.2319-2323, 2000

  15. Isomap • Algorithm • Find the nearest neighbors of each data point and create a weighted graph by connecting a point to its nearest neighbors • Redefine the distances between points to be the length of the shortest path between the two points • Apply classical MDS(Multidimensional scaling)

  16. Example by Isomap

  17. Kernel-based dimension reduction F x12 2 x1x2 x22 linear dimension reduction F(x) = How to define a nonlinear mapping F?

  18. Kernel methods • If a kernel function k(x,y) satisfies Mercer’s condition, • then there exists a mapping   A (A) < x, y > < (x), (y)>= k(x,y) • Gaussian kernel • Polynomial kernel k(x,y) = (<x,y>+ß)d • As long as an algorithm can be written in terms of • inner products, it can be performed on the feature space

  19. Nonlinear discriminant analysis • As long as an algorithm can be written in terms of inner products, it can be performed on the feature space • In LDA, • LDA written in terms of inner products

  20. Graph-based dimension reduction • Graph G=<X, W> X={xi, 1  i n} : n nodes of data points W={ wij }1  i,j  n : similarity (or weight) matrix W can be sparse by using -neighborhoods or k nearest neighbors for edge construction • Let yi be the embedding of xi • [LPP(Locality preserving projection)] ensures that if xi and xj are close then yi and yjare close as well minimize

  21. Graph-based methods • Let yi be the embedding of xi minimize • Linearization: minimize • Using a penalty graph <X, P=(pij )> maximize

  22. Applications: Face recognition ? … … dimension reduction … … … … high data dimensionality

  23. Applications: text mining • Each document becomes a `term' vector, • each term is a component (attribute) of the vector, • the value of each component is the number of times the corresponding term occurs in the document. • Document data has high dimensionality

  24. Reference • Introduction to data mining, P.Tan, M. Steinbach and V. Kumar, Addison Wesley, 2006 • Pattern classification, R.Duda, P.Hart and D. Stork, Wiley-interscience, 2001 • [LPP] Locality preserving projections, X.He and P.Niyogi, Proc. conf. Neural information processing systems, 2003 • Graph embedding and extensions: a general framework for dimensionality reduction, S. Yan, D. Xu, H. Zhang, Q. Yang and S. Lin, IEEE transactions on pattern analysis and machine intelligence, Vol. 29(1), 2007

More Related