1 / 15

Non-negative Matrix Factorization

Non-negative Matrix Factorization. Recent algorithms, extensions and available software Atina Dunlap Brooks (adbrook2@stat.ncsu.edu) North Carolina State University. Recent Algorithms. Lee & Seung’s multiplicative updates are easy to understand and to implement Can be very slow to converge

Télécharger la présentation

Non-negative Matrix Factorization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Non-negative Matrix Factorization Recent algorithms, extensions and available software Atina Dunlap Brooks (adbrook2@stat.ncsu.edu) North Carolina State University

  2. Recent Algorithms • Lee & Seung’s multiplicative updates are easy to understand and to implement • Can be very slow to converge • ALS can speed things up • Convergence theory is not particularly strong • Most NMF methods do not have robust convergence, but work well in practice

  3. Projected GradientDescent Method • Chih-Jen Lin (2007) • Bound-constrained optimization • Projected Gradient

  4. Projected GradientDescent Method • Can be applied to both the multiplicative updates and the ALS solution • Generally, greatest speed was achieved with the projected gradient combined with ALS

  5. Fast Non-NegativeMatrix Approximation • Kim, Sra & Dhillon (2007) • Employs Newton-type methods to solve NMF • Uses curvature information vs. gradient descent approach • Provide an exact method (good accuracy, but still slow) and a very fast inexact method

  6. References forAlgorithm Comparisons • Algorithms and Applications for Approximate Nonnegative Matrix Factorization by Berry, Browne, Langville, Pauca & Plemmons (2006) • Optimality, Computation, and Interpretations of Nonnegative Matrix Factorizations by Chu, Diele, Plemmons & Ragni (2004)

  7. Extensions • Tri-Factorization • Semi-NMF • Convex-NMF • Non-negative Tensor Factorization • Inferential Robust Matrix Factorization

  8. OrthogonalTri-factorization • Ding, Li, Peng & Park (2006) • Requiring orthogonality introduces uniqueness and improves clustering interpretations A = WSH, where WTW=I and HTH=I • W gives row clusters while H gives column clusters

  9. Semi-NMF • Ding, Li & Jordan (2006) • Allows A and W to contain negative values, but H is restricted to non-negative • Provides more flexibility (negative entries) and a clustering which is usually better than k-means

  10. Convex-NMF • Ding, Li & Jordan (2006) • Restricts W to be convex combinations of the columns of A • Ensures meaningful cluster centroids • W and H tend to be sparse

  11. Non-NegativeTensor Factorization • Uses n-way arrays instead of the 2-dimensional arrays used by NMF • Presentations during the workshop by Michael Berry and Bob Plemmons

  12. Inferential RobustMatrix Factorization • Fogel, Young, Hawkins & Ledirac (2007) • Uses the same method for robustness as Liu et al. (2003) for robust SVD • Paul Fogel will be presenting on an application

  13. Software - Matlab • Matlab Code • Patrik Hoyer • http://www.cs.helsinki.fi/u/phoyer/ • Includes Lee & Seung’s multiplicative updates and Hoyer’s sparseness • Chih-Jen Lin • http://www.csie.ntu.edu.tw/~cjlin/nmf/ • Includes projected gradient descent applied to multiplicative updates and ALS

  14. Software • C code – nnmf() • Simon Sheperd • http://www.simonshepherd.supanet.com/nnmf.htm • Very fast algorithm (as of 2004) • JMP script - irMF • Paul Fogel • http://www.niss.org/irMF/ • Inferential Robust Matrix Factorization

  15. Thank You

More Related