SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Algorithm for Sparse Non-negative Matrix Factorization

lorne avatar
lorne
12 diapositives · 249 Views · Dec 22, 2013

Presents a new efficient algorithm for sparse Non-negative Matrix Factorization (NMF) based on the non-negative version of the Least Angle Regression and Selection (LARS) algorithm. The algorithm controls sparsity using L1 as a convex proxy for the L0 norm, allowing for easy interpretation and model selection. It outperforms previous methods in degree of sparsity control and computational efficiency, particularly when utilizing NLARS. The method can be extended to tensor decompositions like Tucker and PARAFAC. Carried out in collaboration with PhD Kristoffer Hougaard Madsen and Professor Lars Kai Hansen at the Technical University of Denmark.

Algorithm for Sparse Non-negative Matrix Factorization - slide 1
01 / 12Algorithm for Sparse Non-negative Matrix Factorization
1 / 17

Télécharger la présentation

Algorithm for Sparse Non-negative Matrix Factorization
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Morten Mørup Informatics and Mathematical Modeling Technical University of Denmark Approximate L0 constrained NMF/NTF Work done in collaboration with PhD Kristoffer Hougaard Madsen Informatics and Mathematical Modeling Technical University of Denmark Professor Lars Kai Hansen Informatics and Mathematical Modeling Technical University of Denmark Morten Mørup

  2. Non-negative Matrix Factorization (NMF)VWH, V≥0,W≥0, H≥0  (Lee & Seung – Nature 1999) NMF gives Part based representation! Morten Mørup

  3. NMF based on Multiplicative updates Step size parameter Morten Mørup

  4. fast Non-Negative Least Squares, fNNLS Active Set procedure (Lawson and Hanson, 1974) Morten Mørup

  5. NMF not in general unique!! V=WH=(WP)(P-1H)=W’H’ (Donoho & Stodden, 2003) Morten Mørup

  6. FIX: Impose sparseness (Hoyer, 2001,2004 Eggert et al. 2004) • Ensures uniqueness • Eases interpretability (sparse representation  factor effects pertain to fewer dimensions) • Can work as model selection(Sparseness can turn off excess factors by letting them become zero) • Resolves over complete representations (when model has many more free variables than data points) L1 used as convex proxy for the L0 norm, i.e. card(H) Morten Mørup

  7. Least Angle Regression and Selection(LARS)/Homotopy Method     Morten Mørup

  8. Controlling sparsity degree (Patric Hoyer 2004) Controlling sparsity degree (Mørup et al., 2008) Sparsity can now be controlled by evaulating the full regularization path of the NLARS Morten Mørup

  9. New Algorithm for sparse NMF: 1: Solve for each column of H using NLARS and obtain solutions for all values of  (i.e. the entire regularization path) 2: Select -solution giving the desired degree of sparsity 3: Update W such that ||Wd||F=1, according to (Eggert et al. 2004) Repeat from step 1 until convergence Morten Mørup

  10. CBCL face database USPS handwritten digits Morten Mørup

  11. Morten Mørup

  12. Conclusion • New efficient algorithm for sparse NMF based on the proposed non-negative version of the LARS algorithm • The obtained full regularization path admit to use L1 as a convex proxy for the L0 norm to control the degree of sparsity given by • The proposed method is more efficient than previous methods to control degree of sparsity. Furhtermore, NLARS is even comparable in speed to the classic efficient fNNLS method. • Proposed method directly generalizes to tensor decompositions through models such as Tucker and PARAFAC when using an alternating least squares approach. Morten Mørup