Learning to Sense Sparse Signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization

1. Learning to Sense Sparse Signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization Julio Martin Duarte-Carvajalino, and Guillermo Sapiro University of Minnesota IEEE Transactions on Image Processing, Vol. 18, No. 7, July 2009 Presented by Haojun Chen

2. Outline • Introduction • Sensing Matrix Learning • KSVD Algorithm • Coupled-KSVD • Experiment Results • Conclusion

3. Introduction • Compressive Sensing(CS) • Two fundamental principles • Sparsity • Incoherent Sampling Gramm Matrix: is with all columns normalized Gramm matrix should be as close to the identity as possible = S non-zero N x N m x 1 m x N N x 1 Image source: www.usna.edu/Users/weapsys/avramov/Compressed%20sensing%20tutorial/cs1v4.ppt

4. Sensing Matrix Learning • Assume the dictionary is known, the goal is to find the sensing matrix such that • Let be the eigen-decomposition of , then • Define • Objective is to compute to minimize • Let be the eigenvalues of , , , • Solution: ,

5. Sensing Matrix Learning • Replacing back in terms of (rows of ) • Once we obtain , • Algorithm summary

6. KSVD Algorithm • The objective of the KSVD algorithm is to solve, for a given sparsity level S, • Two stages in KSVD algorithm • Sparse Coding Stage: Using MP or BP • Dictionary Update Stage • Let and

7. KSVD Algorithm • Define the group of examples that use the atom • Let , then • Let be the SVD of and define • Solution:

8. KSVD Algorithm • KSVD algorithm consists of the following key steps: • Initialize • Repeat until convergence: • Sparse Coding Stage: For fixed, solve using OMP to obtain • Dictionary Update Stage: For j=1 to K • Define the group of examples that use this atom where P is the number of training square patches and • Let where • Obtain the largest singular value of and the corresponding singular vectors • Update using

9. Coupled-KSVD • To simultaneously training a dictionary and the projection matrix , the following optimization problem is considered • Define , then the above equation can be rewritten as • Solution obtained from KSVD: where and

10. Coupled-KSVD • Coupled-KSVD algorithm consists of the following key steps: • Initialize • Repeat until convergence: • For fixed, compute using the algorithm in sensing matrix learning • For fixed, solve using OMP to obtain • For j=1 to K • Define the group of examples that use this atom • where P is the number of training square patches and • Let • where • Obtain the largest singular value of and the corresponding singular vectors • Update using

11. Experiment Strategies • Uncoupled random (UR) • Uncoupled learning (UL) • Coupled random (CR) • Coupled learning (CL)

12. Experiment Results • Training data: 6600 8 x 8 patches extracted at random from 440 images • Testing data 120000 8 x 8 patches from 50 images Comparison of the average MSE of retrieval for the testing patches at different noise level and α K=64 Complete K=256 Overcomplete

13. Experiment Results Comparison of the retrieval MSE ratio for CL/CR and CL/UL at different noise level and α K=64 Complete K=256 Overcomplete

14. Experiment Results Best values of that produced the minimum retrieval MSE and at the same time the best CL/CR and CL/UL ratios, for a representative noise level of 5%.

15. Experiment Results Testing image consisting of non-overlapping 8 × 8 patches reconstructed from their noisy projections (5% level of noise)

16. Experiment Results Distribution of the off-diagonal elements of the Gramm matrix for each one of four strategies

17. Conclusions • Framework for learning optimal sensing matrix for given sparsifying dictionary was introduced • Novel approach for simultaneously learning the sensing matrix and sparsifying dictionary was proposed