Compressive Sensing Techniques for Video Acquisition

1. Compressive Sensing Techniques for Video Acquisition EE5359 Multimedia Processing December 8,2009 Madhu P. Krishnan

2. Contents • Introduction to Image Acquisition • Problem Statement • Compressive Sensing • Concept • System • Results • Conclusion • References

3. Introduction to Image Acquisition • Long-established paradigm for Digital image acquisition – Sample the complete image to get N pixel values – Represent the sampled image in some transform domain – Discard the non-significant coefficients(N – K) in the transform domain. –Transmit/store Fig.1Digital image acquisition system

4. Problem Statement • Is it possible to create an efficient sensing process where we economize on the number of pixel measurements required and to reconstruct the scene, provided that we are not interested in the perfect reconstruction of the whole scene ?

5. Compressive Sensing • A framework that enables sampling below Nyquist rate, with a small sacrifice in reconstruction quality. • Compressive sampling shows us how image compression can be implicitly incorporated into the image acquisition process . Fig.2Compressive sensing based data acquisition system

6. Concept • Let x = {x[1], . . . ,x[N]} be a set of N pixels of an image. Let s be the representation of ‘x’ in the transform domain, that is: • Let y be an M-length measurement vector given by: , where is a M×N measurement matrix(independent identically distributed (i.i.d.) Gaussian matrix). The above expression can be written in terms of s as:

7. Concept • Unfortunately, reconstruction of x = {x[1], . . . ,x[N]} (or equivalently, s = {s[1], . . . , s[N]}) from vector y of M samples is not unique. • However, excellent approximation can be obtained via the l1 norm minimization given by:

8. System Fig.3 Block diagram of the acquisition process[8].

9. Results • The system described in Fig.3 is applied to Y-components of the QCIF Akiyo and CIF Stefan sequences. • The sparse blocks are identified using DCT in the following manner. Let C be a small positive constant, and Th an integer threshold that is representative of the average number of significant DCT coefficients over all blocks. If the number of DCT coefficients in the block whose absolute value is larger than C is greater than Th, the block is selected as a reference for compressive sampling. • Fig.4 and Fig.5 shows the number of DCT coefficients less than C for the first frame of Akiyo and Stefan sequence.

10. Results Fig.4 Sparsity determination (Here C = 4 and Th = 100 are chosen as values to determine sparsity).

11. Results Fig.5 Sparsity determination (Here C = 4 and Th = 400 are chosen as values to determine sparsity).

12. Results • The 9th frame of the Akiyo and 3rd frame of Stefan is compressively sampled, with their respective first frames used as reference. The results are shown as PSNR verses the percentage of the collected samples for a fixed Th. Tab.1 Percentage samples vs PSNR(dB) for Akiyo and Stefan

13. Results Fig.69th frame reconstructed from 20% of pels from selected blocks. Fig.79th frame reconstructed from 40% of pels from selected blocks.

14. Results Fig.8 9th frame reconstructed from 60% of pels from selected blocks Fig.9 3rd frame reconstructed from 20% of pels from selected blocks

15. Results Fig.10 3rd frame reconstructed from 40% of pels from selected blocks Fig.11 3rd frame reconstructed from 60% of pels from selected blocks

16. Conclusion • 80% savings in acquisition can be achieved for video sequences like Akiyo, that are mostly static across frames , with good reconstruction quality. • The results on Stefan sequence shows that for scenes with increased dynamics more pixels have to sampled(in this case 80%) for good reconstruction quality. • Simplification of the subsequent processing algorithms.

17. References [1] R. G. Baraniuk, "Compressive Sensing“, Lecture Notes in IEEE Signal Processing Magazine, Vol. 24, pp. 118-120, July 2007. [2] E. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information”, IEEE Trans. Inform. Theory, vol.52, pp. 489–509, Feb. 2006. [3] D. Donoho, “Compressed sensing”, IEEE Trans. Inform. Theory, vol. 52, pp. 1289-1306, Apr. 2006. [4] E. Candès and M. Wakin, “An introduction to compressive sampling”, IEEE Signal Processing Magazine, vol. 25, pp. 21 - 30, March 2008. [5] D.L. Donoho et al. “Data compression and harmonic analysis”, IEEE Trans. Inform. Theory, vol. 44, pp. 2435–2476, Oct. 1998.

18. References [6] M. Vetterli and J. Kovacevic, Wavelets and Subband Coding. Englewood Cliffs, NJ: Prentice-Hall, 1995. [7] J. Romberg, “Imaging via compressive sampling”, IEEE Signal Processing Magazine, vol. 25, pp. 14 - 20, March 2008. [8] V. Stankovic, L. Stankovic, and S. Cheng, “Compressive video sampling”, Proc. Eusipco-2008 16th European Signal Processing Conference, Lausanne, Switzerland, August 2008. [9] J. Tropp and A. C. Gilbert, “Signal recovery from partial information via orthogonal matching pursuit”, IEEE Trans. Info. Theory, vol. 53, pp. 4655--4666, Dec. 2007. [10] N. Ahmed, T. Natarajan, and K. R. Rao, "Discrete Cosine Transform", IEEE Trans. Computers, Vol C-23, pp. 90-93, Jan 1974.