Beyond Nyquist: Compressed Sensing of Analog Signals

1. Beyond Nyquist: Compressed Sensing of Analog Signals Yonina Eldar Technion – Israel Institute of Technology http://www.ee.technion.ac.il/people/YoninaEldar yonina@ee.technion.ac.il Dagstuhl Seminar December, 2008

2. Sampling: “Analog Girl in a Digital World…” Judy Gorman 99 Digital world Analog world Sampling A2D Signal processing Denoising Image analysis… Reconstruction D2A (Interpolation)

3. Compression “Can we not just directly measure the part that will not end up being thrown away ?” Donoho Compressed 392 KB15% Compressed 148 KB6% Compressed 950 KB38% Original 2500 KB100%

4. Outline • Compressed sensing – background • From discrete to analog • Goals • Part I : Blind multi-band reconstruction • Part II : Analog CS framework • Implementations • Uncertainty relations Can break the Shannon-Nyquist barrier by exploiting signal structure

5. CS Setup • K non-zero entries at least 2K measurements • Recovery: brute-force, convex optimization, greedy algorithms, …

6. Brief Introduction to CS • Uniqueness: is uniquely determined by Donoho and Elad, 2003 with high probability is random Donoho, 2006 and Candès et. al., 2006 Recovery: Convex and tractable Donoho, 2006 and Candès et. al., 2006 Greedy algorithms: OMP, FOCUSS, etc. NP-hard Tropp, Elad, Cotter et. al,. Chen et. al., and many others

7. Naïve Extension to Analog Domain Standard CS Discrete Framework Analog Domain Sparsity prior what is a sparse analog signal ? Generalized sampling Continuoussignal Operator Infinite sequence Finite dimensional elements Stability Randomness  Infinitely many Random is stable w.h.p Need structure for efficient implementation Reconstruction Finite program, well-studied Undefined program over a continuous signal

8. Naïve Extension to Analog Domain Standard CS Discrete Framework Analog Domain • Questions: • What is the definition of analog sparsity ? • How to select a sampling operator ? • Can we introduce stucture in sampling and still preserve stability ? • How to solve infinite dimensional recovery problems ? Sparsity prior what is a sparse analog signal ? Generalized sampling Continuoussignal Operator Infinite sequence Finite dimensional elements Stability Randomness  Infinitely many Random is stable w.h.p Need structure for efficient implementation Reconstruction Finite program, well-studied Undefined program over a continuous signal

9. Goals • Concrete analog sparsity model • Reduce sampling rate (to minimal) • Simple recovery algorithms • Practical implementation in hardware

10. no more than N bands, max width B, bandlimited to • More generally only sequences are non-zero (Eldar 2008) Analog Compressed Sensing What is the definition of analog sparsity ? • A signal with a multiband structure in some basis • Each band has an uncountable number of non-zero elements • Band locations lie on an infinite grid • Band locations are unknown in advance (Mishali and Eldar 2007)

11. Multiband “Sensing” (Mishali and Eldar 2007) bands Sampling Reconstruction Analog Infinite Analog Goal: Perfect reconstruction We are interested in unknown spectral support (a union of subspace prior) • Known band locations (subspace prior): • Minimal-rate sampling and reconstruction (NB) with known band locations (Lin and Vaidyanathan 98) • Half blind system (Herley and Wong 99, Venkataramani and Bresler 00) • Next steps: • What is the minimal rate requirement ? • A fully-blind system design

12. Rate Requirements • The minimal rate is doubled • For , the rate requirement is samples/sec (on average) Theorem (blind recovery) Mishali and Eldar (2007) Theorem (non-blind recovery) Landau (1967) Average sampling rate

13. Sampling Multi-Coset: Periodic Non-uniform on the Nyquist grid In each block of samples, only are kept, as described by 2 Analog signal 0 Point-wise samples 0 3 3 2 0 3 2

14. The Sampler in vector form unknowns Length . known matrix known Observation: is sparse DTFT of sampling sequences Constant Problems: • Undetermined system – non unique solution • Continuous set of linear systems is jointly sparse and unique under appropriate parameter selection ( )

15. Paradigm Solve finiteproblem Reconstruct 0 S = non-zero rows 1 2 3 4 5 6

16. Continuous to Finite Solve finiteproblem Reconstruct CTF block MMV • span a finite space • Any basis preserves the sparsity Continuous Finite

17. 2-Words on Solving MMV Find a matrix U that has as few non-zero rows as possible • Variety of methods based on optimizing mixed column-row norms • We prove equivalence results by extending RIP and coherence to allow for structured sparsity (Mishali and Eldar, Eldar and Bolcskei) • New approach: ReMBo – Reduce MMV and Boost • Main idea: Merge columns of Vto obtain a single vector problem y=Aa • Sparsity pattern of a is equal to that of U • Can boost performance by repeating the merging with different coeff.

18. Algorithm Perfect reconstruction at minimal rate Blind system: band locations are unknown Can be applied to CS of general analog signals Works with other sampling techniques Continuous-to-finite block: Compressed sensing for analog signals CTF

19. Framework: Analog Compressed Sensing (Eldar 2008) Sampling signals from a union of shift-invariant spaces (SI) Subspace generators

20. Framework: Analog Compressed Sensing There is no prior knowledge on the exact indices in the sum What happen if only K<<N sequences are not zero ? Not a subspace ! Only k sequences are non-zero

21. Framework: Analog Compressed Sensing Step 1: Compress the sampling sequences Step 2: “Push” all operators to analog domain CTF System A High sampling rate = m/TPost-compression Only k sequences are non-zero

22. Framework: Analog Compressed Sensing Low sampling rate = p/TPre-compression System B CTF Theorem Eldar (2008)

23. Simulations Reconstruction filter Signal Amplitude Amplitude Output Time (nSecs) Time (nSecs)

24. Minimal rate Minimal rate Simulations Sampling rate Sampling rate Brute-Force M-OMP

25. Simulations 0% Recovery 100% Recovery 0% Recovery 100% Recovery Noise-free Sampling rate Sampling rate SBR4 SBR2 Empirical recovery rate

26. Multi-Coset Limitations Analog signal 2 Point-wise samples 0 0 3 3 2 0 3 2 Delay ADC @ rate • Impossible to match rate for wideband RF signals(Nyquist rate > 200 MHz) • Resource waste for IF signals 3. Requires accuratetime delays

27. Efficient Sampling (Mishali, Eldar, Tropp 2008) Efficientimplementation Use CTF

28. Hardware Implementation A few first steps…

29. Pairs Of Bases • Until now: sparsity in a single basis • Can we have a sparse representation in two bases? • Motivation: A combination of bases can sometimes better represent the signal Both and are small!

30. Uncertainty Relations • How sparse can be in each basis? • Finite setting: vector in Elad and Brukstein 2002 Different bases Uncertainty relation

31. Theorem Eldar (2008) Analog Uncertainty Principle Theorem Eldar (2008)

32. Bases With Minimal Coherence In the DFT domain Fourier Spikes What are the analog counterparts ? • Constant magnitude • Modulation • “Single” component • Shifts

33. Analog Setting: Bandlimited Signals • Minimal coherence: • Tightness:

34. Finding Sparse Representations • Given a dictionary , expand using as few elements as possible: minimize • Solution is possible using CTF if is small enough • Basic idea: • Sample with basis • Obtain an IMV model: maximal value • Apply CTF to recover • Can establish equivalence with as long as is small enough

35. Conclusion • Extend the basic results of CS to the analog setting - CTF • Sample analog signals at rates much lower than Nyquist • Can find a sparse analog representation • Can be implemented efficiently in hardware Questions: Other models of analog sparsity? Other sampling devices? Compressed Sensing of Analog Signals

36. Some Things Should Remain At The Nyquist Rate Thank you Thank you High-rate

37. References • M. Mishali and Y. C. Eldar, "Blind Multi-Band Signal Reconstruction: Compressed Sensing for Analog Signals,“ to appear in IEEE Trans. on Signal Processing. • M. Mishali and Y. C. Eldar, "Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors", IEEE Trans. on Signal Processing, vol. 56, no. 10, pp. 4692-4702, Oct. 2008. • Y. C. Eldar , "Compressed Sensing of Analog Signals", submitted to IEEE Trans. on Signal Processing. • Y. C. Eldar and M. Mishali, "Robust Recovery of Signals from a Union of Subspaces’’, submitted to IEEE Trans. on Inform. Theory. • Y. C. Eldar, "Uncertainty Relations for Analog Signals",  submitted to IEEE Trans. Inform. Theory. • Y. C. Eldar and T. Michaeli, "Beyond Bandlimited Sampling: Nonlinearities, Smoothness and Sparsity", to appear in IEEE Signal Proc. Magazine.