Ron DeVore University of South Carolina Richard Baraniuk Rice University - PowerPoint PPT Presentation

slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Ron DeVore University of South Carolina Richard Baraniuk Rice University PowerPoint Presentation
Download Presentation
Ron DeVore University of South Carolina Richard Baraniuk Rice University

play fullscreen
1 / 93
Ron DeVore University of South Carolina Richard Baraniuk Rice University
240 Views
Download Presentation
elijah
Download Presentation

Ron DeVore University of South Carolina Richard Baraniuk Rice University

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Compressive Sensing Theory and Applications Ron DeVore University of South Carolina Richard Baraniuk Rice University

  2. Agenda • Introduction to Compressive Sensing (CS) [richb] • motivation • basic concepts • CS Theoretical Foundation [ron] • sparsity • encoding and decoding • restricted isometry principle (RIP) • recovery algorithms • CS Applications [richb]

  3. Compressive SensingIntroduction and Background

  4. Sensing

  5. Digital Revolution

  6. Pressure is on Digital Sensors • Success of digital data acquisition is placing increasing pressure on signal/image processing hardware and software to support higher resolution / denser sampling • ADCs, cameras, imaging systems, microarrays, … x large numbers of sensors • image data bases, camera arrays, distributed wireless sensor networks, … xincreasing numbers of modalities • acoustic, RF, visual, IR, UV, x-ray, gamma ray, …

  7. Pressure is on Digital Sensors • Success of digital data acquisition is placing increasing pressure on signal/image processing hardware and software to support higher resolution / denser sampling • ADCs, cameras, imaging systems, microarrays, … x large numbers of sensors • image data bases, camera arrays, distributed wireless sensor networks, … xincreasing numbers of modalities • acoustic, RF, visual, IR, UV = deluge of data • how to acquire, store, fuse, process efficiently?

  8. Digital Data Acquisition • Foundation: Shannon sampling theorem “if you sample densely enough(at the Nyquist rate), you can perfectly reconstruct the original data” time space

  9. Sensing by Sampling • Long-established paradigm for digital data acquisition • uniformly sampledata at Nyquist rate (2x Fourier bandwidth) sample

  10. Sensing by Sampling • Long-established paradigm for digital data acquisition • uniformly sampledata at Nyquist rate (2x Fourier bandwidth) too much data! sample

  11. Sensing by Sampling • Long-established paradigm for digital data acquisition • uniformly sampledata at Nyquist rate (2x Fourier bandwidth) • compress data (signal-dependent, nonlinear) sample compress transmit/store sparsewavelettransform receive decompress

  12. Sparsity / Compressibility largewaveletcoefficients largeGaborcoefficients pixels widebandsignalsamples frequency time

  13. What’s Wrong with this Picture? • Long-established paradigm for digital data acquisition • sampledata at Nyquist rate (2x bandwidth) • compressdata (signal-dependent, nonlinear) • brick wall to resolution/performance sample compress transmit/store sparse /compressiblewavelettransform receive decompress

  14. Beyond Uniform Sampling • Recall Shannon/Nyquist theorem • Shannon was a pessimist • 2x oversampling Nyquist rate is a worst-case bound for any bandlimited data • sparsity/compressibility irrelevant • Shannon sampling is a linear process while compression is a nonlinear process

  15. Compressive Sensing (CS) • Recall Shannon/Nyquist theorem • Shannon was a pessimist • 2x oversampling Nyquist rate is a worst-case bound for any bandlimited data • sparsity/compressibility irrelevant • Shannon sampling is a linear process while compression is a nonlinear process • Compressive sensing • new sampling theory that leverages compressibility • based on new uncertainty principles • randomness plays a key role

  16. Compressive Sensing • Directly acquire “compressed” data • Replace samples by more general “measurements” compressive sensing transmit/store receive reconstruct

  17. Sampling • Signal is -sparse in basis/dictionary • WLOG assume sparse in space domain sparsesignal nonzeroentries

  18. Sampling • Signal is -sparse in basis/dictionary • WLOG assume sparse in space domain • Samples sparsesignal measurements nonzeroentries

  19. Compressive Data Acquisition • When data is sparse/compressible, can directly acquire a condensed representation with no/little information lossthrough dimensionality reduction sparsesignal measurements nonzero entries

  20. Compressive Data Acquisition • When data is sparse/compressible, can directly acquire a condensed representation with no/little information loss • Random projection will work sparsesignal measurements nonzero entries [Candes-Romberg-Tao, Donoho, 2004]

  21. Compressive Sensing • Directly acquire “compressed” data • Replace samples by more general “measurements” random measurements transmit/store … receive reconstruct

  22. Why Does It Work? • Random projection not full rank…… and so loses information in general

  23. Why Does It Work? • Random projection not full rank… … but preserves structureand informationin sparse/compressible signals models with high probability

  24. Why Does It Work? • Random projection not full rank… … but preserves structureand informationin sparse/compressible signals models with high probability K-dim hyperplanesaligned with coordinate axes K-sparsemodel

  25. Why Does It Work? • Random projection not full rank… … but preserves structureand informationin sparse/compressible signals models with high probability K-sparsemodel

  26. CS Signal Recovery • Random projection not full rank… … but is invertiblefor sparse/compressible signals models with high probability(solves ill-posed inverse problem) K-sparsemodel recovery

  27. CS Signal Recovery • Random projection not full rank • Recovery problem:givenfind • Null space • So search in null space for the “best” according to some criterion • ex: least squares (M-N)-dim hyperplaneat random angle

  28. CS Signal Recovery • Reconstruction/decoding: given(ill-posed inverse problem) find • L2 fast

  29. CS Signal Recovery • Reconstruction/decoding: given(ill-posed inverse problem) find • L2 fast, wrong

  30. Why L2 Doesn’t Work least squares,minimum L2 solutionis almost never sparse for signals sparse in the space/time domain null space of translated to(random angle)

  31. CS Signal Recovery • Reconstruction/decoding: given(ill-posed inverse problem) find • L2 fast, wrong • L0 number ofnonzeroentries

  32. CS Signal Recovery • Reconstruction/decoding: given(ill-posed inverse problem) find • L2 fast, wrong • L0correct, slowonly M=2K measurements required to perfectly reconstruct K-sparse signal stably[Bresler; Rice] number ofnonzeroentries

  33. CS Signal Recovery • Reconstruction/decoding: given(ill-posed inverse problem) find • L2 fast, wrong • L0 correct, slow • L1correct, efficient mild oversampling[Candes, Romberg, Tao; Donoho] linear program

  34. Why L1 Works minimum L1solution= sparsest solution (with high probability) if for signals sparse in the space/time domain

  35. Universality • Random measurements can be used for signals sparse in any basis

  36. Universality • Random measurements can be used for signals sparse in any basis

  37. Universality • Random measurements can be used for signals sparse in any basis sparsecoefficient vector nonzero entries

  38. Compressive Sensing • Directly acquire “compressed” data • Replace samples by more general “measurements” random measurements transmit/store … receive linear pgm

  39. CS Hallmarks • CS changes the rules of the data acquisition game • exploits a priori signal sparsity information • Universal • same random projections / hardware can be used forany compressible signal class (generic) • Democratic • each measurement carries the same amount of information • simple encoding • robust to measurement loss and quantization • Asymmetrical(most processing at decoder) • Random projections weakly encrypted

  40. Theoretical Foundations ofCompressive Sensing[Ron DeVore]

  41. Applications of Compressive Sensing[Richard Baraniuk]

  42. Recall: CS Hallmarks • CS changes the rules of the data acquisition game • exploits a priori signal sparsity information • Universal • same random projections / hardware can be used forany compressible signal class (generic) • Democratic • each measurement carries the same amount of information • simple encoding • robust to measurement loss and quantization • Asymmetrical(most processing at decoder) • Random projections weakly encrypted

  43. Gerhard Richter 4096 Farben / 4096 Colours 1974254 cm X 254 cmLaquer on CanvasCatalogue Raisonné: 359 Museum Collection:Staatliche Kunstsammlungen Dresden (on loan) Sales history: 11 May 2004Christie's New York Post-War and Contemporary Art (Evening Sale), Lot 34US$3,703,500  

  44. Gerhard Richter Cologne CathedralStained Glass

  45. Rice Single-Pixel CS Camera single photon detector imagereconstructionorprocessing DMD DMD random pattern on DMD array

  46. Single Pixel Camera Object LED (light source) Lens 2 Lens 1 Photodiode circuit DMD+ALP Board

  47. Single Pixel Camera Object LED (light source) Lens 2 Lens 1 Photodiode circuit DMD+ALP Board

  48. Single Pixel Camera Object LED (light source) Lens 2 Lens 1 Photodiode circuit DMD+ALP Board

  49. Single Pixel Camera Object LED (light source) Lens 2 Lens 1 Photodiode circuit DMD+ALP Board