1 / 71

Richard Baraniuk Kevin Kelly Rice University

Compressive Optical Imaging Systems – Theory, Devices, Implementation. Richard Baraniuk Kevin Kelly Rice University. David Brady Rebecca Willett Duke University. Project Overview Richard Baraniuk. Digital Revolution. camera arrays. hyperspectral cameras. distributed camera networks.

ghazi
Télécharger la présentation

Richard Baraniuk Kevin Kelly Rice University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Compressive Optical Imaging Systems – Theory, Devices, Implementation Richard Baraniuk Kevin Kelly Rice University David Brady Rebecca Willett Duke University Rice/Duke | Compressive Optical Devices | August 2007

  2. Project OverviewRichard Baraniuk Rice/Duke | Compressive Optical Devices | August 2007

  3. Digital Revolution

  4. camera arrays hyperspectral cameras distributed camera networks

  5. Sensing by Sampling • Long-established paradigm for digital data acquisition • sampledata at Nyquist rate (2x bandwidth) • compressdata (signal-dependent, nonlinear) • brick wall to resolution/performance sample compress transmit/store sparsewavelettransform receive decompress

  6. Compressive Sensing (CS) • Directly acquire “compressed” data • Replace samples by more general “measurements” compressive sensing transmit/store receive reconstruct

  7. Compressive Sensing • When data is sparse/compressible, can directly acquire a condensed representation with no/little information lossthrough dimensionality reduction sparsesignal measurements sparsein somebasis

  8. Compressive Sensing • When data is sparse/compressible, can directly acquire a condensed representation with no/little information loss • Random projection will work sparsesignal measurements sparsein somebasis for signal reconstruction [Candes-Romberg-Tao, Donoho, 2004]

  9. Compressive Optical Imaging Systems –Theory, Devices, and Implementation • $400k budget for roughly April 2006-2007 • administered by ONR • Rice portion expended; Duke portion in NCE • Goals: • forge collaboration between Rice and Duke teams • demonstrate new Compressive Imaging technologies • hardware testbeds/demos at Rice and Duke • new theory/algorithms • quantify performance • articulate emerging directions • Collaborations: • telecons, visits, joint projects, joint papers, artwork

  10. Gerhard Richter 4096 Farben / 4096 Colours 1974254 cm X 254 cmLaquer on CanvasCatalogue Raisonné: 359 Museum Collection:Staatliche Kunstsammlungen Dresden (on loan) Sales history: 11 May 2004Christie's New York Post-War and Contemporary Art (Evening Sale), Lot 34US$3,703,500   Rice/Duke | Compressive Optical Devices | August 2007

  11. Gerhard Richter Dresden CathedralStained Glass Rice/Duke | Compressive Optical Devices | August 2007

  12. Agenda • Rebecca Willett, Duke [theory/algorithms] • Kevin Kelly, Rice [hardware] • David Brady, Duke [hardware] • Richard Baraniuk, Rice [theory/algorithms] • Discussion and Conclusions

  13. Compressive Image ProcessingRichard Baraniuk Rice/Duke | Compressive Optical Devices | August 2007

  14. Matthew Moravec Mona Sheikh Jason Laska Mike WakinMarco DuarteMark DavenportShri SarvothamPetrosBoufounos

  15. Image Classification/Segmentationusing Duke Hyperspectral System(with Rebecca Willett) Rice/Duke | Compressive Optical Devices | August 2007

  16. Information Scalability • If we can reconstruct a signal from compressive measurements, then we should be able to perform other kinds of statistical signal processing: • detection • classification • estimation … • Hyperspectral image classification/segmentation

  17. Classification Example spectrum 2 spectrum 1 spectrum 3

  18. Nearest Projected Neighbor • normalize measurements • compute nearest neighbor

  19. Naïve Results block size 32 8 16

  20. Results naïve independent classification tree-based classification

  21. Voting / Cycle Spinning block radius in pixels 16 20 24 28 32

  22. Summary • Direct hyperspectral classification/segmentation without reconstructing 3D data cube • Future directions • replace nearest projected neighbor with more sophisticated methods • smashed filter • projected SVM • quad-tree based multiscale segmentation (HMTseg, …) • Joint paper in the works

  23. Performance Analysis of Multiplexed Cameras Rice/Duke | Compressive Optical Devices | August 2007

  24. Single-Pixel Camera Analysis • Analyze performance in terms of • dynamic range and #bits of A/D • MSE due to photon counting noise • number of measurements photon detector imagereconstructionorprocessing DMD DMD random pattern on DMD array

  25. Single Pixel Image Acquisiton For a N-pixel, K-sparse image under T-second exposure: Raster Scan: Acquire one pixel at a time, repeat N times Basis Scan: Acquire one coefficient of image in a fixed basis at a time, repeat N times CS Scan: Acquire one incoherent/random projection of the image at a time, repeat times Rice/Duke | Compressive Optical Devices | August 2007

  26. Worst-Case Performance N: Number of pixels P: Number of photons per pixel T: Total capture time M: Number of measurements CN: CS noise amplification constant Sensor array shown as baseline Table shows requirements to match worst-case performance CS beats Basis Scan if Rice/Duke | Compressive Optical Devices | August 2007

  27. Single Pixel Camera Experimental Performance N = 16384 M = 1640  = Daub-8 Rice/Duke | Compressive Optical Devices | August 2007

  28. Multiplexed Camera Analysis Dude, you gotta multiplex! S photon detectors lens(es) imagereconstructionorprocessing DMD DMD random pattern on DMD array

  29. S-Pixel Camera Performance N: Number of pixels P: Number of photons per pixel T: Total capture time M: Number of measurements CN: CS noise amplification constant Sensor array shown as baseline M measurements split across S sensors Single pixel camera: S = 1 Rice/Duke | Compressive Optical Devices | August 2007

  30. S-Pixel Camera Performance N: Number of pixels P: Number of photons per pixel T: Total capture time M: Number of measurements CN: CS noise amplification constant Sensor array shown as baseline M measurements split across S sensors Single pixel camera: S = 1 CS beats Basis Scan if Rice/Duke | Compressive Optical Devices | August 2007

  31. Smashed Filter –Compressive Matched Filtering Rice/Duke | Compressive Optical Devices | August 2007

  32. Information Scalability • If we can reconstruct a signal from compressive measurements, then we should be able to perform other kinds of statistical signal processing: • detection • classification • estimation … • Smashed filter: compressive matched filter

  33. Matched Filter • Signal classification in additive white Gaussian noise • LRT: classify test signal as from Class iif it is closest to template signal i • GLRT: when test signal can be a transformed version of template, use matched filter • When signal transformations are well-behaved, transformed templates form low-dimensional manifolds • GLRT = matched filter= nearest manifold classification M1 M3 M2

  34. Compressive LRT • Compressive observations • By the Johnson-Lindenstrauss Lemma, random projection preserves pairwise distances with high probability

  35. Smashed Filter • Compressive observations of transformed signal • Theorem: Structure of smooth manifolds is preserved by random projection w.h.p. provided distances, geodesic distances, angles, volume, dimensionality, topology, local neighborhoods, … [Wakin et al 2006; to appear in Foundations on Computational Mathematics] M1 M1 M3 M2 M3 M2

  36. Stable Manifold Embedding Theorem: Let F ½ RN be a compact K-dimensional manifold with • condition number 1/t (curvature, self-avoiding) • volume V Let F be a random MxN orthoprojector with Then with probability at least 1-r, the following statement holds: For every pair x,y2F • [Wakin et al 2006]

  37. Manifold Learning from Compressive Measurements LaplacianEigenmaps ISOMAP HLLE R4096 RM M=15 M=15 M=20

  38. Smashed Filter – Experiments • 3 image classes: tank, school bus, SUV • N = 65536 pixels • Imaged using single-pixel CS camera with • unknown shift • unknown rotation

  39. Smashed Filter – Unknown Position • Object shifted at random (K=2 manifold) • Noise added to measurements • Goal: identify most likely position for each image class identify most likely class using nearest-neighbor test more noise classification rate (%) avg. shift estimate error more noise number of measurements M number of measurements M

  40. Smashed Filter – Unknown Rotation • Object rotated each 2 degrees • Goals: identify most likely rotation for each image class identify most likely class using nearest-neighbor test • Perfect classification with as few as 6 measurements • Good estimates of rotation with under 10 measurements avg. rot. est. error number of measurements M

  41. How Low Can M Go? • Empirical evidence that many fewer than measurements are needed for effective classification • Late-breaking results (experimental+nascent theory)

  42. Summary – Smashed Filter • Compressive measurementsare info scalable reconstruction > estimation > classification > detection • Random projections preserve structure of smooth manifolds (analogous to sparse signals) • Smashed filter: dimension-reduced GLRT for parametrically transformed signals • exploits compressive measurements and manifold structure • broadly applicable: targets do not have to have sparse representation in any basis • effective for detection/classification • number of measurements required appears to be independent of the ambient dimension

  43. Compressive Phase Retrievalfor Fourier Imagers Rice/Duke | Compressive Optical Devices | August 2007

  44. Coherent Diffraction Imaging • Image by sampling in Fourier domain • Challenge: we observe only the magnitude of the Fourier measurements

  45. Phase Retrieval • Given: Fourier magnitude + additional constraints (typically support) • Goal: Estimate phase of Fourier transform • Compressive Phase Retrieval (CPR) replace image support constraint with a sparsity/compressibility constraintnonconvex reconstruction

  46. Conclusions and Future Directions Rice/Duke | Compressive Optical Devices | August 2007

  47. Project Outcomes • Forged collaboration between Rice and Duke teams • several joint papers in progress • Demonstrated new Compressive Imaging technologies • hardware testbeds/demos • hyperspectral, low-light, infrared DMD cameras • coded aperture spectral imagers • new theory/algorithms • spectral image reconstruction/classification methods • smashed filter • Quantified performance • coded aperture tradeoffs • multiplexing tradeoff • number of measurements required for reconstruction/classification Rice/Duke | Compressive Optical Devices | August 2007

  48. Emerging Directions • Nonimaging cameras • exploit information scalability • attentive/adaptive cameras • meta-analysis • separating “imaging process” from “display” • Multiple cameras • image beamforming, 3D geometry imaging, … • Deeper links between physics and signal processing • significance of coherence and spectral projections • Links to analog-to-information program • nonidealities as challenges vs. opportunities • Other modalities • THz, LWIR/MWIR, UV, soft x-rays, …

More Related