1 / 62

Frequency domain methods for demosaicking of Bayer sampled color images Eric Dubois

Frequency domain methods for demosaicking of Bayer sampled color images Eric Dubois. Problem Statement.

dillan
Télécharger la présentation

Frequency domain methods for demosaicking of Bayer sampled color images Eric Dubois

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Frequency domain methods for demosaicking of Bayer sampled color imagesEric Dubois

  2. Problem Statement • Problem: Most digital color cameras capture only one color component at each spatial location. The remaining components must be reconstructed by interpolation from the captured samples. Cameras provide hardware or software to do this, but the quality may be inadequate. • Objective: Develop new algorithms to interpolate each color plane (called demosaicking) with better quality reconstruction, and with minimal computational complexity. Frequency-domain Bayer demosaicking

  3. Retinal Cone Mosaic The human visual system must solve a similar problem! Frequency-domain Bayer demosaicking

  4. Construction of color image from color planes + Frequency-domain Bayer demosaicking

  5. Lighthouse original

  6. Lighthouse red original

  7. Lighthouse green original

  8. Lighthouse blue original

  9. Formation of Color planes Frequency-domain Bayer demosaicking

  10. Lighthouse red subsampled

  11. Lighthouse green subsampled

  12. Lighthouse blue subsampled

  13. Lighthouse Bayer CFA image

  14. Color plane interpolation Green channel: bilinear interpolation GA GL GR GI GB Frequency-domain Bayer demosaicking

  15. Color plane interpolation Red channel: bilinear interpolation RNE RNW RC RSE RS RSW Frequency-domain Bayer demosaicking

  16. Lighthouse red interpolated

  17. Lighthouse green interpolated

  18. Lighthouse blue interpolated

  19. Lighthouse Interpolated color image

  20. Lighthouse original

  21. Can we do better? • Color planes have severe aliasing. Better interpolation of the individual planes has little effect. Frequency-domain Bayer demosaicking

  22. Lighthouse red interpolated with bilinear interpolator

  23. Lighthouse red interpolated with bicubic interpolator

  24. Can we do better? • Color planes have severe aliasing. Better interpolation of the individual planes has little effect. • We could optically prefilter the image (blur it) so that aliasing is less severe. Frequency-domain Bayer demosaicking

  25. Lighthouse red interpolated with bilinear interpolator

  26. Lighthouse prefiltered red interpolated with bilinear interpolator

  27. Lighthouse Interpolated color image

  28. Lighthouse Prefiltered & Interpolated color image

  29. Lighthouse original

  30. Can we do better? • Color planes have severe aliasing. Better interpolation of the individual planes has little effect. • We could optically prefilter the image (blur it) so that aliasing is less severe. • We can process the three color planes together to gather details from all three components. Frequency-domain Bayer demosaicking

  31. Can we do better? • There have been numerous papers and patents describing different algorithms to interpolate the color planes – they all work on the three planes together, exploiting the correlation between the three components. • Gunturk et al. published an extensive survey in March 2005. The best methods were the projection on convex sets (POCS) algorithm (lowest MSE) and the adaptive homogeneity directed (AHD) algorithm (best subjective quality). • We present here a novel frequency-domain algorithm. Frequency-domain Bayer demosaicking

  32. Spatial multiplexing model subsampling multiplexing Frequency-domain Bayer demosaicking

  33. Spatial multiplexing model Frequency-domain Bayer demosaicking

  34. Frequency-domain multiplexing model Re-arranging the spatial multiplexing expression Frequency-domain Bayer demosaicking

  35. Frequency-domain multiplexing model David Alleysson, EPFL Frequency-domain Bayer demosaicking

  36. Luma and chrominance components Frequency-domain Bayer demosaicking

  37. Luma and chrominance components Luma fL Chroma_1 fC1 Chroma_2 fC2 Frequency-domain Bayer demosaicking

  38. Lighthouse Bilinearly Interpolated color image

  39. Frequency-domain demosaicking algorithm • Extract modulated C1 using a band-pass filter at (0.5,0.5) and demodulate to baseband • Extract modulated C2 using band-pass filters at (0.5,0.0) and (0.0, 0.5), demodulate to baseband, and combine in some suitable fashion (the key) • Subtract modulated C1 and remodulated C2 from the CFA to get the estimated luma component L. • Matrix the L, C1 and C2 components to get the RGB representation. Frequency-domain Bayer demosaicking

  40. Spectrum of CFA signal b a Frequency-domain Bayer demosaicking

  41. Using C2b only Using C2a only

  42. Demosaicking using C2a only or C2b only -- details From C2b only Original From C2a only Frequency-domain Bayer demosaicking

  43. fC2am fC2a  h2a fC2 fC2 (-1)n1 fR combine fC2bm fC2b  h2b fCFA (-1)n1-(-1)n2 -(-1)n2  matrix - fG + + fL - fB  h1 fC1m fC1 fC1 (-1)n1+n2 Demosaicking Block Diagram Frequency-domain Bayer demosaicking

  44. Spectrum of CFA signal b a Frequency-domain Bayer demosaicking

  45. Design Issues • How to choose the filters h1, h2a and h2b • Frequency domain design methods • Least-squares design methods • Size of the filters • How to combine the two estimates and • Choice of features to guide weighting • The two above issues may be inter-related. Frequency-domain Bayer demosaicking

  46. Filter design • Gaussian filters (Alleysson) • Window design or minimax design • Define ideal response, with pass, stop and transition bands • Approximate using the window design method • Refine using minimax or least pth optimization • Can design low-pass filters and modulate to the center frequency Frequency-domain Bayer demosaicking

  47. Filter specification u v val 0.000 0.00 1.0 0.110 0.00 1.0 0.110 0.02 1.0 0.000 0.10 1.0 0.030 0.10 1.0 0.070 0.06 1.0 0.338 0.00 0.0 0.338 0.05 0.0 0.050 0.36 0.0 0.000 0.36 0.0 0.184 .205 0.0 0.500 0.00 0.0 0.000 0.50 0.0 0.500 0.50 0.0 Frequency-domain Bayer demosaicking

  48. Ideal response – perspective view Frequency-domain Bayer demosaicking

  49. Ideal response – contour plot Frequency-domain Bayer demosaicking

  50. Window design – perspective view Frequency-domain Bayer demosaicking

More Related