Create Presentation
Download Presentation

Download Presentation
## Image Enhancement [DVT final project]

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Image Enhancement [DVT final project]**Speaker: Yu-Hsiang Wang Advisor: Prof. Jian-Jung Ding Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University DISP Lab, Graduate Institute of Communication Engineering, NTU**Outline**• Target • Possible enhancement methods • Interpolation • Adaptive osculatory rational interpolation • Enhancement • Bilateral Enhancers • Non Local Means • Conclusion • Reference**Target**• Scale image from SD (Standard-definition) to HD (High-definition). • Given: 720x480, YCbCr, 4:2:2, 8 bits per channel. • Target: 1920x1080, YCbCr, 4:4:4, 10 bits per channel.**Possible enhancement methods**• Sharpness • Noise reduction • Edge smoothness • Skin-tone enhancement • Texture enhancement • Super resolution (SR) Multi-Image SR [1] Single-Image Multi-Patch SR [1]**Interpolation:Bilinear**• 1. Interpolate R1: • 2. Interpolate R2: • 3. Interpolate P: [2]**Interpolation: Adaptive osculatory rational interpolation**• The interpolation kernel function of adaptive osculatory rational interpolation (AORI) is more accurately approximate to the ideal interpolation not only in space domain but also in frequency domain. • Apply 4 points to interpolate 1 point in one direction.**Interpolation: Adaptive osculatory rational interpolation**• The interpolation function • where r(x) is the interpolated value, g(xk) are the sample values, RI(x) is the interpolation kernel.**Interpolation: Adaptive osculatory rational interpolation**• The interpolation kernel [M. Hu and J. Q. Tan, ”Adaptive osculatory rational interpolation for image processing,” Journal of Computational and Applied Mathematics, vol. 195, pp. 46-53, 2006.]**Interpolation: Adaptive osculatory rational interpolation**• The original 1920x1080 image:**Interpolation: Adaptive osculatory rational interpolation**• The interpolated image by AORI:**Interpolation: Adaptive osculatory rational interpolation**• The original 1920x1080 image:**Interpolation: Adaptive osculatory rational interpolation**• The interpolated image by AORI:**Enhancement: Bilateral Enhancers**• Extended from the bilateral filter (BF). • BF: • A nonlinear filter adopts a low-pass Gaussian filter for both the domain filter and the range filter. • Smooth the noise while preserving edges.**Enhancement: Bilateral Enhancers**• Bilateral filter (BF) • with the normalization factor • where r is the input image, h is the output image, Ω(x) is a subset of the input image r with x and ξ are the pixels coordinates. (row, column) [C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. ICCV, pp. 839–846, 1998.]**Enhancement: Bilateral Enhancers**• Bilateral filter (BF) • with the normalization factor • The function c operates on the spatial domain designed as • The s operates on the range domain designed as**Enhancement: Bilateral Enhancers**• Problem of Bilateral filter: • Only edge-preserving and de-noising. • Do not enhance the sharpness of an image.**Enhancement: Bilateral Enhancers**• Bilateral enhancers (sharpness + smoothness) • The c function is set as the same as in BF. (Gaussian function on spatial domain) [C. Gatta and P. Radeva, “Bilateral enhancers,” IEEE ICIP, pp. 3161-3164, 2009.] vs**Enhancement: Bilateral Enhancers**• Bilateral enhancers (sharpness + smoothness) • The p composes of two parts (p = ps + pe): • the edge-preserving smoothing (ps) • the selective sharpening (pe) vs**Enhancement: Bilateral Enhancers**• The edge-preserving smoothing (ps) • ηs: adjusts the intensity of the blurring. • σs: controls how strong should be an edge to be preserved from the blurring. • If the intensity difference is small, Gaussian smoothing is performed.**Enhancement: Bilateral Enhancers**• The selective sharpening (pe) • ηe and σe have similar meaning as for the edge-preserving smoothing. • If the intensity difference is small, no enhancement is performed.**Enhancement: Bilateral Enhancers**• The interpolated image by AORI:**Enhancement: Bilateral Enhancers**• Enhanced the previous page’s image by BE.**Enhancement: Bilateral Enhancers**• The original 1920x1080 image:**Enhancement: Bilateral Enhancers**• The interpolated image by AORI:**Enhancement: Bilateral Enhancers**• Enhanced the previous page’s image by BE.**Enhancement: Non Local Means**• Purpose: Noise reduction. • Given an interpolated image • The estimated value • W is a search window of fixed size (we choose 5x5 here) centered at pixel i. • The weights depend on the similarity between the pixel i and j. [A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, Jun. 2005.]**Enhancement: Non Local Means**• The weighted Euclidean distance • Nk denotes a square neighborhood of fixed size (we choose 3x3) and centered at a pixel k. • α is the standard deviation of the Gaussian kernel. • G(N) is the Gaussian kernel of the same size as Nk.**Enhancement: Non Local Means**• The weights are defined as • Z(i) is the normalizing constant • h denotes a degree of filtering. (It controls the decay of the weights function of the Euclidean distances.)(h = 2.5)**Enhancement: Non Local Means**• Scheme of NL-means strategy. [6]**Enhancement: Non Local Means**• Advantage: • NL-meanscompares the gray level of pixels. • Compare the geometrical configuration in a whole neighborhood. • Disadvantage: • Do not perform sharpness. • Blur some edges.**Enhancement: Non Local Means**• The interpolated image by AORI:**Enhancement: Non Local Means**• De-noise the previous image by NL-means.**Enhancement: Non Local Means**• The interpolated image by AORI:**Enhancement: Non Local Means**• De-noise the previous image by NL-means.**Enhancement: Non Local Means**• Enhanced the previous page’s image by BE.**Conclusion**• Adaptive osculatory rational interpolation is more accurately approximate to the ideal interpolation. • Bilateral enhancers performs well at sharpness and smoothness. • Non local means is mainly used for noise reduction.**Reference**• [1] D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” ICCV, pp. 349-356, Sep. 2009. • [2]http://en.wikipedia.org/wiki/Bilinear_interpolation • [3] M. Hu and J. Q. Tan, ”Adaptive osculatory rational interpolation for image processing,” Journal of Computational and Applied Mathematics, vol. 195, pp. 46-53, 2006. • [4] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. ICCV, pp. 839–846, 1998. • [5] C. Gatta and P. Radeva, “Bilateral enhancers,” IEEE ICIP, pp. 3161-3164, 2009. • [6] A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, Jun. 2005.