COLOR ENHANCEMENT IN THE COMPRESSED DOMAIN

1. Jayanta Mukhopadhyay Department of Computer Science & Engineering Indian Institute of Technology, Kharagpur, 721302, India jay@cse.iitkgp.ernet.in Sanjit K. Mitra Ming Hsieh Dept. of Electrical Engineering University of Southern California Los Angeles, CA 90089, USA skmitra@usc.edu COLOR ENHANCEMENT IN THE COMPRESSED DOMAIN

2. Image Enhancement: An Example (a) Original (b) Enhanced

3. Enhancement Factors • Dynamic Range of Intensity Values. Limited number of bit-planes. • Varying illumination over the image space. • Brightness • Contrast • Color

4. Enhancement Techniques • Spatial Domain Techniques. Work with the pixels values. Histogram Stretching, Pixel Mapping, Image Sharpening • Compressed Domain Techniques. Works with DCT coefficients

5. DCT: Definition Let x (m,n), 0 m,n  N-1 be a 2-D image of size NxN. Its N-point DCT C(l,k), 0 l,k  N-1 is defined as:

6. Motivations • Computation with reduced storage. • Avoid overhead of entropy decoding and encoding. • Exploit spectral factorization for improving the quality of result and speed of computation.

7. Existing Approaches • Alpha Rooting Aghaglzadeh and Ersoy (1992), Opt.Engg • Multi Contrast Enhancement Tang, Peli and Acton (2003), IEEE SPL A spectral band is defined with Yijs such that i+j=n; 0<n<15 Contrast measure (Hn)= Cumulative Energy Sum till nth band in original image ----------------------------------------------------------------------- Cumulative Energy Sum till nth band in enhanced image Each coefficient in a band is scaled by : Yij = λ Hn. Yij, , i+j=n

8. Existing Approaches • Multi-Contrast Enhancement with Dynamic Range Compression (S.Lee (2007), IEEE CSVT) Modification of DC coefficients and AC coefficients (following similar strategy of multi-contrast enhancement). Normalized DC coefficients (x) are modified as follows:

9. Proposed Approach • Adjust background illumination. Use DC coefficients of the Y component. • Preserve Local Contrast. Scale AC coefficients of the Y component appropriately. • Preserve Colors. Preserve Color Vectors in the DCT domain. DCT coefficients of Cb and Cr components.

10. Contrast : Definition Let μand σdenote the mean and standard deviation of an image. Contrast ζof an image is defined here as: . Weber Law: where is the difference in luminance between a stimulus and its surround, and L is the luminance of the surround

11. Theorem on Contrast Preservation in the DCT Domain Let d be the scale factor for the DC coefficient and aa be the scale factor for the AC coefficients of a DCT block Y. The processed DCT block Ye is given by: The contrast of the processed image then becomes a / d times of the contrast of the original image. In this algorithmd = a = for preservation of the contrast.

12. Preservation of Colours in the DCT Domain Let U and V be the DCT coefficients of the Cb and Cr components, respectively. If the luminance component Y of an image is uniformly scaled by a factor  , the colors of the processed image with Ye , Ue and Ve are preserved by the following operations:

13. Enhancement by Scaling Coefficients • Find the scale factor by mapping the DC coefficient with a monotonically increasing function. • Apply scaling to all other coefficients in all the components. • For blocks having greater details, apply block decomposition and re-composition strategy.

14. Mapping functions for adjusting the local background illumination (DRC) Lee, CSVT’07 (TW) Mitra and Yu , CVGIP’87 (SF) De, TENCON’89

15. Monotonic Mapping Functions

16. Scaling only DC coefficients

17. Scaling both DC and AC coefficients

18. Preservation of Contrast and Color original

19. Enhancement of Blocks with more details Apply CES on smaller blocks Block Decompos. 8x8 block Smaller DCT blocks Block Composition Enhanced Block

20. Removal of Blocking Artifacts original

21. Some Results original AR MCE MSR MCEDRC TW-CES-BLK

22. Enhancement near Edges AR MCE MCEDRC SF-CES-BLK TW-CES-BLK DRC-CES-BLK

23. Some Results AR MCE original TW-CES-BLK MCEDRC MSR

24. Enhancement near edges AR MCE MCEDRC TW-CES-BLK DRC-CES-BLK SF-CES-BLK

25. Some Results original AR MCE MCEDRC TW-CES-BLK MSR

26. Enhancement near edges MCEDRC MCE AR TW-CES-BLK DRC-CES-BLK SF-CES-BLK

27. Metrics for Comparison Wang and Bovic (SPL, 2002) JPEG Quality Metric (JPQM) Wang and Bovic (ICIP,2002) Susstrunk and Winkler (SPIE, 2004)

28. Other Approaches • Alpha Rooting (AR): Aghaglzadeh and Ersoy (1992), Opt.Engg. • Multi-Contrast Enhancement (MCE): Tang, Peli and Acton (2003), IEEE SPL • Multi-Contrast Enhancement with Dynamic Range Compression (MCEDRC): S.Lee (2007), IEEE CSVT • Contrast Enhancement by Scaling (CES): Proposed work • Multi-Scale Retinex (MSR) (a reference spatial domain technique): Jobson, Rahman and Woodell (1997), IEEE IP

29. Average Performance Measures

30. Computational Complexities aE+bM+cA implies a Exponentiation, b Multiplication and c Addition operations.

31. Iterative Enhancement Iteration no.=1 Iteration no.=3 original Iteration no.=4 Iteration no.=2

32. Conclusion • A novel algorithm for enhancement of color images in the block DCT domain. • It takes care of adjustment of back-ground illumination, local contrast, as well as preservation of colours. • Comparative study shows the superiority of the proposed scheme compared to some of the existing schemes.

33. References • [1] J. Mukherjee and S.K.Mitra, Enhancement of Color Images by Scaling the DCT coefficients,IEEE trans. on Image Processing, vol. 17, no. 10, Oct., pp. 1783-1794, 2008. • [2] J. Mukherjee and S.K. Mitra, Color Enhancement in the Compressed Domain,Int. Conf. on Image Proc. (ICIP-2008), San Diego, USA, Oct. 12 -15, pp. 3144-3147, 2008 .  

34. Thanks!