Introduction

1. Introduction CSc I6716 Spring 2012 Part I Feature Extraction (1) Image Enhancement Zhigang Zhu, City College of New York zhu@cs.ccny.cuny.edu

2. What are Image Features? • Local, meaningful, detectable parts of the image.

3. More Color Woes Squares with dots in them are the same color

4. Topics • Image Enhancement • Brightness mapping • Contrast stretching/enhancement • Histogram modification • Noise Reduction • ……... • Mathematical Techniques • Convolution • Gaussian Filtering • Edge and Line Detection and Extraction • Region Segmentation • Contour Extraction • Corner Detection

5. Image Enhancement • Goal: improve the ‘visual quality’ of the image • for human viewing • for subsequent processing • Two typical methods • spatial domain techniques.... • operate directly on image pixels • frequency domain techniques.... • operate on the Fourier transform of the image • No general theory of ‘visual quality’ • General assumption: if it looks better, it is better • Often not a good assumption

6. Spatial Domain Methods • Transformation T • point - pixel to pixel • area - local area to pixel • global - entire image to pixel • Neighborhoods • typically rectangular • typically an odd size: 3x3, 5x5, etc • centered on pixel I(x,y) • Many IP algorithms rely on this basic notion T(I(x,y)) neighborhood N I(x,y) I’(x,y) I’(x,y) = T(I(x,y)) O=T(I)

7. 255 OUTPUT 0 255 INPUT Point Transforms: General Idea Transfer Function T O = T(I) Input pixel value, I, mapped to output pixel value, O, via transfer function T.

8. Grayscale Transforms • Photoshop ‘adjust curve’ command Output gray value I’(x,y) Input gray value I(x,y)

9. Point Transforms: Brightness

10. Imax if I(x,y) > t Imin if I(x,y) ≤ t I’(x,y) = Point Transforms:Thresholding • T is a point-to-point transformation • only information at I(x,y) used to generate I’(x,y) • Thresholding 255 0 0 t 255 t=89

11. Point Transforms: Linear Stretch 255 OUTPUT 0 INPUT 255

12. Linear Scaling • Consider the case where the original image only utilizes a small subset of the full range of gray values: • New image uses full range of gray values. • What's F? {just the equation of the straight line}

13. Linear Scaling • F is the equation of the straight line going through the point (Imin , 0) and (Imax , K) • useful when the image gray values do not fill the available range. • Implement via lookup tables

14. Scaling Discrete Images • Have assumed a continuous grayscale. • What happens in the case of a discrete grayscale with K levels?

15. Non-Linear Scaling: Power Law g O = I •  < 1 to enhance contrast in dark regions •  > 1 to enhance contrast in bright regions. 1 g<1 0 . 5 g=1 g>1 0 0 0 . 5 1

16. Square Root Transfer: g=.5

17. g=3.0

18. Examples • Technique can be applied to color images • same curve to all color bands • different curves to separate color bands:

19. 255 Imax if I(x,y) > t Imin if I(x,y) ≤ t I’(x,y) = 0 0 t 255 Point Transforms:Thresholding • T is a point-to-point transformation • only information at I(x,y) used to generate I’(x,y) • Thresholding t=89

20. Threshold Selection • Arbitrary selection • select visually • Use image histogram Threshold

21. Histograms • The image shows the spatial distribution of gray values. • The image histogram discards the spatial information and shows the relative frequency of occurrence of the gray values.

22. Image Histogram • The histogram typically plots the absolute pixel count as a function of gray value: For an image with dimensions M by N

23. Probability Interpretation • The graph of relative frequency of occurrence as a function of gray value is also called a histogram: • Interpreting the relative frequency histogram as a probability distribution, then:   P(I(x,y) = i) = H(i)/(MxN)

24. Cumulative Density Function • Interpreting the relative frequency histogram as a probability distribution, then: • Curve is called the cumulative distribution function P(I(x,y) = i) = H(i)/(MxN) Gray Value

25. Examples

26. Color Histograms

27. Histogram Equalization • Image histograms consist of peaks, valleys, and low plains • Peaks = many pixels concentrated in a few grey levels • Plains = small number of pixels distributed over a wider range of grey levels

28. Histogram Equalization • The goal is to modify the gray levels of an image so that the histogram of the modified image is flat. • Expand pixels in peaks over a wider range of gray-levels. • “Squeeze” low plains pixels into a narrower range of gray levels. • Utilizes all gray values equally • Example Histogram: • Note low utilization of small gray values

29. Desired Histogram • All gray levels are used equally. • Has a tendency to enhance contrast.

30. Brute Force How are the gray levels in the original image changed to produce the enhanced image? Method 1. Choose points randomly. Method 2. Choice depends on the gray levels of their neighboring points. Computationally expensive. Approximations.

31. CH(j) i = MAX 0, round -1 Np M*N Np = G Histogram Equalization • Mapping from one set of grey-levels, I, to a new set, O. • Ideally, the number of pixels, Np, occupying each grey level should be: • To approximate this, apply the transform • Where CH is the cumulative histogram (see next slide) • j is the gray value in the source image • i is the gray value in the equalized image

32. j CH(j) = S H(i) i=0 Example j H(j) CH(j) i 100 900 1600 2100 2200 2300 2400 2400 0 100 0 1 800 2 G=8 MxN=2400 Np=300 2 700 4 3 500 6 4 100 6 5 100 7 6 100 7 7 0 7 800 600 400 200 0 800 600 400 200 0 ideal 0 1 2 3 4 5 6 8 0 1 2 3 4 5 6 8

33. Example

34. Comparison Original >1 Histogram equalization

35. m n S G(qi) = S H(pj) j=0 i=0 Why? • Given MxN input image I with gray scale p0….pk and histogram H(p). • Desire output image O with gray scale q0….qk and uniform histogram G(q) • Treat the histograms as a discrete probability density function. Then monotonic transfer function m = T(n) implies: • The sums can be interpreted as discrete distribution functions. From Image Processing, Analysis, and Machine Vision, Sonka et al.

36. m n S = S H(pi) i=0 i=0 MxN qk - q0 pn = H(s)ds p0 MN(qm-q0) qk - q0 Why? continued MxN G(i) = qk - q0 • The histogram entries in G must be: (because G is uniform) • Plug this into previous equation to get: • Now translate into continuous domain (can’t really get uniform distributions in the discrete domain) qm MxN ds qk - q0 q0

37. qk - q0 qm = T(p) = MN pn qk - q0 pn S H(i) + q0 qm = T(p) = H(s)ds + q0 MN i=p0 p0 Why? continued • Solve last equation for q to get: • Map back into discrete domain to get: • Leads to an algorithm…..

38. G-1 T(p) = round ( Hc (p)) NM gq = T(gp) Histogram Equalization Algorithm • For an NxM image of G gray levels, say 0-255 • Create image histogram • For cumulative image histogram Hc • Set • Rescan input image and write new output image by setting

39. Observations • Acquisition process degrades image • Brightness and contrast enhancement implemented by pixel operations • No one algorithm universally useful •  > 1 enhances contrast in bright images •  < 1 enhances contrast in dark images • Transfer function for histogram equalisation proportional to cumulative histogram

40. Noise Reduction • What is noise? • How is noise reduction performed? • Noise reduction from first principles • Neighbourhood operators • linear filters (low pass, high pass) • non-linear filters (median) + = image ‘grainy’ image noise

41. Noise Image • Plot of image brightness. • Noise is additive. • Noise fluctuations are rapid • high frequency. Noise Image + Noise

42. Sources of Noise • Sources of noise = CCD chip. • Electronic signal fluctuations in detector. • Caused by thermal energy. • Worse for infra-red sensors. • Electronics • Transmission Radiation from the long wavelength IR band is used in most infrared imaging applications

43. 2ps2 Noise Model • Plot noise histogram • Typical noise distribution is normal or Gaussian • Mean(noise)  = 0 • Standard deviation  2 2 1 1 (x-m) h(x) = exp - 2 s 

44. Effect of s • Integral under curve is 1 =10 =20 =30

45. Two Dimensional Gaussian

46. Noise • Noise varies above and below uncorrupted image. Image Image + Noise

47. Noise Reduction - 1 • How do we reduce noise? • Consider a uniform 1-d image and add noise. • Focus on a pixel neighbourhood. • Averaging the three values should result in a value closer to original uncorrupted data • Especially if gaussian mean is zero Ai-1 Ai Ai+1 Ci

48. Noise Reduction - 1 • Averaging ‘smoothes’ the noise fluctuations. • Consider the next pixel Ai+1 • Repeat for remainder of pixels. Ai-1 Ai Ai+1 Ai+2 Ci+1

49. Ai-1 + Ai + Ai+1 3 Neighborhood Operations C = A * B B = [ B1 B2 B3 ] Ci = Ai-1´B1 + Ai´B2 + Ai+1´B3 B = [1 1 1] Ci = • All pixels can be averaged by convolving 1-d image A with mask B to give enhanced image C. • Weights of B must equal one when added together. • Why? 1 3

50. 2D Analog of 1D Convolution • Consider the problem of blurring a 2D image • Here’s the image and a blurred version: • How would we do this? • What’s the 2D analog of 1D convolution?