Introduction to Computer Vision

1. Introduction to Computer Vision Lecture 10 Dr. Roger S. Gaborski

2. Straight Line Detection • Man made object often contain straight lines • Straight lines can be detected by first filtering the image with line kernels and then linking together edges that are on a straight line • The Hough Transform is another method for detecting straight lines- works with EDGE DATA • Hough transforms can also detect other shapes, such as, circles Roger S. Gaborski

3. Consider a set of points • We would like to find a subset of the points that lie on a straight line Roger S. Gaborski

4. Parameter Space • Consider a particular point (xi,yi) in the x,y plane • An infinite number of lines can pass through this point • All satisfy slope-intercept equation: yi= a xi + b for some a,b • Solve for b: b = -a xi + yi • Parameter space a-b plane yields single line for (xi,yi) Roger S. Gaborski

5. Parameter Space • Consider a second point (xj,yj) in the x,y plane • Also has a line associated with it in parameter space • This line intersects the line associated with (xi,yi) at ( a’,b’ ) • a’ is the slope and b’ is the intercept of the line containing both (xi,yi) and (xj,yj) in the x-y space • All points on this line have lines in the parameter space that intersect at ( a’, b’ ) Roger S. Gaborski

6. Chapter 10 Image Segmentation Roger S. Gaborski

7. Parameter Space • (a, b) space usually referred to as (, ) space • Each line plots to a point in (, ) parameter space, but must satisfy: •  = x1 cos + y1 sin  where x1 y1 are constants Roger S. Gaborski

8. Parameter Space • Locus of such lines in x,y space is a sinusoid in parameter space • Any point in x,y plane corresponds to a sinusoid curve in ,  space Roger S. Gaborski

9. Chapter 10 Image Segmentation Roger S. Gaborski

10. ( ,  ) Space • Define  and  • For each point from from edge detector • Plug in values for x and y:  = x cos + ysin • For each value of  in parameter space, solve for  • For each   pair (from previous step) record in quantized space (this is a hit) • After evaluation of all edge points, number of hits in each block corresponds to the number of pixels on the line as defined by the values   Roger S. Gaborski

11. Hough Transform – General Idea (one point) min max min  Sample of two lines  (x1y1)  max Two of an infinite number of lines through point (x1,y1)   Space Roger S. Gaborski

12. Hough Transform – General Idea (three points) min max min   Line through three Points (count=3)  (x1y1)   max Two of an infinite number of lines through point (x1,y1)   Space Roger S. Gaborski

13. MATLAB Hough >> [H,theta,rho]=hough(f); >> figure, imshow(theta,rho,H, [ ], 'notruesize') >> axis on, axis normal; >> xlabel('\theta'),ylabel('\rho'), >> colorbar >> colormap(jet) Roger S. Gaborski

14. Roger S. Gaborski

15. rho is 80 units Roger S. Gaborski

16. Roger S. Gaborski

17. Peak Detection • Need to find peaks in Hough Transform • Due to quantization, pixels not in perfectly straight line, peaks lie in more than one cell • Approach: • Find Hough cell containing maximum number of entries and record location • Set immediate neighbors to zero • Look for next maximum peak, • etc Roger S. Gaborski

18. Find Peaks >> [H,theta,rho]=hough(f); >> figure, imshow(theta,rho,H, [ ], 'notruesize') >> axis on, axis normal; >> xlabel('\theta'),ylabel('\rho'), >> colorbar >> colormap(jet) >>[r,c]=houghpeaks(H,5); >> hold on >> plot(theta(c), rho(r), 'linestyle', 'none','marker' , ... 's', 'color', 'w') Roger S. Gaborski

19. Peak Roger S. Gaborski

20. Find Lines >> lines = houghlines(f, theta, rho, r, c) lines = point1: [100 79.5571] point2: [1 79.5571] length: 99 theta: -90 rho: -78.5571 >> figure, imshow(f), hold on >> for k = 1:length(lines) %lines is a structure xy=[lines(k).point1;lines(k).point2]; plot(xy(:,2), xy(:,1), 'LineWidth', 4, 'Color', [.6 .6 .6]); end Roger S. Gaborski

21. Detected Line Input image with original line Line detected Roger S. Gaborski

22. f Roger S. Gaborski

23. Hough Do not know how long line is Roger S. Gaborski

24. Input image f Output image Roger S. Gaborski

25. Roger S. Gaborski

26. Roger S. Gaborski

27. Crop Out Sign Region Gray Scale Image Edge Image (Canny) Roger S. Gaborski

28. Moose Sign – 6 Peaks Roger S. Gaborski

29. Hough Lines Detected ( Six) Roger S. Gaborski

30. Hough Line Detection Cropped image – 6 lines Roger S. Gaborski

31. Hough Line Detection 10 Lines Roger S. Gaborski

32. Roger S. Gaborski

33. Chapter 10 Image Segmentation Roger S. Gaborski

34. Color Image Processing • RGB Color Image • M x N x 3 array of color pixels • The three planes correspond to red, green and blue values. Each pixel location is actually a triplet, r, g and b • Each plane drives the respective red, green and blue inputs of a color monitor • Data classes: • Double  [0,1] • Uint8  [ 0 255] Roger S. Gaborski

35. Color Image imread('RedCar'); Roger S. Gaborski

36. Chapter 6 Color Image Processing Roger S. Gaborski

37. Chapter 6 Color Image Processing hsv_image = rgb2hsv(rgb_image); rgb_image = hsv2rgb(hsv_image); Roger S. Gaborski

38. Chapter 6 Color Image Processing Roger S. Gaborski

39. Extract individual color planes: • fRed = rgb_image(:,:,1); • fGreen = rgb_image(:,:,2); • fBlue = rgb_image(:,:,3); • Combine individual frames • Rgb_image = cat(3, fRed, fGreen, fBlue); Roger S. Gaborski

40. Image Segmentation in RGB • Partition image into regions • Regions have a ‘common feature’ • Color • Texture • Gray level • Color-based segmentation • Obtain a sample of color pixels that match region we wish to segment • Estimate the mean of the three colors • Store color values in column vector m • Classify each pixel in image as having a color specified by m Roger S. Gaborski

41. Classify Pixels- Euclidean Distance • We need a measure of similarity to classify pixels – how ‘close’ is a given pixel to m? • Let z denote arbitrary pixel in the image • z is similar to m if the distance between them is less than some threshold, T • Euclidean distance: D(z,m) = z-m Where x is the norm of x Roger S. Gaborski

42. Euclidean Metric • D(m,z) = [ (z-m)T (z-m) ]1/2 = [(zR-mR)2 + (zG-mG)2 + (zB-mB)2]1/2 D(m,z)  T is a solid sphere of radius T. Points within, or on the surface are defined as being part of the segmented region Roger S. Gaborski

43. Mahalanobis Metric • Generalization D(z,m) = [ (z-m)T C-1 (z-m) ]1/2 where C is the covariance of the sample data • D(z,m)  T is a solid 3-D elliptical body • The principal axes are orientation in direction of maximum data spread • If C = I, the Mahalanobis reduces to the Euclidean distance Roger S. Gaborski

44. Covariance • Variance – measure of the deviation from the mean for points in one dimension e.g. heights • Covariance as a measure of how much each of the dimensions vary from the mean with respect to each other. • Covariance is measured between 2 dimensions to see if there is a relationship between the 2 dimensions e.g. number of hours studied & grades obtained. • The covariance between one dimension and itself is the variance Roger S. Gaborski

45. Covariance variance (X) = Σi=1n(Xi – X) (Xi – X) (n -1) covariance (X,Y) = Σi=1n(Xi – X) (Yi – Y) (n -1) • So, if you had a 3-dimensional data set (x,y,z), then you could measure thecovariance between the x and y dimensions, the y and z dimensions, and the x and z dimensions. Measuring the covariance between x and x , or y and y , or z and z wouldgive you the variance of the x , y and z dimensions respectively. Roger S. Gaborski

46. Covariance • What is the interpretation of covariance calculations? e.g.: 2 dimensional data set x: number of hours studied for a subject y: grades obtained in that subject covariance value is say: 104.53 what does this value mean? Roger S. Gaborski

47. Covariance • Exact value is not as important as it’s sign. • A positive value of covariance indicates both dimensions increase or decrease together e.g. as the number of hours studied increases, the marks in that subject increase. • A negative value indicates while one increases the other decreases, or vice-versa e.g. active social life at RIT vs performance in CS dept. • If covariance is zero: the two dimensions are independent of each other e.g. heights of students vs the marks obtained in a subject Roger S. Gaborski

48. Covariance Matrix • Representing Covariance between dimensions as a matrix e.g. for 3 dimensions: cov(x,x) cov(x,y) cov(x,z) C = cov(y,x) cov(y,y) cov(y,z) cov(z,x) cov(z,y) cov(z,z) • Diagonal is the variances of x, y and z • cov(x,y) = cov(y,x) hence matrix is symmetrical about the diagonal • N-dimensional data will result in nxn covariance matrix Roger S. Gaborski

49. Covariance • MATLAB: cov • RG cov = GR cov, etc. • Diagonal are variances • Matrix is symmetric Roger S. Gaborski