Computational Vision

1. Computational Vision • Edge Detection • Canny Detector • Line Detection • Hough Transform • Trucco: Chapter 4, pp. 76 – 80 Chapter 5, pp. 95 - 100 Computational Vision / Ioannis Stamos

2. Finding Corners CSc 83020 3-D Computer Vision – Ioannis Stamos

3. What Is a Corner? Large gradients in more than one direction. CSc 83020 3-D Computer Vision – Ioannis Stamos

4. Edges vs. Corners Edges = maxima in intensity gradient

5. Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhood

6. Detecting Corners How to detect this variation?

7. Detecting Corners How to detect this variation? Not enough to check average and

8. Detecting Corners Claim: the following covariance matrix summarizes the statistics of the gradientSummations over local neighborhoods

9. Detecting Corners Examine behavior of C by testing its effect in simple cases Case #1: Single edge in local neighborhood

10. Case#1: Single Edge Let (a,b) be gradient along edge Compute C (a,b):

11. Case #1: Single Edge However, in this simple case, the only nonzero terms are those where f = (a,b) So, C (a,b) is just some multiple of (a,b)

12. Case #2: Corner Assume there is a corner, with perpendicular gradients (a,b) and (c,d)

13. Case #2: Corner • What is C (a,b)? • Since (a,b)  (c,d) = 0, the only nonzero terms are those where f = (a,b) • So, C (a,b) is again just a multiple of (a,b) • What is C (c,d)? • Since (a,b)  (c,d) = 0, the only nonzero terms are those where f = (c,d) • So, C (c,d) is a multiple of (c,d)

14. Corner Detection Matrix times vector = multiple of vector Eigenvectors and eigenvalues! In particular, if C has one large eigenvalue, there’s an edge If C has two large eigenvalues, have corner Tomasi-Kanade corner detector

15. Corner Detection Implementation • Compute image gradient • For each mm neighborhood,compute matrix C • If smaller eigenvalue 2 is larger thanthreshold , record a corner • Nonmaximum suppression: only keep strongest corner in each mm window

16. Corner Detection Results Checkerboardwith noise Trucco & Verri

17. Corner Detection Results

18. Corner Detection Results Histogram of l2 (smaller eigenvalue)

19. A Simple Corner Detector Find eigenvalues of C If the smaller eigenvalue is above a threshold then we have a corner. CSc 83020 3-D Computer Vision – Ioannis Stamos

20. A Simple Corner Detector Find eigenvalues of C If the smaller eigenvalue is above a threshold then we have a corner. CSc 83020 3-D Computer Vision – Ioannis Stamos

21. A Simple Corner Detector CSc 83020 3-D Computer Vision – Ioannis Stamos

22. Canny Edge Detection • Can we derive an optimal detector? • Good Detection: minimize false positives and false negatives. • Good Localization: close as possible to the true edges. • Single Response Constraint: one edge should be detected for each true edge

23. Comparison Sobel Canny CSc 83020 3-D Computer Vision – Ioannis Stamos

24. Canny Edge Detection • 3 STEPS: • CANNY_ENHANCER • NONMAX_SUPPRESION • HYSTERESIS_THRESH

25. Localization-Detection Tradeoff • Filter’s spatial scale • Location & • Detection criteria. • Good tradeoff 1-D step edge detector: (optimal) • Approximated by the 1st derivative of the Gaussian.

26. Optimal 1-D step edge detectors… Pixel (i,j) Keep the one that gives you maximum response. Expensive.

27. Solution… y x Pixel (i,j) Compute derivatives with respect to x & y directions. Compute edge normal.

28. CANNY_ENHANCER • Compute Ix and Iy the gradient of the image using a derivative of Gaussian filter. • Compute the edge strength from the magnitude of the gradient: Es • Compute the orientation of the edge from arctan(Iy / Ix ): Eo [Canny ’86] n

29. Problem with detector gradient magnitude (Es) original image • Compute image derivatives • if gradient magnitude >  and the value is a local max. along gradient • direction – pixel is an edge candidate • how to detect one pixel thin edges ?

30. NONMAX_SUPPRESSION • Locate local maxima from Es. • Find direction d that best approximates Eo(i,j) • IfEs(i,j)is smaller than at least one neighbor along dIn(i,j)=0elseIn(i,j)=Es(i,j). 90 135 45 0

31. Non-Maximum Supression • Non-maximum suppression: • Select the single maximum point across the width of an edge. CSc 83020 3-D Computer Vision – Ioannis Stamos

32. Linking to the Next Edge Point Assume the marked point q is an edge point. Take the normal to the gradient at that point and use this to predict continuation points (either r or p). CSc 83020 3-D Computer Vision – Ioannis Stamos

33. Thresholding • Edges are found by thresholding the output of NONMAX_SUPRESSION • If the threshold is too high: • Very few (none) edges • High MISDETECTIONS, many gaps • If the threshold is too low: • Too many (all pixels) edges • High FALSE POSITIVES, many extra edges

34. Edge Detection With Hysteresis Low threshold High threshold Hysteresis (high and low threshold)

35. Edge Hysteresis • Hysteresis:A lag or momentum factor • Idea: Maintain two thresholds khigh and klow • Use khigh to find strong edges to start edge chain • Use klow to find weak edges which continue edge chain • Typical ratio of thresholds is roughly khigh/klow=2 CSc 83020 3-D Computer Vision – Ioannis Stamos

36. Weak edges Strong edge Edge Tracking Hysteresis thresholding [Canny ’86] edges Strong edges reinforce weak edges. Weak edge removed We call a pixel an edge if it is strong. We also call a pixel an edge if it is weak but is connected to an edge. A pixel is connected to an edge if it is in a direction perpendicular to the edge normal

37. gap is gone Canny Edge Detection (Example) Strong + connected weak edges Original image Strong edges only Weak edges courtesy of G. Loy CSc 83020 3-D Computer Vision – Ioannis Stamos

38. HYSTERESIS_THRESH • Input: In, Eo, high threshold, low threshold. • Output: Lists of connected edges (contours).

39. Edge Relaxation Parallel – iterative method to adjust edge values on the basis of neighboring edges Crack edges:

40. Edge Relaxation Parallel – iterative method to adjust edge values on the basis of neighboring edges Notation: Crack edges: Edge to be updated Edge No edge Vertex types: (0) (1) (2) (3)

41. Action table for edge type: Decrease Increase Unchanged 0-0 1-1 0-1 0-2 1-2 2-2 0-3 1-3 2-3 3-3 Edge Relaxation Parallel – iterative method to adjust edge values on the basis of neighboring edges Notation: Crack edges: Edge to be updated Edge No edge Vertex types: (0) (1) (2) (3)

42. Action table for edge type: Decrease Increase Unchanged 0-0 1-1 0-1 0-2 1-2 2-2 0-3 1-3 2-3 3-3 Edge Relaxation Parallel – iterative method to adjust edge values on the basis of neighboring edges Notation: Crack edges: Algorithm: Edge to be updated Edge • 0. Compute initial confidence C0(e) of • each edge e • C0(e) = • k = 1 • Compute edge type for all e • Modify Ck(e) based on Ck-1(e) and • edge type. • If all Ck(e) have converged to 1 or 0 • else go to step 1 No edge Vertex types: (0) (1) (2) (3)

43. Canny Edge Detector Original: Lena Edges Computational Vision / Ioannis Stamos

44. Towards Global Features Local versus global CSc 83020 3-D Computer Vision – Ioannis Stamos

45. From Edges to Lines CSc 83020 3-D Computer Vision – Ioannis Stamos

46. Detecting Lines • What is the difference between line detection and edge detection? • Edges = local • Lines = nonlocal • Line detection usually performed on the output of an edge detector From Szymon Rusinkiewicz, Princeton

47. Detecting Lines • Possible approaches ? From Szymon Rusinkiewicz, Princeton

48. Detecting Lines • Possible approaches: • Brute force: enumerate all lines, check if present • Hough transform: vote for lines to which detected edges might belong • Fitting: given guess for approximate location, refine it • Second method efficient for finding unknown lines, but not always accurate From Szymon Rusinkiewicz, Princeton

49. Hough Transform • General idea: transform from image coordinates to parameter space of feature • Need parameterized model of features • For each pixel, determine all parameter values that might have given rise to that pixel; vote • At end, look for peaks in parameter space From Szymon Rusinkiewicz, Princeton

50. Hough Transform for Lines • Generic line: y = ax+b • Parameters: a and b From Szymon Rusinkiewicz, Princeton