Lecture 06

1. Lecture 06 06/12/2011 ShaiAvidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.

2. Today • Local invariant features • Detection of interest points • Harris corner detection • Scale Estimation

3. What would we like our local features to be invariant to? Geometric Photometric Noise Blur Compression Artifacts

4. Main questions • Where will the interest points come from? • What are salient features that we’ll detect in multiple views? • How to describe a local region? • How to establish correspondences, i.e., compute matches?

5. Finding Corners • Key property: in the region around a corner, image gradient has two or more dominant directions • Corners are repeatable and distinctive C.Harris and M.Stephens. "A Combined Corner and Edge Detector.“Proceedings of the 4th Alvey Vision Conference: pages 147--151.  Source: Lana Lazebnik

6. Local measures of uniqueness • Suppose we only consider a small window of pixels • What defines whether a feature is a good or bad candidate? Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.

7. Feature detection • Local measure of feature uniqueness • How does the window change when you shift by a small amount? “corner”:significant change in all directions “flat” region:no change in all directions “edge”: no change along the edge direction Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.

8. Feature detection: the math • Consider shifting the window W by (u,v) • how do the pixels in W change? • compare each pixel before and after bySum of the Squared Differences (SSD) • this defines an SSD “error” E(u,v): W

9. Small motion assumption • Taylor Series expansion of I: • If the motion (u,v) is small, then first order approx is good • Plugging this into the formula on the previous slide…

10. = Feature detection: the math • Consider shifting the window W by (u,v) • how do the pixels in W change? • compare each pixel before and after bysumming up the squared differences • this defines an “error” of E(u,v): W

11. Quick eigenvalue/eigenvector review • The eigenvectors of a matrix A are the vectors x that satisfy: • The scalar  is the eigenvalue corresponding to x • The eigenvalues are found by solving: • In our case, A = H is a 2x2 matrix, so we have • The solution: • Once you know , you find x by solving

12. Feature detection: the math This can be rewritten: x- x+ • Eigenvalues and eigenvectors of H • Define shifts with the smallest and largest change (E value) • x+ = direction of largest increase in E. • + = amount of increase in direction x+ • x- = direction of smallest increase in E. • - = amount of increase in direction x+

13. Feature detection: the math • How are +, x+, -, and x- relevant for feature detection? • What’s our feature scoring function? • Want E(u,v) to be large for small shifts in all directions • the minimum of E(u,v) should be large, over all unit vectors [u v] • this minimum is given by the smaller eigenvalue (-) of H

14. Feature detection summary • Here’s what you do • Compute the gradient at each point in the image • Create the H matrix from the entries in the gradient • Compute the eigenvalues. • Find points with large response (-> threshold) • Choose those points where - is a local maximum as features

15. Feature detection summary • Here’s what you do • Compute the gradient at each point in the image • Create the H matrix from the entries in the gradient • Compute the eigenvalues. • Find points with large response (-> threshold) • Choose those points where - is a local maximum as features

16. The Harris operator • - is a variant of the “Harris operator” for feature detection • The trace is the sum of the diagonals, i.e., trace(H) = h11 + h22 • Very similar to - but less expensive (no square root) • Called the “Harris Corner Detector” or “Harris Operator” • Lots of other detectors, this is one of the most popular • Here’s another variant: α: constant (0.04 to 0.06)

17. Interpreting the eigenvalues Classification of image points using eigenvalues of M: 2 “Edge” 2 >> 1 “Corner”1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions “Edge” 1 >> 2 “Flat” region 1

18. Corner response function α: constant (0.04 to 0.06) “Edge” R < 0 “Corner”R > 0 |R| small “Edge” R < 0 “Flat” region

19. Harris Detector: Workflow Slide adapted form Darya Frolova, Denis Simakov, Weizmann Institute.

20. Harris Detector: Workflow Compute corner response R

21. Harris Detector: Workflow Find points with large corner response: R>threshold

22. Harris Detector: Workflow Take only the points of local maxima of R

23. Harris Detector: Workflow

24. Harris Detector: Properties Rotation invariance Ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner response R is invariant to image rotation

25. Harris Detector: Properties Not invariant to image scale All points will be classified as edges Corner !

26. How can we detect scale invariant interest points?

27. Scale-invariant features: Blobs

28. Recall: Edge detection Edge f Derivativeof Gaussian Edge = maximumof derivative Source: S. Seitz

29. Edge detection, Take 2 f Edge Second derivativeof Gaussian (Laplacian) Edge = zero crossingof second derivative Source: S. Seitz

30. From edges to blobs Edge = ripple Blob = superposition of two ripples maximum Spatial selection: the magnitude of the Laplacianresponse will achieve a maximum at the center ofthe blob, provided the scale of the Laplacian is“matched” to the scale of the blob

31. Scale selection We want to find the characteristic scale of the blob by convolving it with Laplacians at several scales and looking for the maximum response However, Laplacian response decays as scale increases: original signal(radius=8) increasing σ Why does this happen?

32. Scale normalization The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases

33. Scale normalization The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases To keep response the same (scale-invariant), must multiply Gaussian derivative by σ Laplacian is the second Gaussian derivative, so it must be multiplied by σ2

34. Effect of scale normalization Scale-normalized Laplacian response maximum Original signal Unnormalized Laplacian response

35. Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D

36. Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D Scale-normalized:

37. Scale selection At what scale does the Laplacian achieve a maximum response for a binary circle of radius r? r image Laplacian

38. Scale selection The 2D Laplacian is given by Therefore, for a binary circle of radius r, the Laplacian achieves a maximum at (up to scale) Laplacian response r scale (σ) image

39. Characteristic scale We define the characteristic scale as the scale that produces peak of Laplacian response characteristic scale T. Lindeberg (1998). "Feature detection with automatic scale selection."International Journal of Computer Vision30 (2): pp 77--116.

40. What Is A Useful Signature Function? Laplacian-of-Gaussian = “blob” detector 40 K. Grauman, B. Leibe K. Grauman, B. Leibe

41. Characteristic scale • We define the characteristic scale as the scale that produces peak of Laplacian response characteristic scale T. Lindeberg (1998). "Feature detection with automatic scale selection."International Journal of Computer Vision30 (2): pp 77--116. Source: Lana Lazebnik

42. Laplacian-of-Gaussian (LoG) s5 s4 s3 s2  List of(x, y, σ) s • Interest points: Local maxima in scale space of Laplacian-of-Gaussian K. Grauman, B. Leibe K. Grauman, B. Leibe

43. Scale-space blob detector: Example Source: Lana Lazebnik

44. Scale-space blob detector: Example Source: Lana Lazebnik

45. Scale-space blob detector: Example Source: Lana Lazebnik