1 / 65

Advanced Computer Vision Introduction

Advanced Computer Vision Introduction. Lecture 02 Roger S. Gaborski. Corner Detection: Basic Idea. We should easily recognize the point by looking through a small window Shifting a window in any direction should give a large change in intensity.

drake
Télécharger la présentation

Advanced Computer Vision Introduction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Advanced Computer VisionIntroduction Lecture 02 Roger S. Gaborski Roger S. Gaborski

  2. Corner Detection: Basic Idea We should easily recognize the point by looking through a small window Shifting a window in anydirection should give a large change in intensity “flat” region:no change in all directions “edge”:no change along the edge direction “corner”:significant change in all directions Roger S. Gaborski Source: A. Efros

  3. Wikipedia: Corner Detection-No shortage of corner detectors • Moravec • Harris • Laplacian of gaussians, Difference of Gaussian and Determinant of the Hessian • SIFT • SUSAN Corner Detector • Trajkovic – Hedley Corner Detector Roger S. Gaborski

  4. Corner Detection: Weighted Sum of Differences Window function Shifted intensity Intensity Window function w(u,v) = Area of patch: (u,v) or 1 in window, 0 outside Gaussian Change in appearance for the shift [x,y]: E(x,y) = ΣΣ w(u,v) (I(u+x, v+y) – I(u,v))2 u v Roger S. Gaborski Source: Wikipedia

  5. Corner Detection: Mathematics Change in appearance for the shift [x,y]: I(u,v) E(x,y) E(3,2) E(0,0) Roger S. Gaborski Source: R. Szeliski

  6. Corner Detection Change in appearance for the shift [x,y]: • E(x,y) = ΣΣ w(u,v) (I(u+x, v+y) – I(u,v))2 We want to find out how this function behaves for small shifts Taylor expansion of I(u+x, v+y)= I(u,v)+Ix(u,v)x+Iy(u,v)y Ix and Iy are partial derivatives of x and y Results in E(x,y) = ΣΣ w(u,v) (Ix(u,v)x+Iy(u,v)y)2 Roger S. Gaborski

  7. Corner Detection: Mathematics The approximation simplifies to E(x,y) = (x,y) A (x,y)T Where Ais a second moment matrixcomputed from image derivatives: A = ΣΣ w(u,v) Ix2IxIy IxIy Iy2 Roger S. Gaborski

  8. Interpreting the second moment matrix Analyze eigenvalues of A to determine if a corner exists If λ1and λ2= 0, then pixel (x,y) is not of interest If one eigenvalue λ is close to 0, and the other is large, (x,y) is on an edge If both eigenvalues λ are large, (x,y) is a corner Roger S. Gaborski

  9. Interpreting the eigenvalues Classification of image points using eigenvalues of M: 2 “Edge”2 >> 1 “Corner”1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions “Edge”1 >> 2 “Flat” region 1 Roger S. Gaborski

  10. Defining Corner Response Function, R Instead of calculating eigenvalues, Harris defined the following function: R = det(A)- α trace(A)2 = λ1λ2 - α (λ1 + λ2) Where α = .04 to .06 Recall: A = [ a b; c d] Det(A) = ad – bc Roger S. Gaborski

  11. Corner Response Function R < 0 R > 0 Edge Corners |R| small R < 0 “Flat” region Edge Roger S. Gaborski

  12. Harris Detector Algorithm • Compute Derivatives at each point • Compute Second Moment Matrix A • Compute Corner Response Function • Threshold R • Find Local Maxima Roger S. Gaborski

  13. Harris Corner Detector • Reference: C.G. Harris and M.J. Stephens “A Combined Corner and Edge Detector” • Code inspired by Peter Kovesi • Derivative Masks: dx = [-1, 0, 1;-1, 0, 1;-1, 0, 1] • dy = dx’ • Image Derivatives; • Ix = imfilter(im, dx, 'conv',‘same’); • Iy = imfilter(im, dy, 'conv',‘same’); • Gaussian Filter • g = fspecial(‘gaussian’, 6*sigma, sigma); Roger S. Gaborski

  14. Smooth squared image derivative • Ix2 = imfilter (Ix.^2, g, 'conv', ‘same’); • Iy2 = imfilter (Iy.^2, g, 'conv', ‘same’); • IxIy = imfilter (Ix .* Iy, g, 'conv', ‘same’); c = (Ix2.*Iy2 – IxIy.^2)./(Ix2+Iy2).^2; Roger S. Gaborski

  15. Non-maximal Suppression and Threshold • Extract local maxima – gray scale morphological dilation • size = 2*radius+1; %radius is parameter • mx = imdilate(c,ones(size)); %gray scale dilate • cc = (c==mx)&(c>thresh); %find maxima • [r,c] = find(cc) %find row, col coordinates • figure, imagesc( im), colormap(gray) • hold on • plot(c,r, ‘rs’), title(‘Corners) Roger S. Gaborski

  16. 100x100 Grid background =1, lines = 1 Roger S. Gaborski

  17. Roger S. Gaborski

  18. Image rotated 45 Degrees Roger S. Gaborski

  19. Image rotated 45 Degreessame parameters Roger S. Gaborski

  20. Roger S. Gaborski

  21. Roger S. Gaborski

  22. Porsche Image Roger S. Gaborski

  23. Harris Points Roger S. Gaborski

  24. 1983 Porsche Roger S. Gaborski

  25. HW#3 – Due Tuesday, 12/13 10am • Work in teams of 2 or 3 • Write a Harris Detector Function (do not simply copy one from web, write your own) • Experiment with ‘grid image’ and Flower2 image and two ‘interesting’ images of your choice. NOTE: Create your own grid image using Matlab (for example, a matrix of all zeros with white lines, pixels = 1) • Goals: - Find all intersections on grid image • Detect all petal end points on flower image – better results that class lecture slide • Email: rsg.advcv@gmail.com • 1- write up including result images, observations and 2- MATLAB code Roger S. Gaborski

  26. Object Recognition • Issues: • Viewpoint • Scale • Deformable vs. rigid • Clutter • Occlusion • Intra class variability Roger S. Gaborski

  27. Goal • Locate all instances of automobiles in a cluttered scene Roger S. Gaborski

  28. Acknowledgements • Students (Thesis in RIT Library): • Tim Lebo • Dan Clark • Images used in presentation: • ETHZ Database, UIUC Database Roger S. Gaborski

  29. Object Recognition Approaches • For specific object class: • Holistic • Model whole object • Parts based • Simple parts • Geometric relationship information • We could use a similar approach to match patches representing different image categories (‘sand’ patches located on lower half of beach scenes) Roger S. Gaborski

  30. Training Images and Segmentation Roger S. Gaborski

  31. Implicit Shape Model • Patches – local appearance prototypes • Spatial relationship – where the patch can be found on the object • For a given class w: ISM(w) = (Iw ,Pw ) where Iw is the codebook containing the patches and Pw is the probability distribution that describes where the patch is found on the object • How do we find ‘interesting’ patches? Roger S. Gaborski

  32. Harris Point Operator Roger S. Gaborski

  33. Harris Points Roger S. Gaborski

  34. Segmented Training Mask Segmented mask ensures only patches containing valid car regions are selected A corresponding segmentation patch is also extracted Roger S. Gaborski

  35. Selected Patches Roger S. Gaborski

  36. How is spatial information represented? • Estimate the center of the object using the centroid of the segmentation mask • Displacement between: • Center of patch • Centroid of segmentation mask Roger S. Gaborski

  37. Individual Patch and Displacement Information Roger S. Gaborski

  38. Typical Training Example Roger S. Gaborski

  39. Typical Training Example Roger S. Gaborski

  40. Extracted Training Patches Roger S. Gaborski

  41. Cluster Patches • Many patches will be visually similar • Normalized Grayscale Correlation is used to cluster patches • All patches within a certain neighborhood defined by the NGC are grouped together • The representative patch is determined by mean of the patches • The geometric information for each patch in the cluster is assigned to the representative patch Roger S. Gaborski

  42. Patches Roger S. Gaborski

  43. Wheel Patch Example Roger S. Gaborski

  44. Clusters Opportunity for better clustering method Roger S. Gaborski

  45. Clusters Roger S. Gaborski

  46. Roger S. Gaborski

  47. Object Detection • Harris point operator to find interesting points • Extract patches • Match extracted patches with model patches • Spatial information predicts center of object • Create voting space Roger S. Gaborski

  48. Ideal Voting Space Example Roger S. Gaborski

  49. Multiple Votes Multiple geometric interpretations Roger S. Gaborski

  50. Resolving False Detections Roger S. Gaborski

More Related