1 / 58

OpenCV

Università del Salento Facoltà di Ingegneria Image Processing (Elaborazione delle Immagini) A.A. 2012/2013. OpenCV. PART II – Two case studies. Dario Cazzato, INO – CNR dario.cazzato@ino.it. Introduction. This lesson introduce the use of the OpenCV library on real cases .

beulah
Télécharger la présentation

OpenCV

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Università del Salento Facoltà di Ingegneria Image Processing (Elaborazione delle Immagini) A.A. 2012/2013 OpenCV PART II – Twocase studies Dario Cazzato, INO – CNR dario.cazzato@ino.it

  2. Introduction • Thislesson introduce the useof the OpenCVlibrary on realcases. • Twostudycases: • Stereo CorrespondenceProblem; • Segmentationof Video Sequences.

  3. WhatWe’ve seen • WhyOpenCV? • Free, open source, for real-time application, cross-platform, constantlyupdated, strong partners and research. • C/C++: • “cv::Mat vs CvMat”. • Basiccomponents: • Matrixes, vectors, rectangles, sizes, images, datatype… • Some example.

  4. Stereo CorrespondanceProblem

  5. Whatis Stereo Vision? • Men: • Binocular vision. • Averagedistancebetweeneyes: 6cm. • An object/pointseenwitheyesisviewedasone, altough in the retina Wehavetwoimages. • The combined image is more than the sum of its parts. It’snottrivial!

  6. Whatis Stereo Vision? • Otherconfigurations: • Animals: • Predator: binocularsight. • Prey: lateraleyes (toenlarge the fieldofsight). • Intersectinglineofsight (typical in stereo vision!). • Let’ssee some key conceptfrom a practicalpointofview, butall the problemisabsolutelylarger! • Youwillsee more detailaboutepipolargeometry and matrixescomputation at lesson.

  7. Pinhole camera model • The simplestmodelof the camera. • Only a single rayentersfromanyparticularpoint, the pinhole aperture. • Thispointisprojectedonto the imageplane. • The focallenghtis the distancefrom the pinhole aperture to the imageplane.

  8. Pinhole camera model • A realpointQisprojectedonto the imageplaneby the raypassingthrough the center ofprojection. • Thisintersectiongives q. Calibration Matrix (3X4) From “LearningOpenCV”, G.Bradski, A.Kaehler, O’Reilly.

  9. Pinhole camera model • Homogeneous coordinate system. • IfyouhaveNdimension, use N+1 coordinates. Homography PerspectiveGeometry

  10. Whatis Stereo Vision? 1 Camera isnotenough!

  11. Whatis Stereo Vision? Withtwo (or more) cameraswe can computedepthbytriangulation, ifwe are abletofindhomologouspoints in the twoimages. EpipolarGeometry

  12. Whatis Stereo Vision? • Foursteps: • Undistortion; • Rectification; • DisparityMap; • Triangulation.

  13. Whatis Stereo Vision? • Undistortion: removaloftangential and radiallensdistortion. Thisproblemconcerns the single camera! Distortionvector (1X5)

  14. Whatis Stereo Vision? • Undistortion: removaloftangential and radiallensdistortion.

  15. Whatis Stereo Vision? • Rectify: output row-alignedimages (coplanar, with the same y-coordinate). Withrectifiedimages, we can searchfor a point in oneimage in the sameline (y-coordinate) of the secondone!

  16. Whatis Stereo Vision? • Ofcourse, a stereo calibrationisneeded (extrinsic and intrinsicparameters): • Intrinsic: focallenght, distortion. • Extrinsic: MatrixesR, Tthataligns the twocameras (Essential Matrix E, youwillsee more at lesson). • We can divide the procedure in : • Stereo Calibration: computationof the geometric relations between the twocameras in space; • Stereo Rectification: “correction” ofindividualimagesasmadewithrow-alignedimageplane and parallelopticalaxes.

  17. Whatis Stereo Vision? Look at the exampleofOpenCV, and the source stereo_calib.cpp. Rectificationturns the cameras in standard form!

  18. Whatis Stereo Vision? Example1

  19. Whatis Stereo Vision? From “LearningOpenCV”, G.Bradski, A.Kaehler, O’Reilly.

  20. Whatis Stereo Vision? • Disparitymap: difference in x-coordinatesof the samepointviewed in the 2cameras. A mapiscreatedcomputing the disparityforall the points. It’sencodedas a grayscaleimage, wherefarerpoint are darker.

  21. Whatis Stereo Vision? • Triangulation: difference in x-coordinatesof the samepointviewed in the 2cameras. Idea: (d:T = f:Z)

  22. Stereo CorrespondenceProblem • Howtofindhomologouspoints? • Correlation-based - checking if one location in one image looks/seems like another in another image; • Feature-based - finding features in the image and seeing if the layout of a subset of features is similar in the two images.

  23. So, why so challenging? • Occlusions

  24. So, why so challenging? • Photometric transformations

  25. So, why so challenging? • Uniformregions

  26. So, why so challenging? • Noise • Specular surfaces • Perspectiveviews • Repetition • “No resultspossible” detection: • Sometimewe just wouldliketosay: “No correspondentpoint in the otherimageforthispoint” • …

  27. Stereo CorrespondenceProblem LocalAlgorithm Winner take allstrategy

  28. WTA Strategy • Sum ofAbsoluteDifference: • Sum ofSquaredDifferences:

  29. WTA Strategy • Zero-meanNormalized Cross Correlation: • Not just the window, but a fast normalization; • ZNCC hasrange [-1,1]; • Wecompute the ZNCC foreach pixel center; • We take the maxvalue.

  30. ZNCC

  31. ZNCC • Whatwe can do toenhounce the model? • Ratiobetween first and secondmax; • Idea behind: if the maximum and a localmaximumhavesimilarvalue, the probabilityoferrorincrease, and wecouldrejectthesevalues (putting a treshold). • Spread; • Idea behind: a flat area meansrepetitivetexture. Just discardmaximum in flatpeaks. • Multiple windows; • Kernel shapebased on segmentation. • Computationtimeincreases!

  32. ZNCC –Ourmodel • Twoenhouncements: • Check the epipolarline±size: • We can deal withnoise in the epipolargeometry; • For a fast computation, keepsizesmall! (1,2,3). • Inverse function: • We take the maximum, and wemake ZNCC againstartingfrom the right image; • If the newwinnerisn’t the startingpoint (or is more than a treshold far, anerroroccurred, so discard the point). Demo 1

  33. Segmentationof Video Sequences

  34. Background subtraction • Oneof the Video SequencesSegmentationalgorithms. • Goodwithfixed camera and static background. • High level goal: • People detection. • Surveillance: • Reactive; • Proactive.

  35. From the Web

  36. CodebookModel • BS: subtract the current frame from the background model. • Twophases: • Background training; • Foreground detection. • Improvementsto the base version.

  37. Segmentation - background model • A codebookisbuiltforevery pixel; • A codebookiscomposedbycodewords, boxes; thatgrowto cover the common valuesseenover the time; • Samplesofeach pixel are clustered in set ofcodewords; • Incoming pixel: • Ithas a brightness in the brightnessrange AND Color Distortionlessthan a treshold = BACKGROUND; • Othervise FOREGROUND.

  38. Segmentation–codebookmodel

  39. Segmentation - mnrl MNRL (Maximum Negative RunLenght): letustomake the background learningwithobjectsmovement. Itrefinescodebookseparatingcodebooksthat can haveforegroundfrom the real background. MNRL = 50%.

  40. Segmentation–detectingforeground The foregroundissimplydetectedcomputing the distanceof the sample from the nearest cluster mean.

  41. Segmentation–codebookalgorithm

  42. Segmentation–codebookimprovements Leftobject in the scene:

  43. Segmentation–codebookimprovements Holesproblem:

  44. Segmentation–codebookimprovements • LayeringModeling/Detection - 3classesofcodebook and 3parametersthatlettoswitch in the categories: • Permanent; • Non-permanent; • Training. • AdaptativeCodebookUpdating: • Retrainingisnot the solution!! • Global status updating at each frame; • Periodicalcleaniningof the oldcodebook.

  45. Filtering the binary frame Medianfilter:

  46. Filtering the binary frame Medianfilter:

  47. Filtering the binary frame MorphologicalOperators Opening and Closing:

  48. Filtering the binary frame Opening and Closing:

  49. Filtering the binary frame

  50. Filtering the binary frame

More Related