1 / 29

Objective:

Objective:. Find correspondence between two image Why? Motion analysis Tracking Stereo 3D reconstruction Object detection Image enhancement Video editing … …. Reference Source Correspondence. Why else?. Restoration.

zofia
Télécharger la présentation

Objective:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Objective: • Find correspondence between two image • Why? • Motion analysis • Tracking • Stereo • 3D reconstruction • Object detection • Image enhancement • Video editing • … • … Reference Source Correspondence

  2. Why else? Restoration • Huge digital photo database availableon line. • A lot of redundancy in personalphoto collections. 3D Deblurring

  3. Correspondence-Land: Assumptions dViewpoint = Small dTime = Small Commoncontent Non-rigid transform dViewpoint = Small Mostly rigid objects Result: Dense: Optical Flow Stereo reconstruction Sparse: SIFT Dense

  4. Main contestants – Sparse SIFT: • Find Feature points + descriptors. • Use Scale to get region. • Dense result! • Properties: • Robust and accurate! • Limitations: • Smooth regions undetected • Fixed scale = bad matches.

  5. Main contestants – SIFT Flow: Optical Flow Energy: • Dense  • Handles large geometrical • distortions poorly  SIFTFlow Energy:

  6. Main algorithm - abstract: For scale = coarse to fine do end

  7. Nearest-neighbor search: Objective: For each 8x8 patch Find Where Features per pixel: Final wanted result: For example:

  8. PatchMatchCorrespondence Algorithm: • Iterative two stage operation: • 1. Random Search • 2. Propagation Barnes, Connelly, et al. "PatchMatch: a randomized correspondence algorithm for structural image editing." ACM Transactions on Graphics-TOG 28.3 (2009): 24.

  9. Generalized PatchMatch: • PatchMatchGeneralizedPatchMatch • Searching for : • Candidates: • Example: Barnes, Connelly, et al. "The generalized patchmatch correspondence algorithm.“ Computer Vision–ECCV 2010. Springer Berlin Heidelberg, 2010. 29-43.

  10. Nearest-neighbor search – cont.: Objective: For each 8x8 patch Find Where - Extracted with PatchMatch. Bias and Gain:

  11. Aggregating consistent regions: Objective: Find reliable matches, remove outliers. Define a consistency error measure:

  12. Aggregating consistent regions: Nearest neighbor search result:

  13. Aggregating consistent regions: 1.2 1.7 3.6 0.5 Apply consistency threshold:

  14. Aggregating consistent regions: Remove incoherentregions:

  15. Progress so far: After NN search After region aggregation Source

  16. Global color mapping: • Why? • Improve performance of correspondencealgorithm. • Produce color matched reference image. • Color Model: • Piecewise cubic spline with 7 breaks per RGB channel. Red Channel 80 50

  17. Search constraints: • Our transformation contains 12 DOF Match ambiguity increased. • Solution: Limit search range for . • Constrained Locally, inside reliable regions. • of LabG Constrained Globally, usingreliable regions.

  18. Evaluation - Correspondence: • Compared to: • GPM – Their setup, same features ( ) • But without: Multi-scale, region aggregation, search narrowing. • Sparse SIFT – 15 pixels radios around the descriptor centers. • SIFT-Flow – [Liu et al. 2008a] Viewpoint Display Metric: Correct if Zoom+rotation

  19. Evaluation - Correspondence: Viewpoint Zoom + rotation

  20. Evaluation - Correspondence:

  21. Evaluation - Correspondence: Correct if Mikolajczyk et al. Dataset

  22. Evaluation – Global color transfer: SIFT, Main Idea: Global color transfer, SIFT scale. Pitie, Main Idea: 1D pdf matching + Gradient Adj.

  23. Evaluation – Global color transfer:

  24. Applications – Local color transfer: • Global– transform using original Global transform. • Local- 1. Apply local adjustments inside reliable regions (Histogram • matching). • 2. Propagate to other pixels using Poisson blending.

  25. Applications – Mask transfer: • Find correspondence of Background and Foreground patches. • Use result as initialization for Grab-Cut [Rother et al. 2004]

  26. Applications – Deblurring: • Kernel Estimation: using modified Cho and Lee [2009] with a validity mask. • Deconvolution: using modified Levin et al. [2007]. • Source is color matched.

  27. For Conclusion:: • Pros: • Authors present a novel useful correspondence method. • Consistent regions aggregation – Great Idea! • Cons: • Experimental evaluation – Lacks large scale comparison, Focused only on superior cases. • Did not prove superiority over Co-Recognition*

  28. Future directions: • Aid dense 3D reconstruction? • Color transfer between different images (same composition)? • Extract geometric properties of objects in the scene. Thank you!

More Related