1 / 25

Fast Interactive Image Segmentation by Discriminative Clustering

Fast Interactive Image Segmentation by Discriminative Clustering. Dingding Liu * Kari Pulli † Linda Shapiro * Yingen Xiong † † Nokia Research Center, Palo Alto, CA 94304, USA *Dept. Elect. Eng., University of Washington, WA 98095, USA. Research Aim.

paco
Télécharger la présentation

Fast Interactive Image Segmentation by Discriminative Clustering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Fast Interactive Image Segmentation by Discriminative Clustering Dingding Liu * Kari Pulli † Linda Shapiro * YingenXiong † † Nokia Research Center, Palo Alto, CA 94304, USA *Dept. Elect. Eng., University of Washington, WA 98095, USA

  2. Research Aim • Cut out an object from its background fast • Computation time – so can quickly iterate • With as few strokes as possible

  3. Overview • Introduction • Motivation • Related work • Algorithm • Pre-segmentation by the Mean-Shift algorithm • Merge regions by discriminative clustering • Local neighborhood region classification and pruning • Experiments and Results • Conclusions and Future Work

  4. Introduction • Motivation: Image editing on mobile devices • Convenience • Anytime, anywhere • Challenges • Limited computational resources • Smaller screens and imprecise input

  5. Related Work -Interactive Image Segmentation • Lazy Snapping • Li et al., ACM Transactions on Graphics 2004

  6. Related Work -Interactive Image Segmentation • Interactive Image Segmentation by Maximal Similarity Based Region Merging • Ning et al., Pattern Recognition 2010 Insufficient user inputs Sufficient user inputs

  7. Algorithm: Summary Pre-segmentation by the Mean-Shift algorithm Merge regions by discriminative clustering Local neighborhood region classification and pruning

  8. Background: Mean-Shift Segmentation http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html http://robots.stanford.edu/cs223b04/CS%2520223-B%2520L11%2520Segmentation.ppt

  9. The Basic Mean-Shift Algorithm • Choose a search window size • Choose the initial location of the search window • Compute the mean (centroid of the data) within the search window • Center the search window at that mean location • Repeat 3 and 4 until convergence The mean shift algorithm seeks the“mode”or point of highest density of a data distribution

  10. Mean-Shift Segmentation • Convert the image into tokens (via color, gradients, texture measures, etc.) • Choose initial search window locations uniformly in the data • Compute the mean shift window location for each initial position • Merge windows that end up on the same “peak” or mode • Repeat 3 and 4 until convergence

  11. Mean-Shift Segmentation Results

  12. Algorithm: Pre-segmentation using Mean-Shift • Three reasons for choosing the Mean-Shift algorithm: • 1. It preserves the boundaries better than other methods • 2. Its speed has been improved significantly in recent years Pre-segmentation can be done either before or after the user input • 3. Fewer parameters to tune

  13. Algorithm: Merge non-ambiguous regions df > dthresh + db, background df + dthresh < db, foreground Otherwise, ambiguous regions Only consider color, not location • Create two kd-trees in CIELab color space • One for the marked foreground, another for the background regions • For each unmarked region, find the color difference to • the most similar marked background dband foreground region df • Choice of dthresh : • Min difference of mean colors between the marked foreground and background • that is higher than a minimum allowed distance (we chose 2)

  14. Algorithm: Assign ambiguous regions • Now use also location information • Each of the remaining ambiguous regions is assigned • the label of the neighboring region that has the most similar mean color • If the most similar neighboring region is also an unmarked region • merge them to a new unmarked region, repeat the process • If there is a tie in the mean color for assignment to foreground and background • the label of the region that has the most similar color variance is used

  15. Algorithm: Prune / flip isolated regions Find isolated foreground or background regions (use connected components) • Regions are changed to the opposite label when all of the following hold: • The region is not marked by the user (b) The region is not the biggest region with that label (c) The region is smaller than its surrounding regions

  16. Results: Segmentation time – in numbers

  17. Results: Segmentation time – as a graph

  18. Why are we faster? • Two main reasons • No iterative steps in the first stage • and not too many in the second or third • do the easy choices quickly • fast nearest-neighbor lookups with kd-trees • graph-cut on many regions is slow, MSRM iterates unnecessarily much • Merging the region descriptor is fast • only mean and standard deviation of colors • MSRM has complicated 4K bin color histograms to merge

  19. Results: Segmentation time on phone

  20. Results: The best segmentation quality (a) Input image (c) Maximal Similar Region Merging (d) Proposed method (b) Graph-cut over regions

  21. Results: The best segmentation quality (b) Graph-cut over regions (c) Maximal Similar Region Merging (d) Proposed method (a) Input image

  22. Results: The best segmentation quality (b) Graph-cut over regions (c) Maximal Similar Region Merging (d) Proposed method (a) Input image

  23. Results: Video Demo

  24. Conclusions and Future Work • A new region-based interactive image segmentation algorithm • Significantly increases the speed of segmentation • by avoiding global optimization and long iterations • Does not compromise the segmentation quality • Uses a region mean color instead of a single pixel color • Future Work • Further decrease the users input • Combine the individual pixel information to further improve the algorithm

  25. Thank you! • Questions?

More Related