1 / 20

Image Mix and Match

Image Mix and Match. Internet images Colorization Most similar image searching Collage Object insertion. Data-driven approach for robust similarity measure Cross domain(Photo, Photo with different lighting, Painting) No domain specific treatments 異種画像に利用できる画像の類似度計算法. Idea

raisie
Télécharger la présentation

Image Mix and Match

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Mix and Match • Internet images • Colorization • Most similar image searching • Collage • Object insertion

  2. Data-driven approach for robust similarity measure • Cross domain(Photo, Photo with different lighting, Painting) • No domain specific treatments • 異種画像に利用できる画像の類似度計算法

  3. Idea • Detect unique region of the target image (comparing to the other) • Place high weight on the unique regions • 与えられた画像のどの特徴が，Web上の膨大な量の画像に対してユニークかを学習 • ユニークな画像特徴に重みを置く

  4. Image feature vector (画像特徴ベクトル) • Intensity histogram, Gradient Magnitude Histogram, HoG, SIFT • 画像間距離は，この特徴ベクトル間の距離として計算する事が多い Intensity histogram HoG, Histogram of oriented gradient，4k-5k dimension • Linear support vector machine (see Pattern recognition textbook) • Given d-dimensional feature vectors belongs to class A and B, xi∊A, yi∊B xi, yi∊Rd yi Find the maximum-margin hyperplane that divides xi∊A, yi∊B xi • この絵は２Ｄだけど本当は特徴ベクトルと同じだけの次元，5000次元とか

  5. Goal • Given a image Ip • Detect unique parts of feature vector of Ipcomparing to the others • Place high weight on the unique regions Ip 1) Compute feature vectors of Ipxpand the other internet images xi 2) Compute hiperplane by linear SVM 3) Project feature vectors of all images onto the normal of the hiperplane このnormalがPCAの軸のような役割になる Other images on Web

  6. Image colorization from internet image • Gray scale画像の色付けを，Web上の画像を参照して行う • Combination of many techniques • Internet image search, foreground segmentation, suitable image filtering, Image similarity measure, graph-based color transfer, selection UI for weight tuning • Input: Image + text label (e.g. rooster)

  7. Procedure • Input: grayscale image with foreground segmentation & text label • Search images from internet • Google image search / Flickr • Automatic fore ground extraction • Filter similar images for back/fore ground(Ad hoc energy function intensity, texture, density of SIFT) • Color transfer • Graph based color transfer method • Maintain the neighborhood consistency • Compute with different weighting values  The user can select one of them • Output: Colored images with different reference images and different weighting values

  8. Color transfterの計算時に，gray画像と参照画像の両方とも，微小領域に分割し，近傍に矛盾がないようなcolor transferを計算している．例えば蝶の羽などでは，黄色の隣にはオレンジ色が来る事は無いなど，良い結果を生んでいる．

  9. Arcimboldo-style collage generation • Input : Source image and text label for searching element image cutouts • Output: collage consists from internet images

  10. Giuseppe Arcimboldo 1527-1593 • Itary • Collage like drawing • Each element is recognizable (elements are taken from a same theme) • The assembly of the elements resembles something

  11. Best fitting cutout search • Input image segmentation Two problems 1) Search images from internet 2) Cutout foreground image by saliency detection and GrabCut 3) Distance metric between - Hole in the target image - Cutout image from internet Color distance term / Shape distance term With best fitting affine transformation 1) Mean shift clustering Compute modes in color & space feature space エッジ保存フィルタを連続して書けるような物 2) Marge & split strange local regions -Mean shift clustering generates local regions that not match any element cutout image from internet -Trim such regions by Ad hoc iteration Semantic awaresegmentation is difficult…

  12. New tool for inserting objects into Photographs • geometry and light estimation with user’s guide • Less user interaction • Deal with interior light and exterior lights (e.g. sun light from window) • ある点がすごい新しいとかではなく，他と比較して全体的なパフォーマンスが上がっている感じ • Inputs • Single image • User annotation (geometry, light source position) • 3D model that will be inserted into the Image

  13. Overviewof the system • Geometry estimation • Previous work + user’s correction, user interface to add other geometry • Light source estimation • Next page • Object insertion • Add object into 3D scene and render it with estimated parameter

  14. = + Interior light estimation 1) Decompose input image into Albedo and direct light image 2) User points the position of the light 3) Automatically adjust light parameter (position & RGB) !!Full automatic light source detection from single image is very difficult!! Exterior light estimation 1)The user marks boundary of the source and projection of the exterior light 2) The system automatically computes mask and direction

  15. 今年のSIGGRAPHに • Image = Albedo+ Direct • Image = Albedo + Indirect + Directという分解をする論文あった

  16. Results

More Related