1 / 30

2D Texture Synthesis

2D Texture Synthesis. Instructor: Yizhou Yu. Texture synthesis. Goal: increase texture resolution yet keep local texture variation. Synthesis by global statistics. Idea: Obtain statistics of input image, match with output image Histograms: normalized graph of intensity frequencies.

yoshi-mcfarland
Télécharger la présentation

2D Texture Synthesis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2D Texture Synthesis Instructor: Yizhou Yu

  2. Texture synthesis • Goal: increase texture resolution yet keep local texture variation

  3. Synthesis by global statistics • Idea: Obtain statistics of input image, match with output image • Histograms: normalized graph of intensity frequencies

  4. Synthesis by global statistics • Consider extreme case: • Histogram matching can generate two images:

  5. Using higher-order global stats • Histogram matching does not take into account spatial information. • Need additional info: • For every pixel, examine local neighborhood • Gives local but not global features (e.g. – veins) • Use higher order stats: • Correlation • average of the product of the intensities = • tells how interdependent pixel values are

  6. General procedure • 1.) Define error metric: • What do you want to match? • 2.) Match statistics • Histogram matching: • Get the density value of a pixel in the output image. Map this density to a pixel intensity in the input image  Overwrite the pixel in the output image with the mapped pixel intensity in the input image

  7. Pyramid-based Texture Analysis/Synthesis • Paper by David J. Heeger and James R. Bergen from SIGGRAPH 1995 • http://www.cns.nyu.edu/~david/ftp/reprints/heeger-siggraph95.pdf

  8. Pyramid-based Synthesis • Downsample image several times and keep track of only differences in a feature image • To reconstruct, upsample small images and add differences • Pixels in feature images are close to zero • Provides multiple scales with each feature image representing different feature sizes

  9. Laplacian Pyramid • 1-D filter: (1/16) (1 4 6 4 1) • To generalize to 2D, use tensor product • Normalize by dividing by 256

  10. Upsampling • To upsample, copy pixels to every other pixel in larger image • Fill the rest with zeroes, run low pass filter to fill in values

  11. Oriented Filters • Gaussian filters are symmetric, thus edges and contours are not detected • Use multiple oriented filters to catch non-symmetric features Left top series: Oriented filters Right image: Texture Left bottom series: Filtered textures

  12. Pyramids with Oriented Filters • Each oriented filter creates its own feature image • Thus, for each downsampled image, keep one image for each oriented filter • Each oriented filter captures one orientation of lines

  13. Matching Image Features • Input parameters: • noise: initial noisy texture • texture: texture to be matched • Output texture is stored in noise

  14. Matching Image Features • Helper functions: • MatchHistogram(noise, texture) • Matches histogram, using method described last time • MakePyramid(texture) • Create pyramid images (base and feature images) • CollapsePyramid(pyramid) • Constructs high resolution image from base and feature images

  15. Matching Image Features MatchTexture(noise, texture) { MatchHistogram(noise, texture) // first-order matching analysis_pyr = MakePyramid(texture) // create pyramid from input texture for several iterations synthesis_pyr = MakePyramid(noise) // create pyramid from noise for each feature image, fi of analysis_pyr for each feature image, fj of synthesis_pyr of same orientation MatchHistogram(fi, fj) end for end for noise = CollapsePyramid(synthesis_pyr) MatchHistogram(noise, texture) end for }

  16. Matching Image Features • MatchTexture matches histograms of feature images, not pixel values, thus providing a much better matching • Feature images already consider local neighborhoods, resulting in better approximations • Good for randomized textures • Textures with large scale features or thin and long features not matched well • e.g. stripes in wood, lines in coral

  17. Texture Synthesis Using Local Neighborhoods • Main goal is to keep local spatial coherence, but not global stats. • Randomized method • Pick pixel and copy it along with its neighborhood to random parts in synthesized image • If two neighborhoods overlap, just blend • This can result in features getting cut off if larger than local neighborhood

  18. Neighborhood-Based Texture Synthesis • Patch-Based • Patch-based sampling achieves real-time speed. [Liang et. al. 2001] • Image quilting: high-quality results and simple implementation [Efros & Freeman 2001] • Graph cut provides a powerful and refinable scheme. [Kwantra et. al. 2003] • Pixel-Based • [Efros & Leung 99], [Wei & Levoy 2000], [Ashikhmin 2001], [Hertzmann et. al. 2001], [Zhang et. al. 2003]

  19. Pixel-wise Synthesis • Grow pixel by pixel • Start from an existing patch of the texture (as opposed to noise texture like Pyramid-based Synthesis) • Look for regions in input texture most similar to current region in new texture • Copy pixels next to best-match region to expand new texture

  20. Following Raster Order • Blue region: already set by algorithm • Green region: compare this region to input image • Yellow region: closest match • Red region: replace these pixels with magenta region to maintain local integrity

  21. Hierarchical Synthesis • Build a multi-resolution pyramid for the example texture • Generate a synthesized pyramid for the output texture • At each level, follow the raster order

  22. Randomizing Synthesis • Instead of picking pixels from closest matching region, use threshold to pick a few candidate matches • Randomly pick one of the candidates • Refer to “Texture Synthesis by Non-parametric Sampling” • http://www.cs.berkeley.edu/~efros/research/synthesis.html

  23. Results

  24. Patch-Based Synthesis • Search in a sample texture for neighborhoods most similar to a context region • Merge a patch with the partially synthesized output texture

  25. The Seam Problem • Feature discontinuities may appear in patch-based synthesis.

  26. The Second Reason • Didn’t find the smoothest transition between the incoming patch and context region. • Solutions: use dynamic programming [Efros and Freeman 2001] or graph cut [Kwatra et al. 2003] to find an optimal cut. Original Warped

  27. The First Reason • Rigid template matching (SSD) often employed in the first step does NOT consider geometric similarity Original Warped

  28. Feature Map Guided Texture Synthesis • Basic Steps [Wu and Yu 2004] • Maintain an input and output feature map • Match and align curvilinear features • Integrate feature maps into template-based texture synthesis

  29. Comparisons Sample Feature Map Graphcut Quilting Texton Mask

  30. Acceleration Schemes • Fourier Transform • Acceleration techniques for nearest-neighbor search • Tree-structured vector quantization • Kd-trees • Minimizing the candidate set • Coherent synthesis, [Ashikhmin 2001] • Precomputing candidate set • K-coherent search, [Tong et al. 2002] • Jump Map, [Zelinka and Garland 2002] • GPU with parallel synthesis • [Lefebvre and Hoppe 2005] Original Warped

More Related