1 / 22

Nonparametric Bayesian Texture Learning and Synthesis

Nonparametric Bayesian Texture Learning and Synthesis. Leo Zhu Yuanhao Chen William Freeman Antonio Torralba Presented at NIPS 2009 Discussion and Slides by Eric Wang August 13, 2010. Wang 2010. Outline. Introduction Image Patches and Features HDP-2DHMM Inference Texture Synthesis

cadam
Télécharger la présentation

Nonparametric Bayesian Texture Learning and Synthesis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nonparametric Bayesian Texture Learning and Synthesis Leo Zhu Yuanhao Chen William Freeman Antonio Torralba Presented at NIPS 2009 Discussion and Slides by Eric Wang August 13, 2010 Wang 2010

  2. Outline • Introduction • Image Patches and Features • HDP-2DHMM • Inference • Texture Synthesis • Image Segmentation and Synthesis • Conclusion Wang 2010

  3. Introduction • The authors consider texture learning and synthesis with spatial dependence. • Two basic approaches to this problem have been considered • Represent texture using textons and spatial layout. This approach is sensitive to parameter settings, has low rendering quality, and is slow. • Patch based approaches offer improved rendering quality and are faster, but do not have semantic understanding and texture modeling. • In this paper, the authors adopt a patch based approach, and augment it with nonparametric Bayesian modeling and statistical learning. A spatial HMM (2D-HMM) whose states (texton vocabulary) are generated from an HDP. • Once the parameters of the HDP-2DHMM are learned, large textures can be synthesized according to the spatial transition matrix and dictionary of textures. Wang 2010

  4. Introduction Wang 2010

  5. Image Patches and Features • Each patch is characterized by a set of filter responses that correspond to values b of image filter response h at location l. • Specifically, each 24x24 patch is divided into 6x6 cells (l=1,…,36), each of which is size 4x4 pixels. For each pixel in cell l, the response to h=37 image filter responses are computed. • For each 4x4 cell, the responses to each filter are averaged across all pixels and quantized (binned) into 1 of b=6 bins. Therefore, each patch is represented by 37x36x6=7992 dimensional feature responses. • The authors point out that, unlike previous work that used Kmeans to first form a visual vocabulary, their approach directly applies a nonparametric model Wang 2010

  6. HDP-2DHMM • Let L(i), T(i), R(i), D(i) denote the four neighbors of node i. • In the above graphical model, only L(i) and T(i) dependencies are shown. • indexes the state of node i, and associates it with the cluster label of a texton. • and are the emission and transition parameters, respeectively Wang 2010

  7. HDP-2DHMM Wang 2010

  8. HDP-2DHMM Wang 2010

  9. HDP-2DHMM Wang 2010

  10. HDP-2DHMM Wang 2010

  11. HDP-2DHMM Wang 2010

  12. HDP-2DHMM Wang 2010

  13. HDP-2DHMM Wang 2010

  14. HDP-2DHMM Wang 2010

  15. HDP-2DHMM • The authors also allow the textons to be slightly shifted from their original locations by introducing two hidden variables that indicate the displacements of textons relative to location i. • For simplicity, the shift variables are given a uniform prior over a small shift neighborhood (10% in either direction). Wang 2010

  16. Inference • Collapsed Gibbs sampling is used to learn the parameters of HDP-2DHMM. The inference alternates between sampling the state indicators z and the global state probabilities . • The state indicators z are sampled as • The global state probabilities are sampled as in HDP, and the equation not given. Wang 2010

  17. Texture Synthesis • The primary objective of this paper was to synthesize textures from smaller texture observations. • The authors note, however, that HDP-2DHMM is a generative model for image features and not actual image patches. To do this, they integrate image quilting, and define a dictionary of image patches (from the original image) from which the new image is synthesized (with much lower synthesis cost over standard image quilting). Wang 2010

  18. Synthesis Examples

  19. Synthesis Examples Wang 2010

  20. Synthesis Examples Wang 2010

  21. Image Segmentation and Synthesis Wang 2010

  22. Conclusion • A novel nonparametric Bayesian model called HD-2DHMM for texture learning and synthesis was presented in this paper. • The main contributions of this paper are that it learns the textons and spatial HMM structure jointly. • Inference is performed using a collapsed Gibbs Sampler • The clustering nature of the model allows for faster image quilting and synthesis with good qualitative results. • Image segmentation and synthesis results were also presented. Wang 2010

More Related