Fast Texture Synthesis using Tree-structured Vector Quantization

1. Fast Texture Synthesis using Tree-structured Vector Quantization Li-Yi Wei Marc Levoy Computer Graphics Group Stanford University

2. Introduction Texture Synthesis Input Result

3. Desirable Properties • Result looks like the input • Efficient • General • Easy to use • Extensible

4. Previous Work • Procedural Synthesis • Perlin 85, Witkin 91, Worley 96 • Statistical Feature Matching • Heeger 95, De Bonet 97, Simoncelli 98 • Markov Random Fields • Popat 93, Efros 99

5. Outline • Basic algorithm • Multi-resolution algorithm • Acceleration • Applications

6. Texture Model • Textures are • local • stationary • Model textures by • local spatial neighborhoods

7. Basic Algorithm • Exhaustively search neighborhoods

8. Noise Neighborhood • Use causal neighborhoods Input Causal Non-causal

9. Neighborhood • Neighborhood size determines the quality & cost 33 55 77 739 s 423 s 528 s 1111 4141 99 24350 s 1020 s 1445 s

10. High resolution Low resolution Multi-resolution Pyramid

11. Multi-resolution Algorithm

12. Benefit • Better image quality & faster computation 1 level 55 1 level 1111 3 levels 55

13. Results Oriented Random Regular Semi-regular

14. Failures • Non-planar structures • Global information

15. Comparison Input 12 secs Heeger 95 De Bonet 97 Efros 99 Our method 503 secs 1941 secs

16. Acceleration • Computation bottleneck: neighborhood search

17. 1 2 3 4 5 6 7 8 9 10 11 12 High dimensional point/vector 1 2 3 4 5 6 7 8 9 10 11 12 Nearest Point Search • Treat neighborhoods as high dimensional points Neighborhood

18. Acceleration • Nearest point search in high dimensions • [Nene 97] • Cluster-based model for textures • [Popat 93] • Tree-structured Vector Quantization • [Gersho 92]

19. Tree-structured Vector Quantization

20. Timing • Time complexity : O(log N) instead of O(N) • 2 orders of magnitude speedup for non-trivial images Efros 99 Full searching TSVQ 1941 secs 503 secs 12 secs

21. Results: Brodatz Textures D103 D20 Input Exhaustive: 360 secs TSVQ: 7.5 secs

22. Application 1: Constrained Synthesis ?

23. Possible Solution • Multi-resolution blending [Burt & Adelson 83] • produce visible boundaries

24. Possible Solution • Original raster-scan algorithm • discontinuities at right and bottom boundaries

25. Possible Solution • Adaptive neighborhoods [Efros 99] • Hard to accelerate

26. Modifications • Need to use a single symmetric neighborhood • 2 pass algorithm with extrapolation • Spiral order synthesis

27. Result

28. Result • Extrapolation ? ? ? ?

29. Result • Image editing by texture replacement

30. Application 2:Temporal Texture • Indeterminate motions both in space and time • fire, smoke, ocean waves • How to synthesize? • extend our 2D algorithm to 3D

31. Temporal Texture Input Result Fire Smoke Waves

32. Future Work • More general “textures” • light fields, solid textures • motion signals • displacement maps • Real time texture synthesis

33. Kris Popat Alyosha Efros Stanford Graphics Group Intel, Interval, Sony Acknowledgment More information http://graphics.stanford.edu/projects/texture/