1 / 54

Pyramids and Texture

Pyramids and Texture. Big bars and little bars are both interesting Spots and hands vs. stripes and hairs Inefficient to detect big bars with big filters And there is superfluous detail in the filter kernel Alternative: Apply filters of fixed size to images of different sizes

ericas
Télécharger la présentation

Pyramids and Texture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Pyramids and Texture

  2. Big bars and little bars are both interesting Spots and hands vs. stripes and hairs Inefficient to detect big bars with big filters And there is superfluous detail in the filter kernel Alternative: Apply filters of fixed size to images of different sizes Typically, a collection of images whose edge length changes by a factor of 2 (or root 2) This is a pyramid (or Gaussian pyramid) by visual analogy Scaled representations

  3. A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose

  4. Aliasing • Can’t shrink an image by taking every second pixel • If we do, characteristic errors appear • In the next few slides • Typically, small phenomena look bigger; fast phenomena can look slower • Common phenomenon • Wagon wheels rolling the wrong way in movies • Checkerboards misrepresented in ray tracing • Striped shirts look funny on color television

  5. Constructing a pyramid by taking every second pixel leads to layers that badly misrepresent the top layer

  6. Open questions • What causes the tendency of differentiation to emphasize noise? • In what precise respects are discrete images different from continuous images? • How do we avoid aliasing? • General thread: a language for fast changes The Fourier Transform

  7. Represent function on a new basis Think of functions as vectors, with many components We now apply a linear transformation to transform the basis dot product with each basis element In the expression, u and v select the basis element, so a function of x and y becomes a function of u and v basis elements have the form The Fourier Transform transformed image vectorized image Fourier transform base, also possible Wavelets, steerable pyramids, etc.

  8. Fourier basis element • example, real part • Fu,v(x,y) • Fu,v(x,y)=const. for (ux+vy)=const. • Vector (u,v) • Magnitude gives frequency • Direction gives orientation.

  9. Here u and v are larger than in the previous slide.

  10. And larger still...

  11. Fourier transform of a real function is complex difficult to plot, visualize instead, we can think of the phase and magnitude of the transform Phase is the phase of the complex transform Magnitude is the magnitude of the complex transform Curious fact all natural images have about the same magnitude transform hence, phase seems to matter, but magnitude largely doesn’t Demonstration Take two pictures, swap the phase transforms, compute the inverse - what does the result look like? Phase and Magnitude

  12. This is the magnitude transform of the cheetah pic

  13. This is the phase transform of the cheetah pic

  14. This is the magnitude transform of the zebra pic

  15. This is the phase transform of the zebra pic

  16. Reconstruction with zebra phase, cheetah magnitude

  17. Reconstruction with cheetah phase, zebra magnitude

  18. The message of the FT is that high frequencies lead to trouble with sampling. Solution: suppress high frequencies before sampling multiply the FT of the signal with something that suppresses high frequencies or convolve with a low-pass filter A filter whose FT is a box is bad, because the filter kernel has infinite support Common solution: use a Gaussian multiplying FT by Gaussian is equivalent to convolving image with Gaussian. Smoothing as low-pass filtering

  19. Sampling without smoothing. Top row shows the images, sampled at every second pixel to get the next; bottom row shows the magnitude spectrum of these images.

  20. Sampling with smoothing. Top row shows the images. We get the next image by smoothing the image with a Gaussian with sigma 1 pixel, then sampling at every second pixel to get the next; bottom row shows the magnitude spectrum of these images.

  21. Sampling with smoothing. Top row shows the images. We get the next image by smoothing the image with a Gaussian with sigma 1.4 pixels, then sampling at every second pixel to get the next; bottom row shows the magnitude spectrum of these images.

  22. Applications of scaled representations • Search for correspondence • look at coarse scales, then refine with finer scales • Edge tracking • a “good” edge at a fine scale has parents at a coarser scale • Control of detail and computational cost in matching • e.g. finding stripes • terribly important in texture representation

  23. Example: CMU face detection

  24. The Gaussian pyramid • Smooth with gaussians, because • a gaussian*gaussian=another gaussian • Synthesis • smooth and sample • Analysis • take the top image • Gaussians are low pass filters, so representation is redundant

  25. http://web.mit.edu/persci/people/adelson/pub_pdfs/pyramid83.pdfhttp://web.mit.edu/persci/people/adelson/pub_pdfs/pyramid83.pdf

  26. Texture • Key issue: representing texture • Texture based matching • little is known • Texture segmentation • key issue: representing texture • Texture synthesis • useful; also gives some insight into quality of representation • Shape from texture • will skip discussion

  27. Texture synthesis Given example, generate texture sample (that is large enough, satisfies constraints, …)

  28. Texture analysis Compare; is this the same “stuff”?

  29. pre-attentive texture discrimination

  30. pre-attentive texture discrimination

  31. pre-attentive texture discrimination • same or not?

  32. pre-attentive texture discrimination

  33. pre-attentive texture discrimination • same or not?

  34. Textures are made up of quite stylized subelements, repeated in meaningful ways Representation: find the subelements, and represent their statistics But what are the subelements, and how do we find them? recall normalized correlation find subelements by applying filters, looking at the magnitude of the response What filters? experience suggests spots and oriented bars at a variety of different scales details probably don’t matter What statistics? within reason, the more the merrier. At least, mean and standard deviation better, various conditional histograms. Representing textures

  35. Spots and bars at a fine scale

  36. Spots and bars at a coarser scale

  37. Fine scale How many filters and what orientations? Coarse scale

  38. Texture Similarity based on Response Statistics • Collect statistics of responses over an image or subimage • Mean of squared response • Mean and variance of squared response • Euclidean distance between vectors of response statistics for two images is measure of texture similarity

  39. Example 1: Squared response

  40. Example 2: Mean and variance of squared response • Compute the mean and standard deviation of the filter outputs over the window, and use these for the feature vector. (Ma and Manjunath, 1996) Decreasing response vector similarity

  41. The Choice of Scale • One approach: start with a small window and increase the size of the window until an increase does not cause a significant change.

  42. Laplacian Pyramids as Band-Pass Filters courtesy of Wolfram from Forsyth & Ponce Each level is the difference of a more smoothed and less smoothed image ! It contains the band of frequencies in between

  43. v Oriented Pyramids • Laplacian pyramid + direction sensitivity from Forsyth & Ponce

More Related