Texture Mapping

1. Texture Mapping Aaron Bloomfield CS 445: Introduction to Graphics Fall 2006 (Slide set originally by David Luebke)

2. Overview • Motivation and Examples • Fundamentals • Algorithms • Perspective-Correct Texturing • Transparency and Anti-Aliasing • MIP-Maps • Bump and Displacement Maps • Illumination Maps

3. Texture Mapping: Motivation • Scenes created with diffuse lighting look convincingly three-dimensional, but are flat, chalky, and “cartoonish” • Phong lighting lets us simulate materials like plastic and (to a lesser extent) metal, but scenes still seem very cartoonish and unreal • Big problem: polygons are too coarse-grained to usefully model fine surface detail • Solution: texture mapping

4. Texture Mapping: Motivation • Adding surface detail helps keep CG images from looking simple and sterile • Explicitly modeling this detail in geometry can be very expensive • Zebra stripes, wood grain, writing on a whiteboard • Texture mapping pastes images onto the surfaces in the scene, adding realistic fine detail without exploding the geometry

6. Texture Mapping: Examples

7. Texture Mapping: Examples Doom III (ID Software)

8. Texture Mapping • In short: it is impractical to explicitly model fine surface detail with geometry • Solution: use images to capture the “texture” of surfaces • Texture maps can modulate many factors that affect the rendering of a surface • Color or reflectance (diffuse, ambient, specular) • Transparency (smoke effects) • What else?

9. Overview • Motivation and Examples • Fundamentals • Algorithms • Perspective-Correct Texturing • Transparency and Anti-Aliasing • MIP-Maps • Bump and Displacement Maps • Illumination Maps

10. Texture Mapping: Fundamentals • A texture is typically a 2-D image • Can also be • A 2-D procedural texture • A 3-D procedural texture • See video • A 3-D volumetric texture (rarely used)

11. Texture Mapping: Fundamentals • A texture is typically a 2-D image • Image elements are called texels • Value stored at a texel affects surface appearance in some way • Example: diffuse reflectance, shininess, transparency… • The mapping of the texture to the surface determines the correspondence, i.e., how the texture lies on the surface • Mapping a texture to a triangle is easy (why?) • Mapping a texture to an arbitrary 3-D shape is more complicated (why?)

12. Texturing Fundamentals • A texture is typically a 2-D array of texels • Mapping the texture to an arbitrary 3-D shape is complex:

13. Texturing Fundamentals • A texture is typically a 2-D array of texels • Mapping the texture to an arbitrary 3-D shape is complex:

14. Texturing Fundamentals • A texture is typically a 2-D array of texels • Mapping the texture to an arbitrary 3-D shape is complex:

15. Texturing Fundamentals • A texture is typically a 2-D array of texels • Mapping the texture to an arbitrary 3-D shape is complex! http://www.cs.virginia.edu/~gfx/Courses/2003/Intro.spring.03/animations/unfold.mov

16. How to map a texture • There are a number of possibilities: • Flat • Cube • Tube (i.e. cylinder) • Sphere • The slides on the next slide are from http://mediawiki.blender.org/index.php/Manual/Map_Input#2D_to_3D_Mapping

18. Overview • Motivation and Examples • Fundamentals • Algorithms • Perspective-Correct Texturing • Transparency and Anti-Aliasing • MIP-Maps • Bump and Displacement Maps • Illumination Maps

19. Texture Mapping: Rendering • Rendering uses the mapping: • Find the visible surface at a pixel • Find the point on that surface corresponding to that pixel • Find the point in the texture corresponding to that point on the surface • Use the parameters associated with that point on the texture to shade the pixel • Using triangulated meshes reduces the problem to mapping a portion of the image to each triangle:

20. Texture Mapping: Rendering

21. Texture Mapping:User-Generated Mappings • For complex 3-D objects, mapping textures is still something of an art…so we often let the user do it

22. Texture Mapping: Rendering • We typically parameterize the texture as a function in (u, v) • For simplicity, normalize u & v to [0, 1] • Associate each triangle with a texture • Give each vertex of the triangle a texture coordinate (u, v) • For other points on the triangle, interpolate texture coordinate from the vertices • Much like interpolating color or depth • But there’s a catch...

23. Naïve Texture Mapping • A first cut at a texture-mapping rasterizer: • For each pixel: • Interpolate u & v down edges and across spans • Look up nearest texel in texture map • Color pixel according to texel color (possibly modulated by lighting calculations) • McMillan’s demo of this is at http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide05.html • What artifacts do you see in this demo?

24. Naïve Texturing Artifacts • Another serious artifact is warping at the edges of triangles making up the mesh • A more obvious example:http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide06.html • To address this, need to consider the geometry of interpolating parameters more carefully

25. Interpolating Parameters • The problem turns out to be fundamental to interpolating parameters in screen-space • Uniform steps in screen space  uniform steps in world coords

26. Interpolating Parameters • Perspective foreshortening is not getting applied to our interpolated parameters • Parameters should be compressed with distance • Linearly interpolating them in screen-space doesn’t do this • Is this a problem with Gouraud shading? • Is everything I taught wrong? • A: It can be, but we usually don’t notice (why?)

27. Overview • Motivation and Examples • Fundamentals • Algorithms • Perspective-Correct Texturing • Transparency and Anti-Aliasing • MIP-Maps • Bump and Displacement Maps • Illumination Maps

28. Perspective-Correct Interpolation • Skipping a bit of math to make a long story short… • Rather than interpolating u and v directly, interpolate u/z and v/z • These do interpolate correctly in screen space • Also need to interpolate z and multiply per-pixel • Problem: we don’t know z anymore • Solution: we do know w  1/z • So…interpolate uw and vw and w, and compute u = uw/w and v = vw/w for each pixel • This unfortunately involves a divide per pixel (Just 1?)

29. Perspective-Correct Texturing • Known as perspective-correct texture mapping • Early PC cards and game consoles didn’t support it • So how did they avoid the warping problem? • http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide15.html • As mentioned, other interpolation schemes really ought to use perspective correction • E.g., Gouraud shading • Generally get away without it because it is more important to be smooth than correct • Java code fragment from McMillan’s edge-equation triangle rasterizer:

30. Perspective-Correct Texturing: Code ... PlaneEqn(uPlane, (u0*w0), (u1*w1), (u2*w2)); PlaneEqn(vPlane, (v0*w0), (v1*w1), (v2*w2)); PlaneEqn(wPlane, w0, w1, w2); ... for (y = yMin; y <= yMax; y += raster.width) { e0 = t0; e1 = t1; e2 = t2; u = tu; v = tv; w = tw; z = tz; boolean beenInside = false; for (x = xMin; x <= xMax; x++) { if ((e0 >= 0) && (e1 >= 0) && (e2 >= 0))) { int iz = (int) z; if (iz <= raster.zbuff[y+x]) { float denom = 1.0f / w; int uval = (int) (u * denom + 0.5f); uval = tile(uval, texture.width); int vval = (int) (v * denom + 0.5f); vval = tile(vval, texture.height); int pix = texture.getPixel(uval, vval); if ((pix & 0xff000000) != 0) { raster.pixel[y+x] = pix; raster.zbuff[y+x] = iz; } } beenInside = true; } else if (beenInside) break; e0 += A0; e1 += A1; e2 += A2; z += Az; u += Au; v += Av; w += Aw; } t0 += B0; t1 += B1; t2 += B2; tz += Bz; tu += Bu; tv += Bv; tw += Bw; }

31. Texture Tiling • It is often handy to tile a repeating texture pattern onto a surface • The previous code does this via tile(): int uval = (int) (u * denom + 0.5f); uval = tile(uval, texture.width); int vval = (int) (v * denom + 0.5f); vval = tile(vval, texture.height); int pix = texture.getPixel(uval, vval); int tile(int val, int size) { while (val >= size) val -= size; while (val < 0) val += size; } See http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide18.html

32. Overview • Motivation and Examples • Fundamentals • Algorithms • Perspective-Correct Texturing • Transparency and Anti-Aliasing • MIP-Maps • Bump and Displacement Maps • Illumination Maps

33. Texture Transparency • McMillan’s code also includes a “quick fix” for handling transparent texture: if ((pix & 0xff000000) != 0) { raster.pixel[y+x] = pix; raster.zbuff[y+x] = iz; } • Note that this doesn’t handle partial transparency (How might such partial transparency arise?) • Demo at: http://graphics.lcs.mit.edu/classes/6.837/F98/Lecture21/Slide19.html

35. Texture Map Aliasing • Naive texture mapping looks blocky, pixelated • Problem: using a single texel to color each pixel: int uval = (int) (u * denom + 0.5f); int vval = (int) (v * denom + 0.5f); int pix = texture.getPixel(uval, vval); • Actually, each pixel maps to a region in texture • If the pixel is larger than a texel, we should average the contribution from multiple texels somehow • If the pixel is smaller than a texel, we should interpolate between texel values somehow • Even if pixel size  texel size, a pixel will in general fall between four texels • An example of a general problem called aliasing

36. Texture Map Antialiasing • Use bilinear interpolation to average nearby texel values into a single pixel value (Draw it) • Find 4 nearest texture samples • Round u & v up and down • Interpolate texel values in u • Interpolate resulting values in v • Also addresses the problem of many pixels projecting to a single texel (Why?)

37. Texture Map Antialiasing • What if a single pixel covers many texels? • Problem: sampling those texels at a single point (the center of the pixel): • Produces Moire patterns in coherent texture (checkers) • Leads to flicker or texture crawling as the texture moves

38. Moire Patterns

39. Texture Map Antialiasing • What if a single pixel covers many texels? • Approach: blur the image under the pixel, averaging the contributions of the covered texels • But calculating which texels every pixel covers is way too expensive, especially as the texture is compressed • Solution: pre-calculate lower-resolution versions of the texture that incorporate this averaging

40. Overview • Motivation and Examples • Fundamentals • Algorithms • Perspective-Correct Texturing • Transparency and Anti-Aliasing • MIP-Maps • Bump and Displacement Maps • Illumination Maps

41. Original Texture Lower Resolution Versions MIP-maps • For a texture of 2n x 2n pixels, compute n-1 textures, each at ½ the resolution of previous: • This multiresolution texture is called a MIP-map

42. Generating MIP-maps • Generating a MIP-map from a texture is easy • For each texel in level i, average the values of the four corresponding texels in level i-1 • If a texture requires n bytes of storage, how much storage will a MIP-map require? • Answer: 4n/3

43. R G R G B R G B R G B B Representing MIP-maps Trivia: MIP = Multum In Parvo(many things in a small place)

44. Using MIP-maps • Each level of the MIP-map represents a pre-blurred version of multiple texels • A texel at level n represents 2n original texels • When rendering: • Figure out the texture coverage of the pixel (i.e., the size of the pixel in texels of the original map) • Find the level of the MIP map in which texels average approximately that many original texels • Interpolate the value of the four nearest texels

45. Using MIP-maps • Even better: • Likely, the coverage of the pixel will fall somewhere between the coverage of texels in two adjacent levels of the MIP map • Find the pixel’s value in each of the two textures using two bilinear interpolations • Using a third interpolation, find a value in between these two values, based on the coverage of the pixel versus each of the MIP-map levels • This is (misleadingly?) called trilinear interpolation

46. MIP-map Example • No filtering: • MIP-map texturing:

47. Can We Do Better? • What assumption does MIP-mapping implicitly make? • A: The pixel covers a square region of the texture • More exactly, the compression or oversampling rate is the same in uandv • Is this a valid assumption? Why or why not?

48. MIP-maps and Signal Processing • An aside: aliasing and antialiasing are properly topics in sampling theory • Nyquist theorem, convolution and reconstruction, filters and filter widths • Textures are particularly difficult because a tiled texture can easily generate infinite frequencies • E.g., a checkered plane receding to an infinite horizon • Using a MIP-map amounts to prefiltering the texture image to reduce artifacts caused by sampling at too low a rate

49. Summed-Area Tables • A technique called summed-area tableslets us integrate texels covered by the pixel more exactly (but still quickly) • Details in the book • Example: MIP-map texturing Summed-area table texturing

50. Overview • Motivation and Examples • Fundamentals • Algorithms • Perspective-Correct Texturing • Transparency and Anti-Aliasing • MIP-Maps • Bump and Displacement Maps • Illumination Maps