Understanding Image Reconstruction and Aliasing: Filters and Compositing Techniques
This lecture explores the critical concepts of image reconstruction and aliasing, focusing on various filtering techniques such as Bartlett and Gaussian filters. We will discuss the importance of sampling in both the spatial and frequency domains, and how insufficient sampling can lead to aliasing. Additionally, we’ll cover compositing methods, including the use of mattes and alpha channels in creating digital imagery. Gain insights into effective resampling strategies and the role of blurring edges to improve image quality. Ideal for students and professionals in computer graphics and image processing.
Understanding Image Reconstruction and Aliasing: Filters and Compositing Techniques
E N D
Presentation Transcript
Last Time • Filtering • Box filter • Bartlett filter • Gaussian Filter • Edge-detect (high-pass) filter • Enhancement filters • Resampling • Map the point from the new image back into the old image • Locally reconstruct the function using a filter • Today we’ll see why this is the right thing to do © University of Wisconsin, CS559 Spring 2004
Today • Ideal reconstruction and aliasing • Compositing © University of Wisconsin, CS559 Spring 2004
Ideal Reconstruction • When you display an image, ideally you would like to reconstruct the original (ideal) picture • When you resample, you would like to draw new samples from the perfect original function • Last time we saw you generally can’t do this because you need infinitely dense samples to reconstruct sharp edges • What’s the math? © University of Wisconsin, CS559 Spring 2004
Sampling in Spatial Domain • Sampling in the spatial domain is like multiplying by a spike function • You take some ideal function and get data for a regular grid of points © University of Wisconsin, CS559 Spring 2004
Sampling in Frequency Domain • Sampling in the frequency domain is like convolving with a spike function • Follows from the convolution theory: multiplication in spatial equals convolution in frequency • Spatial spike function in the frequency domain is also the spike function © University of Wisconsin, CS559 Spring 2004
Reconstruction (Frequency Domain) • To reconstruct, we must restore the original spectrum • That can be done by multiplying by a square pulse © University of Wisconsin, CS559 Spring 2004
Reconstruction (Spatial Domain) • Multiplying by a square pulse in the frequency domain is the same as convolving with a sinc function in the spatial domain © University of Wisconsin, CS559 Spring 2004
Aliasing Due to Under-sampling • If the sampling rate is too low, high frequencies get reconstructed as lower frequencies • High frequencies from one copy get added to low frequencies from another © University of Wisconsin, CS559 Spring 2004
More Aliasing • Poor reconstruction also results in aliasing • Consider a signal reconstructed with a box filter in the spatial domain (which means using a sinc in the frequency domain): © University of Wisconsin, CS559 Spring 2004
Aliasing in Practice • We have two types of aliasing: • Aliasing due to insufficient sampling frequency • Aliasing due to poor reconstruction • You have some control over reconstruction • If resizing, for instance, use an approximation to the sinc function to reconstruct (instead of Bartlett, as we used last time) • Gaussian is closer to sinc than Bartlett • But note that sinc function goes on forever (infinite support), which is inefficient to evaluate • You have some control over sampling if creating images using a computer • Remove all sharp edges (high frequencies) from the scene before drawing it • That is, blur character and line edges before drawing © University of Wisconsin, CS559 Spring 2004
Compositing • Compositing combines components from two or more images to make a new image • The basis for film special effects (even before computers) • Create digital imagery and composite it into live action • Important part of animation – even hand animation • Background change more slowly than foregrounds, so composite foreground elements onto constant background © University of Wisconsin, CS559 Spring 2004
Very Simple Example = over © University of Wisconsin, CS559 Spring 2004
Mattes • A matte is an image that shows which parts of another image are foreground objects • Term dates from film editing and cartoon production • How would I use a matte to insert an object into a background? • How are mattes usually generated for television? © University of Wisconsin, CS559 Spring 2004
Working with Mattes • To insert an object into a background • Call the image of the object the source • Put the background into the destination • For all the source pixels, if the matte is white, copy the pixel, otherwise leave it unchanged • To generate mattes: • Use smart selection tools in Photoshop or similar • They outline the object and convert the outline to a matte • Blue Screen: Photograph/film the object in front of a blue background, then consider all the blue pixels in the image to be the background © University of Wisconsin, CS559 Spring 2004
Alpha • Basic idea: Encode opacity information in the image • Add an extra channel, the alpha channel, to each image • For each pixel, store R, G, B and Alpha • alpha = 1 implies full opacity at a pixel • alpha = 0 implies completely clear pixels • There are many interpretations of alpha • Is there anything in the image at that point (web graphics) • Transparency (real-time OpenGL) • Images are now in RGBA format, and typically 32 bits per pixel (8 bits for alpha) • All images in the project are in this format © University of Wisconsin, CS559 Spring 2004
Pre-Multiplied Alpha • Instead of storing (R,G,B,), store (R,G,B,) • The compositing operations in the next several slides are easier with pre-multiplied alpha • To display and do color conversions, must extract RGB by dividing out • =0 is always black • Some loss of precision as gets small, but generally not a big problem © University of Wisconsin, CS559 Spring 2004
Compositing Assumptions • We will combine two images, f and g, to get a third composite image • Not necessary that one be foreground and background • Background can remain unspecified • Both images are the same size and use the same color representation • Multiple images can be combined in stages, operating on two at a time © University of Wisconsin, CS559 Spring 2004
Image Decomposition • The composite image can be broken into regions • Parts covered by f only • Parts covered by g only • Parts covered by f and g • Parts covered by neither f nor g • Compositing operations define what should happen in each region: who (f or g) owns each region © University of Wisconsin, CS559 Spring 2004
Basic Compositing Operation • At each pixel, combine the pixel data from f and the pixel data from g with the equation: • F and G describe how much of each input image survives, and cf and cg are pre-multiplied pixels, and all four channels are calculated • To define a compositing operation, define F and G © University of Wisconsin, CS559 Spring 2004
Sample Images Image Alpha © University of Wisconsin, CS559 Spring 2004
“Over” Operator © University of Wisconsin, CS559 Spring 2004
“Over” Operator • Computes composite with the rule that f covers g © University of Wisconsin, CS559 Spring 2004
“Inside” Operator © University of Wisconsin, CS559 Spring 2004
“Inside” Operator • Computes composite with the rule that only parts of f that are inside g contribute © University of Wisconsin, CS559 Spring 2004
“Outside” Operator © University of Wisconsin, CS559 Spring 2004
“Outside” Operator • Computes composite with the rule that only parts of f that are outside g contribute © University of Wisconsin, CS559 Spring 2004
“Atop” Operator © University of Wisconsin, CS559 Spring 2004
“Atop” Operator • Computes composite with the over rule but restricted to places where there is some g © University of Wisconsin, CS559 Spring 2004
“Xor” Operator • Computes composite with the rule that f contributes where there is no g, and g contributes where there is no f © University of Wisconsin, CS559 Spring 2004
“Xor” Operator © University of Wisconsin, CS559 Spring 2004
“Clear” Operator • Computes a clear composite • Note that (0,0,0,>0) is a partially opaque black pixel, whereas (0,0,0,0) is fully transparent, and hence has no color © University of Wisconsin, CS559 Spring 2004
“Set” Operator • Computes composite by setting it to equal f • Copies f into the composite © University of Wisconsin, CS559 Spring 2004
Compositing Operations • F and G describe how much of each input image survives, and cf and cg are pre-multiplied pixels, and all four channels are calculated © University of Wisconsin, CS559 Spring 2004
Unary Operators • Darken: Makes an image darker (or lighter) without affecting its opacity • Dissolve: Makes an image transparent without affecting its color © University of Wisconsin, CS559 Spring 2004
“PLUS” Operator • Computes composite by simply adding f and g, with no overlap rules • Useful for defining cross-dissolve in terms of compositing: © University of Wisconsin, CS559 Spring 2004
Obtaining Values • Hand generate (paint a grayscale image) • Automatically create by segmenting an image into foreground background: • Blue-screening is the analog method • Remarkably complex to get right • “Lasso” is the Photoshop operation • With synthetic imagery, use a special background color that does not occur in the foreground • Brightest blue or green is common © University of Wisconsin, CS559 Spring 2004
Compositing With Depth • Can store pixel “depth” instead of alpha • Then, compositing can truly take into account foreground and background • Generally only possible with synthetic imagery • Image Based Rendering is an area of graphics that, in part, tries to composite photographs taking into account depth © University of Wisconsin, CS559 Spring 2004
Where to now… • We are now almost done with images • Still have to take a quick look at painterly rendering • We will spend several weeks on the mechanics of 3D graphics • Coordinate systems and Viewing • Clipping • Drawing lines and polygons • Lighting and shading • We will finish the semester with modeling and some additional topics © University of Wisconsin, CS559 Spring 2004
