CS559: Computer Graphics

1. CS559: Computer Graphics Lecture 7: Image Warping and Morphing Li Zhang Spring 2010 Most slides borrowed from Yungyu Chuang

2. Last time: edge dection

3. Lat time: edge detection • Edge detection algorithms typically proceed in three or four steps: • Filtering: cut down on noise • Enhancement: amplify the difference between edges and non-edges • Detection: use a threshold operation • Localization (optional): estimate geometry of edges, which generally pass between pixels

4. The Canny Edge Detector original image (Lena)

5. The Canny Edge Detector magnitude of the gradient

6. The Canny Edge Detector After non-maximum suppression

7. Lat time: edge detection • Edge detection algorithms typically proceed in three or four steps: • Filtering: cut down on noise • Enhancement: amplify the difference between edges and non-edges • Detection: use a threshold operation • Localization (optional): estimate geometry of edges, which generally pass between pixels

8. Last time: Mattes • A matte is an image that shows which parts of another image are foreground objects • Term dates from film editing and cartoon production • How would I use a matte to insert an object into a background? • How are mattes usually generated for television?

9. Basic Compositing Operation • At each pixel, combine the pixel data from f and the pixel data from g with the equation: “Over” Operator

10. T Last time: 2D Transformations h([x,y])= [x,y/2] • Transformation T is a coordinate-changing machine: p’ = T(p) • What does it mean that T is global? • can be described by just a few numbers (parameters) • the parameters are the same for any point p • Represent T as a matrix: p’ = Mp p = (x,y) p’ = (x’,y’)

11. x  2,y  0.5 Scaling • Non-uniform scaling: different scalars per component:

12. (x’, y’) (x, y)  2-D Rotation x’ = x cos() - y sin() y’ = x sin() + y cos()

13. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? y’ y shx 1 (1,1) 1 shy 0 1 0 x’ 1 x 2D Shear?

14. 2x2 Matrices • What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis? 2D Mirror over (0,0)?

15. All 2D Linear Transformations • Linear transformations are combinations of … • Scale, • Rotation, • Shear, and • Mirror • Any 2D transform can be decomposed into the product of a rotation, scale, and a rotation

16. All 2D Linear Transformations • Linear transformations are combinations of … • Scale, • Rotation, • Shear, and • Mirror • A symmetric 2D transform can be decomposed into the product of a rotation, scale, and the inverse rotation

17. Today • More on 2D transformation • Use it for image warping and morphing

18. All 2D Linear Transformations • Linear transformations are combinations of … • Scale, • Rotation, • Shear, and • Mirror • Properties of linear transformations: • Origin maps to origin • Lines map to lines • Parallel lines remain parallel • Ratios are preserved • Closed under composition

19. 2D Translation? 2x2 Matrices • What types of transformations can not be represented with a 2x2 matrix? NO! Only linear 2D transformations can be represented with a 2x2 matrix

20. Translation • Example of translation Homogeneous Coordinates tx = 2ty= 1

21. Homogeneous coordinates • Why do we need it? • Can express all linear transformation as special cases

22. Homogeneous coordinates • Why do we need it? • Can express all linear transformation as special cases • Easy to compute a composite transformation that involve several translations and linear transformation

23. Homogeneous coordinates • Why do we need it? • Can express all linear transformation as special cases • Easy to compute a composite transformation that involve several translations and linear transformation • More to come

24. Affine Transformations • Affine transformations are combinations of … • Linear transformations, and • Translations • Properties of affine transformations: • Origin does not necessarily map to origin • Lines map to lines • Parallel lines remain parallel • Ratios are preserved • Closed under composition • Models change of basis

25. Image warping • Given a coordinate transform x’= T(x) and a source image I(x), how do we compute a transformed image I’(x’)=I(T(x))? T(x) x x’ I(x) I’(x’)

26. T(x) Forward warping • Send each pixel I(x) to its corresponding location x’=T(x) in I’(x’) x x’ I(x) I’(x’)

27. Forward warping fwarp(I, I’, T) { for (y=0; y<I.height; y++) for (x=0; x<I.width; x++) { (x’,y’)=T(x,y); I’(x’,y’)=I(x,y); } } I I’ T x x’

28. Forward warping • Send each pixel I(x) to its corresponding location x’=T(x) in I’(x’) • What if pixel lands “between” two pixels? • Will be there holes? • Answer: add “contribution” to several pixels, normalize later (splatting) h(x) x x’ f(x) g(x’)

29. Forward warping fwarp(I, I’, T) { for (y=0; y<I.height; y++) for (x=0; x<I.width; x++) { (x’,y’)=T(x,y); Splatting(I’,x’,y’,I(x,y),kernel); } } I I’ T x x’

30. Splatting • Computed weighted sum of contributed colors using a kernel function, where weights are normalized values of filter kernel k, such as Gauss May get a blurry image! for all q q.color = 0; q.weight = 0; for all p from source image for all q’s dist < radius d = dist(p, q); w = kernel(d); q.color += w*p; q.weight += w; for all q q.Color /= q.weight; p radius d q Destination Image

31. T-1(x’) Inverse warping • Get each pixel I’(x’) from its corresponding location x=T-1(x’) in I(x) x x’ I(x) I’(x’)

32. Inverse warping iwarp(I, I’, T) { for (y=0; y<I’.height; y++) for (x=0; x<I’.width; x++) { (x,y)=T-1(x’,y’); I’(x’,y’)=I(x,y); } } T-1 I I’ x x’

33. Inverse warping • Get each pixel I’(x’) from its corresponding location x=T-1(x’) in I(x) • What if pixel comes from “between” two pixels? • Answer: resample color value from interpolated source image x x’ f(x) g(x’)

34. Inverse warping iwarp(I, I’, T) { for (y=0; y<I’.height; y++) for (x=0; x<I’.width; x++) { (x,y)=T-1(x’,y’); I’(x’,y’)=Reconstruct(I,x,y,kernel); } } T-1 I I’ x x’

35. Reconstruction (interpolation) • Possible reconstruction filters (kernels): • nearest neighbor • bilinear • bicubic • sinc

36. Bilinear interpolation (tent filter) • A simple method for resampling images What might be the problem of bilinear interpolation?

37. Non-parametric image warping

38. Non-parametric image warping • Specify a more detailed warp function [x,y] T [x’,y’]

39. Non-parametric image warping • Specify a more detailed warp function • Tabulate pixel motion (lookup table)

40. P’ Non-parametric image warping • Mappings implied by correspondences • Inverse warping ?

41. Non-parametric image warping

42. Warping between two triangles A’ • Idea: find an affine that transforms ABC to A’B’C’ A C’ C B B’

43. Warping between two triangles A’ • Idea: find an affine that transforms ABC to A’B’C’ • 6 unknowns, 6 equations A C’ C B B’

44. Warping between two triangles A’ • Idea: find an affine that transforms ABC to A’B’C’ • 6 unknowns, 6 equations • A more direct way A C’ C B B’

45. Barycentric coordinates • Idea: represent P using A1,A2,A3

46. Barycentric coordinates • Idea: represent P using A1,A2,A3

47. Barycentric coordinates • Idea: represent P using A1,A2,A3

48. Barycentric coordinate P’ P Non-parametric image warping Turns out to be equivalent to affine transform