1 / 49

CS 395/495-25: Spring 2003

CS 395/495-25: Spring 2003. IBMR: Week 7B Chapter 6.1, 6.2; Chapter 7: More Single-Camera Details Jack Tumblin jet@cs.northwestern.edu. IBMR-Related Seminars. 3D Scanning for Cultural Heritage Applications Holly Rushmeier, IBM TJ Watson Friday May 16 3:00pm, Rm 381, CS Dept.

idola
Télécharger la présentation

CS 395/495-25: Spring 2003

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 395/495-25: Spring 2003 IBMR: Week 7B Chapter 6.1, 6.2; Chapter 7: More Single-Camera Details Jack Tumblin jet@cs.northwestern.edu

  2. IBMR-Related Seminars 3D Scanning for Cultural Heritage Applications Holly Rushmeier, IBM TJ Watson Friday May 16 3:00pm, Rm 381, CS Dept. Light Scattering Models for Rendering Human Hair Steve Marschner, Cornell University Friday May 23 3:00pm, Rm 381, CS Dept.

  3. Reminders • ProjA graded: Good Job! 90,95, 110 • ProjB graded: Good! minor H confusions... • MidTerm graded: • ProjC posted, due Friday, May 16 • ProjD tomorrow, Friday May 16, due Friday May 30 • Start Watson’s Late Policy?: Grade -(3n) points; n = # of class meetings late • Take-Home Final Exam: Thurs June 5, due June 11

  4. Mirror Spheres: Why? Traditional CG Rendering: To make an image, Compute radiance arriving at novel camera position: • Specify: Incoming light: Irradiance function: at each (x,y,z) point from every direction (,) • Specify: Shape, Texture, Reflectance, BRDF, BRSSDF at each surface point (xs,ys,zs) • Compute: Outgoing light: exitance function: (after incoming light bounces around the scene) at any camera point (x,y,z) from any pixel direction (,)

  5. Mirror Spheres: Why? • IBMR: input is far less defined! ‘images, (usually) no depth’ only To make an image, Compute radiance arriving at novel camera position: • Specify: Radiance from images, and perhaps • Specify: Radiance from images • Compute: Outgoing light: exitance function: (after incoming light bounces around the scene) at any camera point (x,y,z) from any pixel direction (,)

  6. ‘Rendering’ from a camera image? Conventional: external camera reads light field (after rendering) Camera Shape, Position, Movement, Emitted Light zc xc yc Reflected, Scattered, Light … BRDF, Texture, Scattering

  7. ‘Rendering’ from a camera image? IBMR: Let camera measure light inside scene Shape, Position, Movement, Emitted Light Camera x2 x1 Reflected, Scattered, Light … BRDF, Texture, Scattering x3

  8. ‘Rendering’ from a camera image? IBMR: Camera measures light inside scene TROUBLE!Camera is an object; reflects light, changes scene. Shape, Position, Movement, Emitted Light Camera x2 x1 Reflected, Scattered, Light … BRDF, Texture, Scattering x3 WANTED: tiny, point-like panoramic camera:a ‘light probe’

  9. One Answer: Light Probe Photograph a small mirror sphere Shape, Position, Movement, Emitted Light Camera xc yc Mirror Sphere zc Reflected, Scattered, Light … BRDF, Texture, Scattering

  10. Light Probes: How? • Tele-photo a mirror sphere (narrow FOV) • warp image to find irradiance .vs. direction High contrast? Higher resol.? More positions? More Pictures! Paul Debevec, SIGGRAPH2001 course “Image Based Lighting”

  11. High Contrasts too! Paul Debevec, SIGGRAPH2001 short course “Image Based Lighting” .

  12. One Answer: Light Probe Light probes can measure irradiance— the incoming light at a point. • Can use them as panoramic cameras, --OR— as local ‘light maps’; • they define intensity of incoming light .vs. direction, as if local neighborhood was lit by lights at infinity. • Caution! may not be valid at nearby locations! • Caution! high dynamic range! (>> 1:255) • Can use them to render synthetic objects...

  13. A Mirror Sphere is... A ‘light probe’ to measure ALL incoming light at a point.How can we use it?

  14. Image-Based Actual Re-lighting

  15. Image-Based Actual Re-lighting Light the actress in Los Angeles Debevec et al., SIGG2001 Film the background in Milan, Measure incoming light, Matched LA and Milan lighting. Matte the background

  16. Measure REAL light in a REAL scene... Debevec et al., SIGG1998

  17. Render FAKE objects with REAL light, And combine with REAL image: Debevec et al., SIGG1998

  18. The Grand Challenges: • Controlled Lights +Controlled Camerassuggest we CAN recover arbitrary BRDF/ BSSSDFand ‘enough’ shape. • Is any method PRACTICAL? • Can we avoid/reduce corrupting interreflections? • Can we understand the shape/texture tradeoff?

  19. The Grand Rewards: • Controlled Lights +Controlled Camerassuggest we CANrecover arbitrary BRDF/ BSSSDFand ‘enough’ shape. • Holodeck? CAVE uncorrupted by interreflections • Historical Preservation? complete optical records • ‘Fake Materials’? a BRDF/BSSRDF display ... • ‘Shader Lamps’? exchange reflectance for illumination • IBMR invisibility?

  20. Image-Based Synthetic Re-lighting Masselus et al., 2002

  21. Image-Based Shape Refinement Fine geometric Details  Fine Texture/Normal Details Rushmeier, 2001

  22. Image-Based Shape Approximation Matusik 2002 Matusik 2002

  23. Image-Based Shape Approximation Matusik 2002

  24. Why all this Projective Tedium? • So you have the tools to try IBMR (and because I’m struggling, slowly, to it boil down to the essentials in this course) • It’s almost over— last 3 weeks of class will be reading good recent research papers, and will • Begin exploring some open research questions...

  25. Camera Matrix P Summary: xfspx0 0 yfpy00 0 1 0 y Input:X 3D World Space Output: x 2D Camera Image x X (world space) z yc C zc xc x(camera space) y x • Basic camera: x = P0 X whereP0 = [K | 0] = • World-space camera: translate world origin to camera location C, then rotate: x = PX = (P0·R·T) X • Rewrite as: P = K [R | -RC] • Redundant notation:P = [M | p4]M = RKp4 = -K R C ~ ~ ~

  26. Chapter 6 In Just One Slide: Given point correspondence sets (xi Xi), How do you find camera matrix P ? (full 11 DOF) Surprise! You already know how ! • DLT method: -rewrite H x = x’ as Hx  x’ = 0 -rewrite P X = x as PX  x = 0 -vectorize, stack, solve Ah = 0 for h vector -vectorize, stack, solve Ap = 0 for p vector -Normalizing step removes origin dependence • More data  better results (at least 28 point pairs)(why so many? rule-of-thumb: #constraints = 5x #DOF = 55 = 27.5 point pairs) • Algebraic & Geometric Error, Sampson Error…

  27. Chapter 7: More One-Camera Details Full 3x4 camera matrix P maps P3world to P2 image ? What does it do to basic 3D world shapes? • Planes • Given any point X on a plane in P3, • Change world’s coord. system: let plane be z=0: • Matrix P reduces to 3x3 matrix H in P2: x = P·X = • THUSP2can do any, all P2 plane transforms xy0t xyt h11 h12 h13 h21 h22 h21 h31 h32 h33 p11 p12 p13 p14 p21 p12 p23 p24 p31 p32 p33 p34 =

  28. Chapter 7: More One-Camera Details y x z Full 3x4 camera matrix P maps P3world to P2 image ? What does it do to basic 3D world shapes? • Points, Directions: World-space P3 direction D image space point xd: Recall directionD = (x,y,z,0)(a point at infinity) sets a R3 finite point d = (x,y,z). xd = PD = [M | p4] D = Md p4column has no effect, because of D’s zero; Recall M = KR xd = Md M-1xd = d D xd yc d ~ p C f zc X (world space) xc

  29. Chapter 7: More One-Camera Details Full 3x4 camera matrix P maps P3world to P2 image ? What does it do to basic 3D world shapes? • Lines: Forward Projection: • Line / Ray in world Line/Ray in image: • Ray in P3 is: X() = A + B • Camera changes to P2: x() = PA + PB yc A PA B p C f zc xc

  30. Chapter 7: More One-Camera Details Full 3x4 camera matrix P maps P3world to P2 image ? What does it do to basic 3D world shapes? • Lines: Back Projection: Line L in image Plane L in world: • Recall: Line L in P2 (a 3-vector): L =[x1 x2 x3]T • Plane L in P3 (a 4-vector): L = PT·L = • (SKIP Plucker Matrix lines…) L p11 p21 p31 p12 p22 p32p13 p23 p33p14 p24 p34 123 yc L zc p C f .. xc

  31. Chapter 7: More One-Camera Details Full 3x4 camera matrix P maps P3world to P2 image ? What does it do to basic 3D world shapes? Conics 1: • Conic C in image Cone Quadric Qco in world Qco = PT·C·P • (Tip of cone is camera centerV) C yc p V f zc xc

  32. Chapter 7: More One-Camera Details Full 3x4 camera matrix P maps P3world to P2 image ? What does it do to basic 3D world shapes? • Conics 2: Dual (line) Quadric Q* in world  Dual (line) Conic C* silhouette in image C* = PT·Q*·P • Works for ANY world quadric!sphere, cylinder, ellipsoid,paraboloid, hyperboloid, line, disk … C* Q* yc p V zc f xc

  33. Chapter 7: More One-Camera Details Full 3x4 camera matrix P maps P3world to P2 image ? What does it do to basic 3D world shapes? • Conics 3: World-space quadric Q  World-space view cone Qco, a degenerate quadric Qco = (VT QV)Q – (QV)(QV)T Qco yc Q p V zc f xc

  34. Chapter 7: More One-Camera Details xfspx 0 yfpy0 0 1 xfs(px-tx) 0 yf(py-ty)0 0 1 k 0 0 0 k 0 0 0 1 Full 3x4 camera matrix P maps P3world to P2 image ? What if the image plane moves? A) Translation: • Given internal camera calibration K = • In (xc, yc)? changes px,py. In zc? focal length f: let k = (f+tz)/f, then: K’ = • Define effect of K’ on image points x,x’; x = [K | 0]X; x’ = [K’ | 0]X x’ = K’ K-1 x yc p c f zc xc

  35. Chapter 7: More One-Camera Details xfspx 0 yfpy0 0 1 Full 3x4 camera matrix P maps P3world to P2 image ? What if the image plane moves? B) Rotation: • Given internal camera calibration K = • Rotate basic camera’s output:about its center Cusing 3D rotation matrix R (3x3):x = [K | 0]X; x’ = [KR | 0] X x’ = [KR(K-1K) | 0] X = (KRK-1) [K|0] X • Get new points x’ from old image pts x (K·R·K-1) x = x’ • aka ‘conjugate rotation’; use this to construct planar panoramas yc R p c f zc xc

  36. Chapter 7: More One-Camera Details • ALL cameras gather the same image content!Same Center Point? Same image, just rearranged! (just a planar reprojection in P3) • The camera center Cmust move to change the image content:no zooming, warping, rotation can change this! • Gathering 3-D image data requires camera movement. Full 3x4 camera matrix P maps P3world to P2 image THUS: if the image plane moves: just rearranges the image points: a) Translations: b) Rotations: (K·R·K-1) x = x’ (K’ K-1) x = x’

  37. Movement Detection? • Can we do it from images only? • 2D projective transforms often LOOK like 3-D; • External cam. calib. affects all elements of P • YES. Camera moved if-&-only-ifCamera-ray points (CxX1,X2,…) will map to LINE (not a point) in the other image • ‘Epipolar Line’ == l’ = image of L • ‘Parallax’ == x1’x2’ vector X2 X1 x2’ x1’ L x C’ l’ C

  38. Cameras as Protractors • Define world-space direction d: • From a P3 infinity point D = [xd yd zd 0]T define d == [xd yd zd] • Use Basic Camera P0, • (e.g. C=(0,0,0,1), R=0, P = P0) • (Danger! now mixing P2, P3…) • Link direction D to image-space pt. xd=(xc,yc,zc):P0 Xd = [K|I]Xd =K d = xd • Ray thru image pt. x has direction d = K-1xd ~

  39. Cameras as Protractors • Angle between C and 2 image points x1,x2(see book pg 199) cos = x1T (K-TK-1) x2(x1T (K-TK-1) x1)(x2T (K-TK-1) x2) • Image line L defines a plane L • (Careful! P3 world =P2 camera axes here!) • Plane normal direction:n = KT L d2 n x2  d1 x1 C L

  40. Cameras as Protractors • Angle between C and 2 image points x1,x2(see book pg 199) cos = x1T (K-TK-1) x2(x1T (K-TK-1) x1)(x2T (K-TK-1) x2) • Image line L defines a plane L • (Careful! P3 world =P2 camera axes here!) • Plane normal direction:n = KT L d2 n Something Special here? Yes! x2  d1 x1 C L

  41. Cameras as Protractors What is (K-TK-1) ? • Recall P3 Conic Weirdness: • Plane at infinity  holds all ‘horizon points’ d (‘universe wrapper’) • Absolute Conic  isimaginary outermost circle of  • for ANY camera,Translation won’t change ‘Horizon point’ images: P Xd = x = KRd (pg200) • Absolute conic is inside ; it’s all ‘horizon points’ • for ANY camera,P  = (K-TK-1) = = ‘Image of Absolute Conic’

  42. Why do we care? P  = (K-TK-1) = = ‘Image of Absolute Conic’ • IAC is a ‘magic tool’ for camera calibration K • Recall let us find H from perp. lines. • Much better than ‘vanishing pt.’ methods • With IAC, find P matrix from an image of just 3 (non-coplanar) squares…

  43. Cameras as Protractors cos = x1T (K-TK-1) x2(x1T (K-TK-1) x1)(x2T (K-TK-1) x2) • Image Direction: d = [xc, yc, zc, 0]T • Image Direction from a point x: d = K-1x • Angle between C and 2 image points x1,x2:(pg 199) • Simplify with absolute conic : P  = (K-TK-1)=  = ‘Image of Absolute Conic’ d2 x2  d1 x1 C L

  44. Cameras as Protractors x1 x2 x3 0 x1 x2 x30 P  = (K-TK-1). OK.Now what was  again? Recall P3 Conic Weirdness: (pg. 63-67) • Plane at infinity  holds all ‘horizon points’ d (‘universe wrapper’) • Absolute Conic imaginary points in outermost circle of  • Satisfies BOTH x12 + x22 + x32 = 0 AND x42 = 0 • Can rewrite equations to look like a quadric (but isn’t— no x4) • AHA! ‘points’ on it are (complex conjugate) directions d ! • Finds right angles-- if d1d2, then: d1T· ·d2= 0 1 0 0 00 1 0 00 0 1 00 0 0 0 =dT··d

  45. Cameras as Protractors 1 23 4 1 2 34 P  = (K-TK-1). OK.Now what was  again? • Dual of Absolute Conic  is Dual QuadricQ* (?!?!) • More compact notation: for imaginary planes  • Same matrix, but different use: --find a plane  for every possible direction d -- is  to, and tangent to the quadric Q* •  is circle in  where tangent planes  are  to • Finds right angles-- if 12, then: 1T· Q*·2= 0 Inconsistent notation! 1 0 0 00 1 0 00 0 1 00 0 0 0 = T·Q*·

  46. Cameras as Protractors p1 p2 p3 0 1 0 0 00 1 0 00 0 1 00 0 0 0 P  = (K-TK-1)=  = ‘Image of Absolute Conic’ • Just as has a dual Q*,  has dual * : * = -1 = K KT • The dual conic * is the image of Q* , so * = P Q* = P = (first 3 columns of P?)

  47. Cameras as Protractors p1 p2 p3 0 1 0 0 0 0 1 0 00 0 1 00 0 0 0 P  = (K-TK-1)=  = ‘Image of Absolute Conic’ • Just as has a dual Q*,  has dual * : * = -1 = K KT • The dual conic * is the image of Q* , so * = P Q* = P = Vanishing points v1,v2of 2  world-space lines:v1T v2 = 0 Vanishing lines L1, L2of 2  world-space planes:L1T* L2 = 0 (first 3 columns of P)

  48. Cameras as Protractors Clever vanishing point trick: • Perpendicular lines in image? • Find their vanishing pts. by construction: • Use v1T v2 = 0, stack, solve for  = (K-TK-1) v3 v2 v1

  49. END

More Related