1 / 30

Applications

Applications. Shmuel Peleg and Joshua Herman, “ Panoramic Mosaics by Manifold Projection ”, Computer Vision and Pattern Recognition (CVPR), 1997 Wolfgang Heidrich and Hans-Peter Seidel , “ View-independent environment maps ”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998

manning
Télécharger la présentation

Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applications Shmuel Peleg and Joshua Herman, “Panoramic Mosaics by Manifold Projection”, Computer Vision and Pattern Recognition (CVPR), 1997 Wolfgang Heidrich and Hans-Peter Seidel, “View-independent environment maps”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998 Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003 K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic Style-based Inverse Kinematics, Siggraph 2004 Richard Souvenir and Robert Pless. Manifold clustering. ICCV, pp. 648-653, 2005

  2. Applications using manifolds • Many problems can be phrased in manifold terminology • Provides an alternative way of viewing the problem • Also provides some formalism Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  3. Applications • Image-based rendering • Environment mapping • Animation • Surfaces • Parameterization • Modeling • Fitting • Consistent parameterization • Multiple, different parameterizations Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  4. Application: Panoramas • Problem statement: • Given images from a known camera movement • Rotation about camera axis • “Push-broom” pan (assumes negligible depth) • “Glue” images together into a single image Rover, nasa.gov Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  5. Push broom/vertical slit camera Peleg and Herman • Translation of camera • Image slit perpendicular to camera motion • Need not travel in straight line • Depth differences negligible • Parallax • Manifold is part of ground plane viewed by camera Direction of travel Ground plane Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  6. Camera rotation • Final image can be rendered on a cylinder • No parallax • Each image samples some number of pixels on cylinder (manifold) image Peleg and Herman Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  7. Practical problem • How to line up individual images to create one seamless image? • Manifold: Final image (3D function RGB on 2D manifold) • Charts: Individual images (2D charts) • Overlap regions/transition functions: Unknown • Assume translation • (Account for optical effects of camera) • Note: Only works for these two camera motions Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  8. General solution • Define a format for the transition function • E.g., translation in x,y • Define an error metric that measures how well two overlap regions agree • E.g., pixel difference • Optimize over free parameters in transition function • E.g., x,y shift between all pairs of overlapping images Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  9. Solving for overlaps, transitions • Find translation that minimizes pixel differences • Find y that minimizes || I0(s) – I1(y(s))|| • y(s) = s + Ds, where Ds is unknown 0 1 0 1 Ds Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  10. Final image • Transition functions align images (abstract manifold) • Final image colors? (RGB function on manifold) • Blend and embedding functions for each chart • Embedding function: Original image • Blending function: How much to use of each overlapping image • Usually favor very short blend regions Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  11. Application: Environment mapping • Place scene/model inside sphere • Light intensity/color found by intersecting normal with sphere • 1-1 mapping between normal direction and sphere • Every point on sphere assigned light intensity/color • Implementation • Store colors in one (or more) texture maps (2D) Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  12. Parameterization • Surface normal (point on sphere) to point in texture map • Atlas/local parameterization • Desirable properties • Even sampling of sphere • Adaptive • Partition • Overlap (mip mapping, continuity) • Simple to compute • Amenable to GPU implementation Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  13. Approach 1 • Single texture map • Not unique (poles) • Poor sampling • Simple to compute Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  14. Approach II • Cube mapping • Six charts • Discontinuities at edges • Sampling better at center of faces than edges • Simple (plane) computation • Which plane? Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  15. Approach III • Parabolic mapping • Chart functions use parabolic function • Better sampling • Slightly more computation • Less-noticeable seams Heidrich and Hans-Peter Seidel Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  16. Approach IV • Use chart approach • Allows for adaptive sampling (more detail where needed) • Chart sizes uniform: Tile texture map • Include overlap • Minimal extra texture map • Mip-mapping/down sampling • Example: 6 charts like cube • Charts extend into others • GPU implementation Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  17. Application: Animation • Human configuration space lies on a manifold of dimension n embedded in m dimensional space, where n << m • Articulated skeleton: over 40 degrees of freedom (shoulders, knees, hips, etc., each 1-3 degrees of rotation) • Individual motions (reaching, walking) certainly lie on lower dimension manifolds • End-point of reach plus time • Shape of manifold of all possible human motions? • Who knows? K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  18. Overview • Manifold learning • Data samples (e.g., motion capture, key frames) • Interpolation equals manifold construction • Editing equals manifold editing • 2D animation example • Manifolds in animation Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  19. 2D illustration • Two joint angles • Circle X Circle manifold (torus) • Animation • Repetitive motion • Joint angle plot • Circle manifold • Animation is a 1D manifold embedded in 2D Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  20. 2D illustration • Two joint angles • Circle X Circle manifold (torus) • Animation • Repetitive motion • Joint angle plot • Circle manifold • Animation is a 1D manifold embedded in 2D Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  21. Manifold construction • Input: Sample points in Rm • E.g., Motion capture sequence, each pose is a data point, m is number of dof of joints • 2D example: q,f for each pose • Assume data lies on a manifold of dimension n • Constraints on manifold shape/geometry (e.g., linear, no self-intersections) • Goal: Find/build manifold • Multiple manifolds Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  22. Manifold construction techniques • Principal components analysis (PCA), Independent components analysis (ICA) • Hyper planes • Support vector machines (SVM) • Deformed hyper planes • Isomap, Local linear embedding (LLE), Semi-definite embedding (SDE) • Planar, cylinder, sphere Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  23. Isomap, LLE, SDE cont. • Non-obvious failure modes • Circular/repetitive data sets • Self-intersections Modified : 1D embedding Raw result: 2D embedding Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  24. Manifold construction as learning • Use K neighbors to define chart domains (Uc) • Charts are “squished” Gaussians • Center, tangent vectors • Find transition functions (affine transformations) • Transformation takes tangent vectors into Rn • Aligns free vectors with neighbors Uc Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003 Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  25. Video Segmentation Manifold Clustering, Souvenir, Pless, ICCV 2005 • Motion Capture Data • 175 markers in 3D • 525 dimensions • 2212 frames • Accuracy: 94.8% • No domain knowledge • No human motion model Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  26. Uses of animation manifold • General idea: • Construct (implicitly or explicitly) a manifold representing valid human poses • Create a new animation sequence • Foot must touch here, reach here, etc. • Not sufficient to constrain all degrees of freedom (dof) • Project on to manifold to fill in remaining dof K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic, Style-Based Inverse Kinematics, Siggraph 2004 Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  27. Re-sequencing as embedded manifold • Goal: Given existing sequence (samples), add more/change samples • Assumptions: • Samples come from some smooth manifold • Some form of interpolation gives new samples on manifold • Current approaches: Interpolation between neighboring samples in sequence for given new time Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  28. Re-phrasing problem • Manifold learning or sequence timing provides parameterization/abstract manifold • Embed manifold with smooth function • Parameterization • Use function fitting • Re-sequencing: Evaluate embedding function E(M) Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  29. Caveats • What makes animation data difficult? • “Distance” loses meaning in >> 10 dimensions • Every point equally far away • Can’t enumerate • Noise • Error in capture process • Skeleton only approximates human motion • Joint angle representation • Don’t explicitly deal with manifold, parameterization Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

  30. Summary • Manifolds provide a formalism for breaking a problem into manageable pieces • Charts provide local parameterization • Planar domains • Overlaps: Natural mechanism for moving between parameterizations • Blend functions instead of geometric constraints • No boundary condition problems • Explicitly encapsulating/representing manifold is beneficial • Cleaner algorithm specifications Siggraph 2006, 7/31/2006 www.cs.wustl.edu/~cmg/

More Related