Compression and Simplification of 3D Models and Animations

1. ACM Solid Modeling Tutorial T3: Compression, simplification, and progressive transmission of 3D models and animations Jarek Rossignac College of Computing and GVU Center Georgia Institute of Technology Atlanta, Georgia - USA http://www.gvu.gatech.edu/~jarek

2. 3D morph, Best Paper T=T+T+T Compression, 2 Best Papers, MPEG-4 Simplification, Best Paper Jarek Rossignac, http://www.gvu.gatech.edu/~jarek • Maitrise ME & Diplome d’Engenieur ENSEM (Nancy, France) • PhD in Solid Modeling in 1985 (U. of Rochester, NY, USA) • IBM TJ Watson Research Center (11 years) • Senior manger of the Visualization, Interaction, and Graphics Department • Simplification: 3D Interaction Acceleration (3DIX), OpenGL Accelerator • Compression: VRLM. MPEG-4, 1 byte/triangle, Best Paper (ACM TOG) • Visualization: Data Explorer (DX) for scientific data, Panoramix for 360 • Georgia Institute of Technology (6 years) • Professor of Computer Science in the College of Computing • Director of GVU (Graphics, Visualization & Usability Center) 1996-2001 • Compression (with A. Szymczak): Edgebreaker: Best Paper (IEEE TVCG) • Multiresolution Analysis of Animations (with Turk, Szymczak, Ghrist) • Digital Clay (with Engineering): physical 3D surface as I/O • Other: Sweeps (Korea), Occlusion (Barcelona), Features (Genova)

3. v4 t3 v5 v2 Introduction Jarek Rossignac GVU Center and College of Computing Georgia Tech, Atlanta http://www.gvu.gatech.edu/~jarek

4. How should one measure shape complexity? • Number of components, handles and holes • Algebraic degree of bounding surfaces • Number of vertices in mesh • Stabbing number • Area3/volume2 • Curvature integral • Number of branches in skeleton • … • Number of bits required to store or transmit the model

5. Error Storage Storage size depends on • The shape, topology, and attributes of the model • Choice of representation • Acceptable accuracy loss • Compression used

6. 2D 1D 0D  Storage size depends on representation

7. x y z c vertex 1 Samples (vertices): x y z c vertex 2 x y z c vertex 3 Triangle/vertex incidence: v4 1 2 3 Triangle 1 3 24 Triangle 2 4 5 2 Triangle 3 t3 7 56 Triangle 4 v5 6 5 8 Triangle 5 v2 8 51 Triangle 6 Focus on explicit representation (T-mesh) • Samples: Location and attributes (color, mass) • Connectivity: Triangle/vertex incidence • Fit: Rule for bending triangles (subdivision surfaces, NURBS) V(3B+k) bits T = 2V V(6log2V) bits

8. Why triangles and tetrahedra? Triangles and tetrahedra are the simplest ways of specifying how irregular point-samples and associated values (color, density...) should be interpolated to approximate (non-homogeneous) sets. Other representations may be easily triangulated/tetrahedralized.

9. Triangle meshes vs. surfaces • Surface rep more compact than its triangulation? • Clearly true for a sphere or a simple curved CSG model • Not true for low-resolution tessellation of small trimmed NURBS patches • Control polygons of curved surfaces must be compressed • Complex models have millions of small parametric surfaces • We need to encode their control polygons (irregular quad/tri meshes) • Subdivision surfaces may be viewed as a compressed format • Most meshes do not have subdivision connectivity • It is tricky to compute a coarse mesh whose subdivision matches shape • Encoding corrective coefficients (wavelets?) may be cheaper • Coarse meshes of subdivision surfaces must be compressed • Coarse meshes of most subdivision surfaces are triangle meshes • You may have many of them in a scene

10. What makes I3DG effective? • Existence and immediate access to 3D models over the Web • Collaborative design • Scientific or business data • Electronic commerce • Entertainment I • Intuitive manipulation of view and model • 3D cues, natural interface, perspective (detail&context&background) • Real-time feedback • Direct manipulation

11. Bandwidth requirements for T-meshes • Naïve representation of a triangle mesh • Each triangle is represented by 3 vertices • Each vertex is represented by 3 coordinates • Each coordinate is represented by a float • Total storage = 576 bits per vertex (bpv) for geoemtry • 3x3x32 bits per triangle • Twice as many triangles as vertices • Not counting colors, normals, textures, motions

12. Problem: delays in accessing remote data • Most graphic 3D models are stored as triangle meshes • Good approximations require 1K-to-100M vertices per object • Uncompressed representation: 576 bits/vertex (+ properties) • Need days to download a 100M vertex model at home

13. Solution: A combination of techniques • Download and render only what is potentially visible • Use images (background, textures) when appropriate • Use low resolution geometry for distant “features” • Progressively refine (upgrade) “features” as they approach • Use geometric compression to transmit features & upgrades

14. Images, Textures, and Geometry • Video: Limits real-time interaction, redundant information • OK for canned trajectories and animated inserts for distant objects • Panorama: For fixed viewpoint or background • Image coherence: 3D info helps reuse • Can reuse previous image: distort it based on 3D info • Must send revealed portions and increased resolution as needed • Texture maps • Can substitute an imposter or textured crude mesh • May needs to replace it as the user moves • 3D Geometry + attributes: Most compact representation • Use to download 3D model of nearby objects for interactive viewing • Cheaper than video • Can’t use panoramas • Lightfields are too expensive

15. (7,7) ET (0,0) T Storage size depends on accuracy • Depends on vertex data quantization • Represent coordinates as normalized integers • Coordinates relative to bounding rectangle • Select unit for desiredresolution [0..2B] • Vertex coordinates = B-bit integers (6<B<14) Error EB • Depends on the sampling density (LOD) • Storage grow with the density of samples • Sub-sampled or simplified down to T triangles: Error ET • Estimate ET = K/T (where K approximates shape complexity) • Optimal choice of B and T (King&Rossignac) • Reduce error with a storage cap: F = T+3BcT/2 • Reduce storage with a cap on error bound: E=EB+ET

16. Different Error Measures screen • Image (color) fidelity (view dependent) • Error on the color of pixels • Sensitive to view direction and surface orientation • Too constraining for most 3D applications • Shading models are approximate • Users can’t detect shading inaccuracies • Screen space geometric error (view-dependent) • Measures silhouette displacement • Must preserve depth order • Bounded by projection of 3D deviation • Geometric 3D deviation (view-independent) • Bound defined by model/application tolerance • Hausdorff: H(A,B)=max(d(a,B),d(b,A)), aA, bB • Expensive to compute: (F1,F2,F3) • Poor measure of discrepancy • Often approximated conservatively eye Silhouette has moved eye Hidden part appeared

17. C R C C R C C R C C A C B C D V V=aA+bB+cC+dD+X Storage may be reduced by compression • Vertex coordinates (compress to 2 Bytes/vertex) • Normalize/quantize coordinates • Predict using decoded neighbors • Code residues using entropy compression • Incidence (< 2 bits/vertex) • Depth-first triangle-tree (spiral) • Encode as string of symbols • Fit rule and parameters (constant cost?) CCCCRCCRCRC…

18. Simplification & quantization Compression Complexity of a shape = Storage/Error curve Storage Error of the approximating model Curve depends on representation and compression scheme used Estimate ET = K/T

19. Terminology • Compression of geometry and connectivity • Loss-less encoding of connectivity, quantized vertices and attributes • Simplification: Levels of Detail (LOD) approximations • Lossy reduction of triangle counts by progressive vertex removal • Progressive Transmission of upgrades • Send crude approximation first. Then upgrades if/when needed • Adaptive refinement and Multi-Resolution Models • Refine only where necessary (close to viewer, near silhouette) • Retiling • Better (more regular) re-sampling of the surface to reduce storage • Error protection for noisy channels • Send extra bits to facilitate recovery of most important info

20. Summary • Interactive access and display of large 3D data is important • Many models are represented using triangle meshes • Others are triangulated for (Internet) graphics • Curved surfaces reps may be more compact than their tessellation, but we need to compress their control mesh • Sending uncompressed meshes take too much time • Sending images may help, but ultimately geometry is needed • Loss-less compression may not suffice • Vertex rounding (quantization) helps compress geometry • Entropy encoding of vertex prediction residues does too • We need to support LODs and progressive refinements • Estimating the simplification error is tricky and expensive