High Resolution Avatars from 3D Body Scans David Bruner, [TC] 2 Siggraph 2008 Booth #136

1. High Resolution Avatars from 3D Body Scans David Bruner, [TC]2 Siggraph 2008 Booth #136

2. 1999 Levi’s San Francisco [TC]2 History of Major Body Scanning Events 2000 Land’s End Mobile Tour 2001 2008 ImageTwin Real Time Avatars 2004-2006 - [TC]2 Scanner NX-12 -Volume Deployments 2002 - 2003 2007 NX-16 Scanner

3. NX-16 Scanner - “Changing Room Size” Body Scanner - 5X4 feet • Safe white light • Stand at floor level • Enhanced privacy • In changing room area • Self-operated scanning mode

4. Instructions and Light Optimization 20 sec Scan time 7 seconds Avatar Mesh 2 seconds 3D Point Cloud 12 seconds Measurement 3 seconds waist = 37.7-in hip = 42.8-in seat = 41.8-in thigh = 22.7-in knee = 15.0-in sideseam = 39.7-in inseam = 29.5-in crotch length = 26.0-in collar = 16.7-in front neck to waist = 20.2-in back neck to waist = 20.8-in cross shoulder = 19.1-in chest = 46.2-in cross chest = 16.3-in cross back = 16.0-in sleeve length = 33.8-in Landmark detection, body segmentation and data reduction 16 seconds Total Duration 60 seconds

5. Digital Humans (Avatars) from 3D body scans • Body Scanning enables the possibility of high quality Avatars that actually look like the consumer

6. Digital Humans (Avatars) from 3D body scans • Virtual Communities – Social Networks • Private Environment – personal Virtual Dressing, Virtual Fitness Assessment • Entertainment Networks – multi-player gaming, content creation • Business Networks • Online Conferences • Speeches • Job Interviews • Product Research • Industrial/Technical Applications -Ergonomics

7. Allen, Curless, Popovic – Siggraph 2003University of Washington

8. Fitting High Resolution 3D scans to a reference mesh • Consistent data organization (polygon count and organization identical to reference mesh) for all individuals • Reference mesh pre-made to be ideal for human shape

9. Allen - weighted optimization problem with 3 component objective function • Three components of objective function: • 1. Data Error • 2. Smoothness Error • 3. Marker Error • Allen proposes solving in 4 stages in which involves updating the objective function component weighting and mesh resolution (from low to high) to avoid local minima in solution convergence and reduce solution time. • Processing Time ([TC]2 implementation of Allen solution) • ~ 1 hour per body scan (700,000 scan data points, 30,000 vertex template mesh) • ~ Allen processes 200 scans and creates new meshes using Principal Compoment Analysis using this reference population

10. Long time vs Real time • [TC]2 – new direct transformation using markers only ~ 2 seconds • Key Similarity – morphing a pre-existing template mesh to 3D scan data • Key Difference – Allen uses sparse (70) manually placed landmark markers vs. [TC]2 dense (3,000) automatic landmark markers • Key Difference – [TC]2 easy integration of any reference mesh with wide ranging mesh densities (~ 4K – 80K vertices) • Key Difference – [TC]2 inclusion of joints/skeleton structure in the skin transformation

11. Automatic Dense (~3000) Geometry Markers ~ 1 second Landmarks intelligently placed based on Body Geometry-Shape Features

12. Reference Mesh – Scan Subject

13. Basic idea – drag template mesh to the 3D scan geometry using a common distribution of dense landmarks as the guide

14. Landmark to Mesh Relationship on Template Mesh

15. Distance Based Weighting

16. For b references over N vertices with affine combination constraint Weight Formulation (Pre-computed) *Similar to James, Twigg “Skinning Mesh Animations”, Carnegie Mellon

17. Marker Landmark (reference mesh) to Marker Landmark (body scan) Transformation Vectors Tj = lref – lscan j landmark markers j~ 3000

18. Updated Vertex Locations Xi = Xi ref + T

20. Accuracy

21. Template Mesh Resulting Avatar

22. All Shapes and Sizes

23. Morph Skeleton and Joints along with Skin

24. Virtual Dressing without draping– texture garments No additional processing time

25. Augmented Morphing - Virtual Fashion without Draping

26. Face Texture from Photo

27. Face Texture from Photo Combining image warping with 3D Mesh Morphing Image Features 10 clicks < 1 minute Scan Features 10 clicks < 1 minute

29. X3D H-ANIM • Any format template (mesh, skeleton/joints, texture), only vertex locations are updated from the template mesh (polygon count and organization unchanged) • For now – VRML, X3D, and .obj

30. X3D H-Anim Using Flux Studio – Flux Player from Vivaty

31. Obj Animation in Maya

32. Simulated Humans using PCANeed reference population of scan data • Personal Modifications (weight loss, fitness enhancement, etc) • Synthetic humans using PCA and limited data on the individual (height, weight, few measurements).

33. Scan Avatars + Principal Components Weight loss/gain Simulation Motivator -55 -20 lbsCurrent +20 lbs +55

34. Principal Components – Weight loss

35. Initial and 20 lb loss Predicted vs Actual

36. You possess and control your encrypted 3D data • Use it where you want • Patent Pending Data network architecture

37. Currently 25 locations around the world