Helper Joints: Advanced Deformations on RunTime Characters

1. Helper Joints:Advanced Deformations on RunTime Characters Jason Parks Character Technical Director Sony Computer Entertainment America

2. Contacts, Reference, Credits • Email: • jason@jason-parks.com • jason_parks@playstation.sony.com • Webpage (Helper Joints-GDC 2005): • http://www.jason-parks.com/HelperJoints • PAN Arts AVIs by Warwick Mellow • Human Model by Sven Jenson via Miller/Thuriot’s MasterClass 2003 (Character Creation Toolkit) • BodyBlend render by Zach Gray • Wrinkle R&D by Tyler Crook (SCEA)

3. Intended Audience • “Advanced – Requires experience and familiarity with subject.” • Maya Technical Artists who ‘Bind Characters’ for RunTime or Offline usage • Paint weights (Artisan and Component Editor) • Apply deformers • Use Set Driven Keys • Write Expressions • Use Utility Nodes

4. Topics • What are Helper Joints? • Definition • Why use Helper Joints? • New uses for Extra Joints • Smooth binding sucks & Where to use Helper Joints • How to use Helper Joints? • Our Goals, Disclaimer, Fusiforms • Research: The Basic Building Blocks • RunTime Rigs • SetDrivenKeys • Expressions • Constraints • Non-RunTime Rigs • Baking Out • Spline IK • Jiggle Deformer • Automating Production • Scripting • Workflow • Muscle Systems • Procedurally generated Helper Joints

5. Stop Talking, No more text, show some Moving Pictures!

6. Helper Joints Defined • “Helper Joints” are secondary joints offset from the standard hierarchy which will translate, rotate, or scale to achieve a “hand sculpted” affect on an area of vertices. • They can be within the primary hierarchy (parented, grouped, rigged), or outside of the primary hierarchy (point or parentConstrained) • They can be controlled by SetDrivenKeys, Expressions, Simple Rigs (constraint system), or Super Advanced Rigs (really any simulation you can think of) that are baked-out.

7. Why use any Extra Joints? Unlike characters rigged, bound, and rendered in Maya or any other high-end 3D package, your characters at run-time need to follow a very specific rule set depending on your Game Engine. Because all run-time engines already support joints, it is a tool we can exploit and push to its limits. • Traditional usage of Extra Joints to animate Props and Accessories • Extra “Helper Joints” to fix Smooth binding • Extra “Helper Joints” to create the next level in realism

8. Traditional Extra Joints • Traditional usage of Extra Joints for Props and Accessories: • Props, Rope • Hair, Ponytails • Capes, Necklaces, Clothing • Fat

9. New Ideas for Extra Joints • Extra “Helper Joints” to create the next level in realism: • Skin sliding • Wrinkles • Fat/Muscle/Flesh Jiggle • Muscle bulging

10. Fix Smooth binding • Extra “Helper Joints” to make skin transform properly: • Volume maintenance • Crease fixing • Complex Areas • Multiple joints affecting a single vert

11. Smooth Binding Limited • Vertex deforms by a weighted combination of Linear Translations around a joint rotating

12. Smooth Binding Shoulder

13. Smooth Binding Problem Areas • Elbow and Knee: • Interpenetration on the inside of the crease • Loss of volume on the outside of the crease (i.e. – the elbow tip or knee cap do not become ‘pointy’ when the elbow or knee bends) • Wrist/Forearm: • Loss of volume when the forearm/wrist is twisted too much down the length of the arm. (i.e. - ‘Candy-wrapper’ affect) • Upper Thigh/Gluteus: • Interpenetration at the bottom of the butt cheek when leg goes too far back • Shoulders: • Interpenetration in armpit • Interpenetration at the neck/shoulder junction area (trapezius) • Lack of representation of scapula movements

14. Our Goals – The “Ideal” • What are we really trying to get? • Reference with Motion • Books • “Anatomy of Movement” by Blandine C. Germain • Internet • PAN Arts by Warwick Mellow • Software • Absolute Character Tools for 3DSMAX

15. Our Goals • PAN Research • Pectorals • Shoulders

16. Our Goals • PAN Research • Clavicle • Forearm

17. Our Goals • Muscles – Independent Research

18. Our Goals • Muscles - Absolute Character Tools

19. Disclaimer: RunTime Rig Support • Individual Developer code will dictate what will be evaluated in Real-Time and what type of relation you can specify between your primary skeleton’s animation and your Helper Joints’ transformation. • Hopefully you can get 1 or more of these RunTime Rigging methods supported: • SetDrivenKeys • Expressions • Constraints

20. Disclaimer: RunTime Rig Support • NO: • Source Code provided for engine side. • YES: • Techniques for art-side production of rigs (in Maya) • Applicable to all games on all consoles

21. How To Get There • Muscles are the drivers in reality so lets create Helper Joints that mimic the way muscles work. • Simple FusiForm muscles are easy to mimic in Maya so lets create those and bind them to our skin

22. Research – FusiForm Muscle • “FusiForm” muscle: Quick Definition • ‘Tapering at each end; spindle-shaped.’

23. Research – The Building Blocks • Let’s pick the Biceps as our test case to create a muscle and mimic its action with a Helper Joint being driven by SetDrivenKeys: • Rotating • Scaling • Translating • The research explores the underlying mechanisms of the rigs • The ‘failed’ attempts still illustrate how rigs can be put together

24. Research – The Building Blocks UNDERSTANDING HOW THE RIGS WORK IS MORE VALUABLE THEN MY SUGGESTED SOLUTIONS

25. Research – Biceps (NURBs) • Muscle as Influence Object - NURBs

26. Research – Biceps (Polys) • Muscle as Influence Object - Polygons

27. Research – Biceps (Short) • SetDrivenKey w/ Single Driver Axis • Rotating • Short

28. Research – Biceps (Long) • SetDrivenKey w/ Single Driver Axis • Rotating • Long

29. Research – Biceps (Scale) • SetDrivenKey w/ Single Driver Axis • Scaling

30. Research – Biceps (Translate) • SetDrivenKey w/ Single Driver Axis • Translate

31. Research – Upper Arm (Translate) • Single Driver Axis • Translate • Works pretty good for Single driving axis! Warning: Not anatomically correct!

32. Research – Biceps (RollBone) • SetDrivenKey w/ Single Driver Axis • Translate • Add Roll • BREAKS!

33. Research – Biceps (RollBone) • SetDrivenKey w/ Two Driver Axis • Translate • Add Roll • w/ Extra Keys compensating for roll • Only fixes in when arm is straight out • When arm is bent it still BREAKS!

34. Research – Biceps (RollBone) • Add parent joint w/ another set of driven keys for roll • Rotation of Helper’s parent joint is percentage of primary hierarchies rotation

35. Research – Shoulder (Multiple Drivers) • SetDrivenKey w/ Single Driver Axis from Two bones • Translate • Combination is difficult

36. Research – Shoulders (Multiple Drivers) • SetDrivenKey w/ Two Driver Axis from Two bones • Translate • 2 Driving bones = BREAKS!(kinda) • + 2 Driving axis = BREAKS!BAD!

37. RunTime Rigs – SetDrivenKeys • Summation • SetDrivenKeys are fast, easy to use and work great for “simple” Helper Joints • 1 driveR joint and 1 driveR axis per driveN Helper Joint • Nest hierarchies of driveN Helper Joints if you have more than 1 driveR joint or axis

38. RunTime Rigs – SDKs for Volume Preservation • Forearm Helper Joint w/ SDK’s for Scale and rotation

39. RunTime Rigs – SDKs for Wrinkles • Scaling of Helper Joints (R&D by Tyler Crook @ SCEA)

40. RunTime Rigs – SDKs for Wrinkles • Joints are scaling on 2-axis (bulging), no scaling on length-wise axis • No translation or rotation

41. RunTime Rigs – SDKs for Wrinkles • Weighting is very tedious

42. RunTime Rigs – SDKs for Wrinkles • For shirt it takes many more joints because of 3 D.O.F. and larger surface. • 6 joints to do it right

43. RunTime Rigs – SDKs for Wrinkles • Different joints scale for each direction in each D.O.F. • Weighting is time consuming

44. RunTime Rigs – SDKs for Wrinkles • Incorporation: • Could theoretically add a scaling joint as a child of the helper joints • Then just weight individual vertices to these scaling joints to cause ‘wrinkling’ • Or just make separate scaling wrinkle joints wherever needed

45. RunTime Rigs • Done with SDKs • Start Expressions

46. RunTime Rigs – Expressions (Simple:Elbow) • Simple Helper Joints which only need a Translation Expression • Based on 1 axis from driving bone dj_helper.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle)))

47. RunTime Rigs - Expressions (Complex:Biceps) • Complex Helper Joints which need Translation & Rotation Expression • Translation: Use another expression to translate dj_bicep joint (parented to Bicep_Helper) for bulge based on elbow rotation. • Expression 2 – Describes bulge from rotation of elbow’s Z-axis dj_bicep.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle))) base.ty = 5, maxOffset = 1.8, maxAngle = 110 • Rotation: Joint needs to be parented to both shoulder and bicep-roll joints. Use Expression to take percentage. • Expression 1 – Describes rotation of parent joint around bicep-roll joint. (60% from shoulder, 40% from bicep-roll) Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx))

48. RunTime Rigs - Expressions (Complex:Biceps) Rotation: Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx)) Translation: dj_bicep.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle))) base.ty = 5, maxOffset = 1.8, maxAngle = 110

49. RunTime Rigs – Expressions & Constraints (Complex:Biceps) • Complex Helper Joints which need Scale & Rotation Expressions but looking at Distance • Requires 6 extra parent helpers (can be joints or ‘nulls’) • Component 1 – Scale (Bulge) • Handled by expression based on distance of ‘shock-absorber’ bicepHelper.sz = (scaleFactor*(1/(boneLength))) : boneLength = sqrt ((foreArmOffset.tx – bicepOffset.tx)² + (foreArmOffset.ty – bicepOffset.ty)² + (foreArmOffset.tz – bicepOffset.tz)²) • Component 2 – Rotate • Handled by expression base on % of angle of bicep_roll Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx)) • Requires 2pointConstraints

50. RunTime Rigs – Expressions & Constraints (Complex:Biceps) • Scale Expression based on distance