Shading Languages

1. Shading Languages Yung-feng Chiu

2. Agenda • Introduction • Pixar’s RenderMan, MS’s HLSL, NV’s CgFX, ATI’s RenderMonkey • Demos • .fx file • Comparsion

3. Key Idea of a Shading Language • Image synthesis can be divided into two basic concerns • Shape: Geometric Objects, Coordinates, Transformations, Hidden-Surface Methods… • Shading: Light, Surface, Material, Texture, … • Control shading not only by adjusting parameters and options, but by telling the shader what you want it to do directly in the form of a procedure

4. Pixar’s RenderMan • Separation of Modeling and Rendering • RenderMan serves as the interface. • Scene = Shape + Shading • The power of RenderMan is in the shading part.

5. RIB File .rib Rman Geom Code Rman Program cc render Program (rendrib) Shader (slc) Rman Shader .sl Byte-code Shader .slc txmake Image File TIFF image Rman texture Pixar’s RenderMan -2

6. RenderMan’s RIB File Structure

7. RenderMan’s Shader • Phong shader surface phong( float Ka = 1, Kd =1, Ks = 0.5; float roughness = 0.1; color specularcolor = 1; ) { normal Nf = faceforward( normalize(N), I ); vector V = -normalize(I); color C = 0; illuminance( P ) { vector R = 2*normalize(N)* (normalize(N) . normalize( L )) - normalize( L ); C += Ka*Cs + Kd*Cs*( normalize(N) . normalize(L) ) + Ks*specularcolor* pow(( R . V ), 10); } Ci = C*Cs; }

8. RenderMan’s Shader -2 • Attaching to the RIB file *********** AttributeBegin Translate 0 -1.5 0 Rotate 20 0 0 1 Color [ 0.8 0.0 0.0 ] Surface "phong" "Ka" [.1] "Kd" [.8] "Ks" [1] "roughness" [0.1] "specularcolor" [1 1 1] Basis "bezier" 3 "bezier" 3 PatchMesh "bicubic" 13 "nonperiodic" 10 "nonperiodic" "P" [1.5 0 0 1.5 0.828427 0 0.828427 1.5 0 0 1.5 0 -0.828427 1.5 0 -1.5 0.828427 0 -1.5 0 0 -1.5 -0.828427 0 -0.828427 -1.5 0 0 -1.5 0 0.828427 -1.5 0 1.5 -0.828427 0 1.5 0 0 1.5 0 0.075 1.5 0.828427 0.075 0.828427 1.5 0.075 0 1.5 0.075 -0.828427 1.5 0.075 -1.5 0.828427 0.075 -1.5 0 0.075 -1.5 -0.828427 0.075 -0.828427 -1.5 0.075 0 -1.5 0.075 0.828427 -1.5 0.075 1.5 -0.828427 0.075 1.5 0 0.075 2 0 0.3 2 1.10457 0.3 1.10457 2 0.3 0 2 0.3 -1.10457 **************

9. Assembly … dp3 r0, r0, r1 max r1.x, c5.x, r0.x pow r0.x, r1.x, c4.x mul r0, c3.x, r0.x mov r1, c2 add r1, c1, r1 mad r0, c0.x, r1, r0 ... Misrosoft’s HLSL (.fx) HLSL … float4 cSpec = pow(max(0, dot(Nf, H)), phongExp).xxx; float4 cPlastic = Cd * (cAmbi + cDiff) + Cs * cSpec; … • Simple Blinn-Phong shader expressed in both assembly and HLSL

10. nVidia’s CgFX Overview

11. nVidia’s CgFX Overview -2 • Supports Microsoft .fx files • Cg plus: • Multi-pass • Hardware fallbacks (techniques) • Complete Hardware states • Tweakables • MS .fx plus: • DirectX8 and OpenGL

12. nVidia’s CgFX Overview -3 • CgFX (.fx) • Manages whole rendering process • Handles render states – cross API support • Convenient exposure of tweakables & artist controls • Cg Shaders (.cg) • semantics directives to match your C++ and other custom hardware shaders • Bind textures/parameters to specific HW registers • Cg Runtime • Thin API to compile on demand at runtime • Optimizes & manages .Cg for range of target HW

13. nVidia’s CgFX Viewer

14. nVidia’s CgFX Viewer -2 • Scene graph GUI • .fx parameters edition • Error reporting for easy .fx file problem identification • Runs OpenGL, DirectX8, DirectX9 • Switch between devices at any point

15. Production Pipeline with CgFX

16. ATI’s RenderMonkey

17. ATI’s RenderMonkey -2 • Try to solve some of the problems developers face when designing software on emerging hardware • Create a flexible, extensible shader development environment that allows easy incorporation of existing APIs • Support for low level DirectX8/9 in the current version • Extensible framework to support emerging HLSL standards • DirectX9 HLSL • OpenGL 2.0 Shading Language • RenderMan • Maya Shade Trees • ……

18. ATI’s RenderMonkey -3 • Encapsulate all effect data in a single XML text file • Each Effect Workspace consists of • Effect Group(s) • Effect(s) • Pass(es) • Render state • Pixel Shader • Vertex Shader • Geometry • Textures • Variables and stream mapping nodes

19. Demos !

20. Bump Map Example • Bump mapping simulates detail with a surface normal that varies across a surface

21. RenderMan Example displacement lumpy ( float Km = 1, frequency = 1, maxoctaves = 6; string shadingspace = "shader"; float truedisp = 1;) { point Pshad = transform (shadingspace, frequency*P); float dPshad = filterwidthp(Pshad); float magnitude = fBm (Pshad, dPshad, maxoctaves, 2, 0.5); N = Displace (normalize(N), shadingspace, Km*magnitude, truedisp); }

23. Pixel Shader Example ps.1.1 tex t0 // base map tex t1 ; // bump map tex t2 ; // light vector from normalizer cube map dp3_sat r1, t1_bx2, t2_bx2 ; // N.L mul r1, r1, c0 ; // N.L * diffuse_light_color mul_sat r0, t0, r1; // (N.L * diffuse_light_color) * base

24. Cg Example f2fb DiffuseBumpPS(v2f IN, uniform sampler2D DiffuseMap, uniform sampler2D NormalMap, uniform float4 bumpHeight) { f2fb OUT; float4 color = tex2D(DiffuseMap);//fetch base color //fetch bump normal float4 bumpNormal = expand(tex2D(NormalMap)) * bumpHeight; //expand iterated light vector to [-1,1] float4 lightVector = expand(passthrough(IN.LightVector)); //compute final color (diffuse + ambient) float4 bump = uclamp(dot3_rgba(bumpNormal.xyz, lightVector.xyz)); OUT.col = color * bump; return OUT; }

26. Fixed Function Example Texture[0] = <normalMap>; Target[0] = Texture2D; MinFilter[0] = Linear; MagFilter[0] = Linear; MipFilter[0] = Linear; Texture[1] = <diffuseTexture>; Target[1] = Texture2D; MinFilter[1] = Linear; MagFilter[1] = Linear; MipFilter[1] = Linear; ColorOp[0] = DotProduct3; AlphaOp[0] = SelectArg1; ColorArg1[0] = Texture; ColorArg2[0] = Diffuse; AlphaArg1[0] = Texture; AlphaArg2[0] = Diffuse; ColorOp[1] = Modulate; ColorArg1[1] = Current; ColorArg2[1] = Texture; AlphaOp[1] = SelectArg1;

28. GL2 Logical Diagram

29. OpenGLFixed Function Vertex Vertex (object) Vertex (clip) Transform[MVP],[MV],[MV]-T Normal Vertex (eye) ColorSecondaryColor Front&BackColor [0,1] Lighting Front&BackSecondaryColor [0,1] Texgen TextureMatrixn TexCoordn TexCoordn EdgeFlag EdgeFlag

30. GL2 Vertex Processor Vertex (object) Vertex (clip) Uniform Normal Vertex (eye) ColorSecondaryColor VertexShader Front&BackColor Front&BackSecondaryColor TexCoordn Temporaries TexCoordn EdgeFlag EdgeFlag

31. OpenGLFixed Function Fragment Color SecondaryColor Texn TEn Sum Fog TexCoord[n] Color [0,1] z (|ze|,f ) Depth Depth Coord Coord FrontFacing FrontFacing

32. GL2 Fragment Processor Color Texture Uniform SecondaryColor FragmentShader TexCoord[n] Color z (|ze|,f ) Depth Depth Temporaries Coord Coord FrontFacing FrontFacing

33. Using FX in Your Application • Load effect • Validate technique for hardware • Detect parameters for technique • Render Loop (for each object in scene): • Set technique for the object • Set parameter values for technique • For each pass in technique • Set state for pass • Draw object

34. Using FX – The FX API LPD3DXBUFFER pError = NULL; D3DXCreateEffectFromFile(m_pd3dDevice, _T("simple.fx"), NULL, NULL, 0, NULL, &m_pEffect, &pError); SAFE_RELEASE(pError); . . . UINT iPass, cPasses; m_pEffect->SetTechnique(“Simple"); m_pEffect->SetVector(“var1", v); m_pEffect->Begin(&cPasses, 0); for (iPass = 0; iPass < cPasses; iPass++) { m_pEffect->Pass(iPass); m_pMesh->Draw(); } m_pEffect->End();

35. Effect File Structure • An effect is made up of multiple rendering algorithms (techniques) each made up of one or more passes • Effect File Structure: Variable declarations Technique 1 • Pass 1 • … • Pass n … Technique n • Pass 1 • … • Pass n

36. .fx file float4x4 worldMatrix : World; // World or model matrix float4x4 mvpMatrix : WorldViewProjection; // Model * View * Projection float4x4 worldViewMatrix : WorldView; // World * View texture diffuseTexture : DiffuseMap// Diffuse Map semantic < string File = "default_color.dds"; // Default texture file annotation >; texture normalMap : NormalMap// Normal Map semantic < string File = "default_bump_normal.dds";// Default texture file annotation >; float4 ambientColor : Ambient < > = {0.1, 0.1, 0.1, 1.0}; float bumpHeight < // GUI annotations string gui = "slider"; float min = 0; float max = 1; float step = 0.1; > = 0.5;

37. .fx file void DiffuseBumpVS(float4 Position : POSITION,// Position in object space float2 TexCoord : TEXCOORD0,// Texture coordinates … out float4 TexCoord0 : TEXCOORD0, // Texture coordinates out float4 Position : POSITION)// Position in projection space { TexCoord0.xy = IN.TexCoord.xy; // Pass texture coordinates for the diffuse map … Position = mul(WorldViewProj, IN.Position);// Compute position in projection space } void DiffuseBumpPS(float4 Position : POSITION, // Position in projection space … out float4 col : COLOR) { float4 color = tex2D(DiffuseMap, UV); // Look up the diffuse map … col = color * light; // Modulate the diffuse color by the light }

38. .fx file sampler2D diffuseSampler = sampler_state { Texture = <diffuseTexture>; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; }; sampler2D normalSampler = sampler_state { Texture = <normalMap>; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; }; technique CgTechnique // Both the vertex and the fragment shaders are in Cg { pass p0 { ZEnable = true; ZWriteEnable = true; CullMode = None; VertexShader = compile vs_1_1 DiffuseBumpVS(mvpMatrix, WorldIMatrix, lightPos); PixelShader = compile ps_1_1 DiffuseBumpPS(diffuseSampler, normalSampler, bumpHeight); } pass p1 { ... } }

39. Cg Pipeline Graphics programs are written in Cg ... Cg Runtime API ... and compiled to ... ... low-level assembly code ... ... that runs on any GPU compatible with DirectX or OpenGL

40. Using the Cg Compiler Application Development Your Application // // Diffuse lighting // float d = dot(normalize(frag.N), normalize(frag.L)); if (d < 0) d = 0; c = d*f4tex2D(t, frag.uv)*diffuse; … Cg program source code • Load/bind program • Specify program parameters • Specify vertex inputs • Render Cg Compiler … DP3 r0.x, f[TEX0], f[TEX0]; RSQ r0.x, r0.x; MUL r0, r0.x, f[TEX0]; DP3 r1.x, f[TEX1], f[TEX1]; RSQ r1.x, r1.x; MUL r1, r1.x, f[TEX1]; DP3 r0, r0, r1; MAX r0.x, r0.x, 1.0; MUL r0, r0.x, DIFFUSE; TEX r1, f[TEX1], 0, 2D; MUL r0, r0, r1; … Shader program assembly code Shader Compiler (nvasm.exe, psa.exe) Shader Binary

41. ASM Shaders (HW1) ASM Shaders (HW2) Game Image DCC Image Typical Production Pipeline Artists create models, textures, maps, … in DCC tool of choice Programmerswrite assembly for different hardware DCC tool (Maya, Max, SoftImage, …) Scene exporter plug-in Models, Textures, Maps, … Not the same! Application (game, renderer, …) Scene manager App Scene Manager hard-coded to choose at run-time the appropriate ASM shaders + state for the hardware

42. FX files Game Image DCC Image FX-Enabled Production Pipeline Artists assign FX files to scene objects and tweak parameters for each object in real-time Programmers and/or artists write FX effects DCC tool (Maya, Max, SoftImage, …) FX material plug-in Scene exporter plug-in Models, Textures, Maps, FX effects + parameters Same Image! Application (game, renderer, …) Scene manager For any FX, App Scene Manager chooses at run-time the appropriate technique for the hardware FX runtime

43. RenderMan OpenGL 2.0 D3D VS (2.0/3.0) D3D PS (2.0/3.0) Program Size No limit No limit 256/256 256/256 Vertex Attributes No limit >=16 16/16 Constants No limit VS:>=512 floats PS: >= 64 floats 128/256 32/128 Varying Parameters No limit >= 40 interpolators 10/10 10/10 Texture Samplers No limit No limit 16/16 Temp Registers No limit No limit 12/16 12/16 Constant-Based Flow Control Yes Yes Yes/Yes No/Yes Variable-Based Flow Control Yes Yes No/No No/No Comparison