MED06: Silverlight 3D Graphics

1. Silverlight 3D Graphics Aaron Oneal Program Manager Microsoft Corporation

2. Session Overview Feature Intro 3D Control Composition 3D Graphics Essentials Animation Texturing Advanced Topics Roadmap

3. 3D Feature Intro

4. 3D Feature IntroWhy 3D graphics? Microsoft�s continued commitment to enhancing Silverlight and serving the needs of .NET developers A top request from our customers (ranked 3rd on user voice) A leap forward in terms of visualizations, performance, and interactivity Modern devices are 3D ready

5. 3D Feature IntroData visualization: Geographic overlays http://blprnt.com

6. 3D Feature IntroData visualization: Science and astronomy http://www.worldwidetelescope.org

7. 3D Feature IntroData visualization: 3D charts and reports Nevron Chart

8. 3D Feature IntroMedical imaging

9. 3D Feature IntroBusiness: Home design

10. 3D Feature IntroTarget applications Data visualization* 3D charts and reports Scatter points Geographic overlays Science & astronomy Education & training Marketing & advertising Business* Manufacturing Industrial processes Home design Realty & virtual tours Customer support Medical Games & simulation

11. 3D Graphics API announcing

12. 3D Feature IntroWhat does the API look like?

13. 3D Feature IntroHigh-level components

14. 3D Feature IntroXNA core for Silverlight 5

15. 3D Feature IntroRuntime requirements

16. 3D Feature IntroSecurity features

17. 3D Feature IntroHow does Silverlight 3D relate to other Microsoft technologies?

18. 3D Feature IntroHow does Silverlight 3D compare to other industry technologies?

19. 3D Feature IntroWhen will Silverlight 3D be available?

20. 3D Control Composition Integrating 3D into a project

21. 3D Control CompositionEnable GPU acceleration In the HTML page hosting the Silverlight control � <object data="data:application/x-silverlight-2," type="application/x-silverlight-2" ...> <param name="EnableGPUAcceleration" value="true" /> </object> Omit this and you get an empty scene.

22. 3D Control CompositionAdd a DrawingSurface Insert the drawing surface into your XAML page � <Grid x:Name="LayoutRoot" Background="Black"> <DrawingSurface Width="800" Height="400" Draw="OnDraw" /> </Grid> The control honors layout so Width and Height are optional.

23. 3D Control CompositionDrawing // Callback is invoked every frame (to max refresh rate) when invalidated public void OnDraw(Object sender, DrawEventArgs args) { // Clear the surface args.GraphicsDevice.Clear(...); // Draw args.GraphicsDevice.DrawPrimitives(...); // Invalidate the surface to schedule a callback // for the next frame. Use this in the callback, // use DrawingSurface.Invalidate() outside. args.InvalidateSurface(); }

24. 3D Graphics Essentials Introducing the 3D graphics API

25. 3D GraphicsModels and Geometry Objects or models in 3D are a mesh of triangles Triangles can be shaded with a color or image (a texture) and then illuminated

26. Triangle Render 3D Graphics Essentials

27. 3D GraphicsPoints, lines, and polygons Point coordinates are represented as a 3-component vector (x,y,z) called a vertex Vertices are connected to form primitives like triangles Triangle lists or strips form a mesh

28. 3D GraphicsCoordinate systems XNA math library uses a right-handed coordinate system by default with Z+ pointing at the viewer

29. 3D GraphicsRendering Pipeline

30. 3D GraphicsVertex pipeline Vertex data: Model vertices are stored in vertex buffers Vertex formats are flexible Position Color Texture coordinates

31. 3D GraphicsVertex declaration public struct VertexPositionColor { public Vector3 Position; public Color Color; public static readonly VertexDeclaration Declaration = new VertexDeclaration ( new VertexElement(0, VertexElementFormat.Vector3, VertexElementUsage.Position, 0), new VertexElement(12, VertexElementFormat.Color, VertexElementUsage.Color, 0) ); }

32. 3D GraphicsVertex buffer VertexBuffer CreateTriangle() { var vertices = new VertexPositionColor[3]; vertices[0].Position = new Vector3(-1, -1, 0); // left vertices[1].Position = new Vector3(0, 1, 0); // top vertices[2].Position = new Vector3(1, -1, 0); // right vertices[0].Color = new Color(255, 0, 0, 255); // red vertices[1].Color = new Color(0, 255, 0, 255); // green vertices[2].Color = new Color(0, 0, 255, 255); // blue // Create a vertex buffer and copy vertex data into it var vb = new VertexBuffer(graphicsDevice, VertexPositionColor.Declaration, vertices.Count, BufferUsage.WriteOnly); vb.SetData(0, vertices, 0, vertices.Length, 0); return vb; }

33. 3D GraphicsVertex pipeline Vertex processing: Vertex shaders apply transformations and lighting to vertices stored in the vertex buffer Shaders: Programs that normally execute on the GPU Shaders are written in high-level shader language (HLSL) with a default entrypoint called main() Shaders are compiled using FXC from the DirectX SDK or other popular shader tools

34. 3D GraphicsVertex pipeline Vertex coordinate transformations

35. 3D GraphicsVertex pipeline Geometry processing Clip vertices outside of viewable area Cull surfaces that are facing away from the view Rasterize geometry to pixels

36. 3D GraphicsPixel pipeline Pixel processing Pixel shaders process input vertex and texture data and output pixel color values Pixel rendering Final rendering processes modify pixel color values with alpha, depth, or stencil testing, or by applying alpha blending or fog

37. 3D GraphicsDraw the triangle void OnDraw(object sender, DrawEventArgs args) { var graphicsDevice = args.GraphicsDevice; graphicsDevice.Clear(ClearOptions.Target | ClearOptions.DepthBuffer, Color.Transparent, 1.0f, 0); // setup vertex pipeline graphicsDevice.SetVertexBuffer(vertexBuffer); graphicsDevice.SetVertexShader(vertexShader); graphicsDevice.SetVertexShaderConstantFloat4(0, ref worldViewProjection); // setup pixel pipeline graphicsDevice.SetPixelShader(pixelShader); // draw and schedule another callback graphicsDevice.DrawPrimitives(PrimitiveType.TriangleList, 0, 1); args.InvalidateSurface(); }

38. Triangle Render 3D Graphics Essentials

39. 3D GraphicsDrawing guidelines When drawing every frame, keep the Draw callback fast and tight Avoid creating graphics resources inside the Draw callback Avoid allocating new objects inside the Draw callback Minimize state changes by rendering objects with similar state requirements in sequence

40. Animation

41. Cube Sample Animation

42. AnimationTransforms Animation: Changing the transform of an object over time Continuous rotation WorldTransform = Scale � (Rotation � TotalTime) � Position

43. AnimationTime Frame rate is inconsistent by nature, use time instead Use delta time between frames for animation calculations Directional movement NewPosition = Origin + Direction � Speed � DeltaTime

44. AnimationTime properties Draw callback provides two convenient time values for animations TotalTime: Total time elapsed since the application was started  DeltaTime: Time since the last draw update

45. AnimationRotating cube public void Draw(GraphicsDevice graphicsDevice, TimeSpan totalTime, Matrix viewProjection) { // create a continuously advancing rotation Matrix rotation = Matrix.CreateFromYawPitchRoll( MathHelper.PiOver4 * (float)totalTime.TotalSeconds, MathHelper.PiOver4 * (float)totalTime.TotalSeconds, 0.0f); Matrix worldViewProjection = rotation * viewProjection; // � graphicsDevice.SetVertexShaderConstantFloat4(0, ref worldViewProjection); graphicsDevice.DrawPrimitives(PrimitiveType.TriangleList, 0, 12); }

46. Rotating Cube Animation

47. Texturing Shading with images

48. TexturingEssentials Textured surface Image color data Normal (bump) maps Light maps Details Texture sampler: Reads pixels from a texture and applies level-of-detail filtering Texture coordinates: (u, v) and range from 0 .. 1 across the image

49. TexturingLoad a texture from an image public Cube() { // Initialize resources required to draw the Cube. Resource initialization should // be done outside of the draw callback to keep the composition thread free. // Load image Stream imageStream = Application.GetResourceStream( new Uri(@"CubeSample;component/SLXNA.png", UriKind.Relative)).Stream; var image = new BitmapImage(); image.SetSource(imageStream); // Create texture texture = new Texture2D(resourceDevice, image.PixelWidth, image.PixelHeight, false, SurfaceFormat.Color); // Copy image to texture image.CopyTo(texture); }

50. TexturingColor formats Silverlight  XNA

51. TexturingNormal map (bump map) =

52. Advanced Topics

53. Advanced TopicsMultiple threads Silverlight uses multiple threads for performance UI / application thread Composition / render / independent animation thread The Draw callback occurs on the composition thread Use different devices for drawing vs. resource creation Use a thread safe data model

54. Advanced TopicsMultiple graphics devices: Drawing public partial class MainPage : UserControl { void OnDraw(object sender, DrawEventArgs args) { // Use args.GraphicsDevice for drawing and only inside the callback args.GraphicsDevice.DrawPrimitives(�); args.InvalidateSurface(); } }

55. Advanced TopicsMultiple graphics devices: Resource creation VertexBuffer CreateTriangle() { � // Use this global device for creating resources, not for drawing GraphicsDevice resourceDevice = GraphicsDeviceManager.Current.GraphicsDevice // Create a vertex buffer and copy vertex data into it var vb = new VertexBuffer(resourceDevice, VertexPositionColor.VertexDeclaration, vertices.Count, BufferUsage.WriteOnly); vb.SetData(0, vertices, 0, vertices.Length, 0); return vb; }

56. Advanced TopicsThread safe data model Update your data model on the UI or background thread based on user input, physics simulation, etc. not the composition thread. Read from the data model in the draw callback Use a lock if necessary for data that needs to be updated or sampled atomically Avoid locking the entire callback by caching data locally at the beginning of the callback Don�t access a DependencyObject from the draw callback

57. Advanced TopicsStoryboards Silverlight has a Storyboard animation system Great for animating a DependencyProperty which follows a deterministic path 3D is commonly dynamic and animated by transforms

58. Advanced TopicsRender modes public class GraphicsDeviceManager { public event EventHandler<RenderModeChangedEventArgs> RenderModeChanged; ... } public enum RenderMode { Unavailable, Hardware } public enum RenderModeReason { Normal, GPUAccelerationDisabled, SecurityBlocked, Not3DCapable, TemporarilyUnavailable } �

59. Advanced TopicsBlending and pre-multiplied alpha Full support for color blending Final Color� =� (Source Color� x �Source Factor)� Operation� (Dest Color x Dest Factor) Surfaces, color, and blends default to pre-multiplied alpha Use Color.FromNonPremultiplied() to specify color using traditional channel values Default BlendState.AlphaBlend uses Blend.One instead of Blend.SourceAlpha

60. Advanced TopicsBuilt-in Effects Built-in Effects will be provided for the most common rendering tasks Basic Effect Dual Texture Skinned Effect Environment Map Alpha Test You can use these in many cases to skip writing shaders.

61. Feature Roadmap

62. Feature RoadmapNot this release Out of scope features and community opportunities Custom effects Content pipeline, Model, etc. 3D audio Input and devices Gamer services WPF Media3D Community content Balder 3D Engine � Declarative XAML 3D for multiple devices SilverSprite � 2D XNA sprites

63. Feature RoadmapStill ahead in Silverlight 5 Multisample Anti-aliasing Render target textures Device management Configurable surface options Built-in effects

64. Session Recap Feature Summary 3D Control Composition 3D Graphics Essentials Animation Texturing Advanced Topics Roadmap

65. Thank You! Please fill out the session evaluation survey ?

66. Questions? Discussion: http://forums.silverlight.net/ Email: aaron.oneal@microsoft.com Blog: http://aarononeal.info

68. Speaker Time Changes