Further Programming for 3D applications CE00849-6

1. Introduction to Further Programming for 3D application Faculty of Computing, Engineering and Technology Staffordshire University Further Programming for 3D applications CE00849-6 Bob Hobbs

2. Outline • Module Details • What is 3D programming? • Typical Processing Steps • Modelling and Rendering • Windows and Direct X • Programmable Pipeline • Summary

3. Module Details • Teaching Team • Bob Hobbs r.g.hobbs@staffs.ac.uk • Steve Foster s.foster@staffs.ac.uk • Semester 2 15 cats • 3 Hours per week • 1 Hour Lecture Thu 4pm • 2 Hours Practical Mon 9-11am& 2-4pm K106 Fri 1-3pm & 3-5pm K106

4. Module Details Course Handbook & Lecture Notes • Blackboard Assignment Details • 50% assignment work • 50% two hour exam

5. Program of Study • Week 01 Introduction to Module and DirectX11 • Week 02 Culling, Clipping and Line drawing • Week 03 Lighting and Shading using Programmable pipeline • Week 04 Textures & Texture Mapping • Week 05 Blending, Stencilling and use of Geometry Shader • Week 06 Meshing and Instancing • Week 07 Data structures for graphics engines • Week 08 Picking, Interaction and Motion Cues • Week 09 Tessellation and Terrain • Week 10 Cube, Normal and Bump mapping • Week 11 Other SFX • Week 12 Review and Assessment

6. Viewer Synthetic image will vary according to: viewing direction, viewer position, illumination, object properties, ... Projection onto 2D surface Object What is 3D rendering? Generally deals with graphical display of 3D objects as seen by viewer

7. Specification Rendering Graphical display such as What is 3D Computer Graphics? • 3D graphics: generation of graphical display (rendering) of 3D object(s) from specification (model(s)) Modelling

8. Light source Transformation Hidden surface removal Vertices Shading Viewpoint Facets Typical Processing Steps Wireframe polygonal model Solid object

9. Illumination model Graphics engine Object model(s) Graphical display Viewing and projection specification Typical Processing Steps Modelling: numerical description of scene objects, illumination, and viewer Rendering: operations that produce view of scene projected onto view surface

10. yv Surface material Light source 2D picture xv zv yw ym 2D device coordinates xm zm xw 3D viewing coordinates & Culling ym zw xm zm 3D world coordinates Typical Processing Steps Graphics pipeline: generation of picture involves sequence of coordinatetransformations, hidden surface removal, shading/texturing and etc... 3D modelling coordinates

11. 2D picture Typical Processing Steps Mesh operations Geometry processing: shape, world geometry, viewpoint Pixel operations Materialprocessing: colour, texture, transparency/occlusion, … Shading, texturing, ... Hiddensurface removal Generation / configuration ofworld Object & back-face culling Viewmapping onto displaydevice Geometry models Viewpoint Viewport Viewpoint Lights & materials • Light pos., orient. • Optical char. • Shapes • World pos., orient., sizes • Pos., orient. • Field of view • Projection type • Pos., orient. of display area Typical graphics pipeline

12. Modelling 1438 facets Human Head Model

13. Modelling 7258 facets Human Head Model

14. Modelling 2074 facets Teacher and Board Model

15. Rendering 1438 facets Shaded Human Head

16. Rendering Shaded Teacher and Board

17. y Point grid (raster) Scan line External point Point / Pixel x Rendering Rasterisation of filled triangle • Common approach: polygon interior filled line by line, one point at a time (rasterisation) • linearinterpolation (i.e. line equation) often used for colour shade, depth and other calculations at each pixel

18. Windows Programming • Rendering takes place in a window • Application must generate window in which to render graphics • IP3DA used C and GLUT to handle windows. OpenGL can run in GLUT or other windowing framework including MS windows • DirectX is integrated into windows programming API

19. What is a “window?” • The most obvious windows are the application windows on your desktop. • The term “window” also refers to any child window or control (such as a button) within a window. • Windows are organized in a hierarchy.

20. Example - Calculator

21. Event Driven Programming • Windows programming is event driven. • An event driven program sits and waits for an event to process. • Events include moving or sizing a window, clicking the mouse button over a window, or typing keys on the keyboard. • Windows programs are notified of events by messages sent from the OS.

22. Win32 API • We will use the Win32 Platform API to write a windows program. • Typically more tedious, but we only need to create one window. • Faster and Smaller executable. • Written in C++ • Other options include: • MFC, Qt, wxWindows, SDL, .NET Forms

23. WinMain • Instead of main, windows programs use WinMain • WinMain has different parameters: • HINSTANCE hInstance – a handle to the program • HINSTANCE hPrevInstance – no longer used. • LPSTR lpCmdLine – unparsed command line. Doesn’t include executable’s filename. • int nCmdShow – Specifies how the window should be initially shown (ie. Minimized, Maximized, Normal) • WinMain must have a WINAPI modifier. • int WINAPI WinMain (…) • WinMain still returns an integer like main.

24. Headers • To use the Win32 API we need to #include windows.h • If we #define WIN32_LEAN_AND_MEAN before #including windows.h, the compiler will skip compiling the more rarely used windows code.

25. Windows Program Outline • Create Window Class • Create Window • Show the Window • Enter Message Loop

26. Window Classes • A window class acts as a template for window creation. • Window classes are not classes in the C++ sense. • We can set such properties as: • Name of the window class – important for identifying it later! • Window style – How it looks and its default behaviors. • Window procedure – a pointer to function that handles messages for a window. • Default cursor, icon, and menu.

27. Creating a Window Class • Fill out a WNDCLASS or WNDCLASSEX structure. • Use the structure to register the class with a call to RegisterClass or RegisterClassEx, respectively.

28. Create the Window • Create a Window with a call to CreateWindow or CreateWindowEx. • Some of the parameters you will need to enter include: • The name of a window class. • The name of the window. • A handle to a parent window. (NULL if no parent) • Initial size and position of the window. • CreateWindow and CreateWindowEx will return a window handle (HWND) if successful or NULL if failed.

29. Showing the Window • When we create a window it starts off hidden. • To show the window you must call ShowWindow (hWnd, nCmdShow) • hWnd – returned from CreateWindow/Ex. • nCmdShow – parameter from WinMain. • This will send a message to the window telling it to show. • Call UpdateWindow (hWnd) to force the window to process the show message immediately.

30. Message Loops • A message loop is needed to forward messages to their windows. • There are different ways to make a message loop but here is the most common one: MSG msg = {0}; // returns FALSE when message is WM_QUIT while (GetMessage (&msg, NULL, 0, 0)) { TranslateMessage (&msg); // translate keyboard messages. DispatchMessage (&msg); // send message to its window. }

31. Message Loop (Cont’d) • The loop uses: • A MSG structure – Holds message information. • GetMessage – Retrieves the next message. Waits for a message if the message queue is empty. • TranslateMessage – Translate some keyboard messages. • DispatchMessage – Dispatches the message to the appropriate windows procedure.

32. Windows Procedure • Every window has an associate procedure for handling messages. • Windows procedures have the following format: • LRESULT CALLBACK ProcName (HWND hwnd, UINT msg, WPARAM wp, LPARAM lp);

33. Window Procedure Example • Here is an example procedure: LRESULT CALLBACK WndProc (HWND hwnd, UINT msg, WPARAM wp, LPARAM lp) { switch (msg) { case WM_DESTROY: PostQuitMessage (0); return 0; } return DefWindowProc (hwnd, msg, wp, lp); }

34. Window Procedure Parameters • The procedure has 4 parameters: • HWND hwnd – handle to the window receiving the message. This is useful when multiple windows share the same procedure. You can think of it sort of like a this pointer. • UINT msg – The message. • WPARAM wp and LPARAM lp – These are parameters to the message and may represent different things.

35. Window Procedure Return • When the procedure completely handles a message, it should return 0. • The procedure should let the OS handle all other messages by calling DefWindowProc and then return its result.

36. WM_DESTROY and WM_QUIT • Initially we only want to handle one message: WM_DESTROY. • This message is called when a window is closed. • Note that just because the window is destroyed, the program will still run. • To end the program we send WM_QUIT by calling PostQuitMessage.

37. WM_* • Other messages you may want to handle include: • WM_CREATE • WM_SIZE • WM_LBUTTONDOWN • WM_LBUTTONUP • WM_MOUSEMOVE • WM_KEYDOWN • WM_KEYUP • WM_PAINT • Check MSDN literature for information about these messages: http://msdn.microsoft.com/

38. Direct X • The window has to obtain a handle to a Direct 3D and swap chain to render graphics objects in the windows object created. • In DirectX 11 this is based on the Programmable Pipeline.

39. Introduction to PPL • Shader: custom program segment inserted into graphics pipeline and executed on GPU • replaces specific parts of fixed-function pipeline • comprises instructions that operate solely on: • vertex data (vertex shader) • or pixel data (pixel shader) • geometric primitive data (geometry shader)

40. Introduction to PPL • Shaders offer code development flexibility and processingspeed: • flexibility: shader implements customised vertex, polygon, or pixel processing • speed: shader runs on fast specialised chips • many current graphics cards support shaders • programmable hardware pipeline on most modern GPUs • Shaders often used for better visual quality in real time

41. Programmable Graphics Pipeline • Feature of DirectX Graphics and OpenGL: • support for programmable pipeline: • API provides high-level or assembly language interface to graphics processing hardware • specialisedprogramming language forvertex and pixel processing • developer can program special code for real-time graphics applications • e.g. rich graphical effects such as simulation of cloth, hair or fur

42. Application D3DX D3D HAL Device driver interface Hardware Programmable Graphics Hardware Layered architecture of fixed-function graphics system DirectX Graphics

43. Programmable Graphics Hardware Graphics processing unit (GPU) • Special purpose microprocessor(s) • multiple programmable processing units for concurrent processing on single chip • optimised for graphics processing, e.g. vector / matrix manipulations • used in computer graphics (accelerator) card • for manipulation and display of graphics data • can perform some tasks of pipeline • evolving hardware: capability improving with time

44. Programmable Graphics Hardware ATI Radeon HD 5670 (supports DirectX 11) Source: Don Woligroski, The Radeon HD 5670 Architecture, Tom's Hardware, January 14, 2010 (http://www.tomshardware.com/reviews/radeon-hd-5670,2533-2.html)

45. Programmable Graphics Hardware CPU GPU VertexProcessor FragmentProcessor Frame bufferOperations Assembly &Rasterization Application Frame buffer Textures GPU programming model (Source: Bill Mark, “NVIDIA Programmable Graphics Technology”, SIGGRAPH 2002 course)

46. Programmable Graphics Hardware DirectX 10 graphics pipeline (see http://msdn.microsoft.com/en-us/library/windows/desktop/bb205123(v=vs.85).aspx)

47. Shader Languages • Shader programming requires languagespecialised for graphics • needs graphics-specific data types (e.g. vector, matrix) and operations (e.g. dot product, matrix multiply) • language often tied to specific graphics API • Actual executableshader code is machine code • but high-level language can be compiled to machine code • compilers available

48. Shader Languages HLSL • Sample intrinsicdatatypes: • vector types: contain from 1 to 4 components (arranged as single row or column), e.g. • int1: vector containing 1 int; float3: vector containing 3 floats; double4: vector containing 4 doubles • matrix types: contain rows and columns of same-type data, e.g. • int2x1: integer matrix with 2 rows and 1 column; float4x4: float matrix with 4 rows and 4 columns

49. Shader Languages HLSL • Sample intrinsicfunctions: • dot(a, b): returns dot product of vectors a and b • mul(A, B): returns product of matrices A and B • reflect(i, n): returns reflection vectorgiven incident ray direction i and surface normal n • round(x): returns x rounded to nearest integer

50. Shader Languages Sample code: simple HLSL shaders Example (from DirectX Graphics documentation (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/HLSL_Shaders.asp)) /* Vertex shader Functionality: • transforms vertex position from object space to projection space • assigns per-vertex colour from vertex input */ // Note: ‘: POSITION’ and ‘: COLOR0’ are semantics that tell compiler usage of variable float4x4 mWorldViewProj; // World * View * Projection transformation // (global variable set by application) // shader output structure struct VS_OUTPUT { float4 Position : POSITION; // vertex position float4 Diffuse : COLOR0; // vertex diffuse color };