10 . 1. HLSL Effects I

10.1.HLSL Effects I Using effects within a SpriteBatch or as a post render effect

Question Clinic: FAQ In lecture exploration of answers to frequently asked student questions

HLSL Effects Introduction to the use of HLSL effects within a SpriteBatch or to provide a post render effect

HLSL Effects Shaders can be used to provide an interesting range of effects of use within 2D games, including: Special effects (e.g. image warping, shockwaves, etc.). Image enhancement (e.g. edge sharpening, glow, lighting effects, blur effects, etc.)

HLSL Effects (using a SpriteBatch) A SpriteBatch (itself a shader) can be used to support user-defined effects SpriteBatch Effect Begin Begin End End

HLSL Effects (using a SpriteBatch) protected override void Draw(GameTimegameTime) { spriteBatch.Begin( SpriteBlendMode.AlphaBlend, SpriteSortMode.Immediate, SaveStateMode.None); effect.Begin(); effect.CurrentTechnique.Passes[0].Begin(); spriteBatch.Draw(....); effect.CurrentTechnique.Passes[0].End(); effect.End(); spriteBatch.End(); } An effect can be included (with some caveats) within a SpriteBatch, as shown: Multiple effects can be chained within the sprite batch, likewise multiple images can be drawn by each effect. The SpriteSortMode must be set to immediate The first technique pass is assumed (a foreach could be used for multiple passes Aside: The SpriteSortMode must be set to Immediate

HLSL Effects (using a SpriteBatch) sampler2D textureSampler: register(s0); float4 PixelShader( float2 texCoord : TEXCOORD0) : COLOR0 { float4 colour = tex2D( textureSampler, texCoord); // Modify pixel colour as desired return colour; } technique effect { pass p0 { PixelShader = compile ps_2_0 PixelShader(); } } If using an effect within a SpriteBatch it is only necessary to define a pixel shader as the SpriteBatch will automatically pass in a full-screen quad containing the vertices to be drawn. A texture sampler is needed to obtain access to the texture drawn within the SpriteBatch. A pixel shader must only be defined, accepting a texture coordinate as input. The technique pass need only define the pixel shader to be used.

HLSL Effects (post-render effects) A post render effect kicks off once the entire screen has been drawn and provides some form of overall effect. This process involves creating a new render target into which the scene is drawn as normal. This render target is then modified by the post render effect and displayed in the back buffer. Terminology: Render target: A buffer where the GPU draws pixels as output from some effect. Back buffer: The default render target, containing the part of video memory that will next be displayed on the screen.

HLSL Effects (post render effects) Render Target Back Buffer Declare a suitable render target In each draw Tell GPU to draw to new render target Draw scene as normal Reset render target to back buffer Start sprite batch in immediate mode and start effect and pass. Obtain rendered texture Draw texture (within effect pass) End pass, effect and sprite batch 5 3 1 2 Sprite Batch 4 Sprite Batch 6 7 Effect

HLSL Effects (post render effects) protected override void Draw( GameTimegameTime) { graphics.GraphicsDevice.SetRenderTarget( 0, renderTarget); sprite.Begin(); // Draw scene as normal sprite.End(); graphics.GraphicsDevice.SetRenderTarget( 0, null); renderedTexture = renderTarget.GetTexture(); sprite.Begin(); effect.Being(); // Draw rendered image using effect effect.End(); sprite.End(); GraphicsDeviceManagergraphics; SpriteBatchspriteBatch; RenderTarget2D renderTarget; Texture2D renderedTexture; renderTarget = new RenderTarget2D( graphics.GraphicsDevice, screenWidth, screenHeight, 0, SurfaceFormat.Color); Tell the GPU to draw to the new render target Define reference to the graphics device manager and a sprite batch for drawing images Draw the scene as normal (drawn to the specified render target) Define a reference to the new render target and a reference to a texture to extract the contents of the render target Tell the GPU to draw to the normal back buffer Extracted the contents of the render target Draw the rendered texture to the screen using the desired effect Create a new render target, matched to the screen size (some effects might use a larger/small size). The surface format is also matched to that of the screen.

Example HLSL Effects Selection of simple HLSL effects

HLSL Effects (colour changes) Different forms of colour enhancement, e.g. strengthening or weakening the red-green-blue channels, can be applied to provide different effects, such as: producing a ‘night-time’ colour scheme, scene transitions, etc. coloured flashes for events, e.g. red flash or red tint for low health, ‘power’ effects, etc. Aside: SpriteBatch.Draw() colour parameter can be used for most basic colour change effects.

HLSL Effects • (colour changes) float4 colourReduction(float2 texCoord : TEXCOORD0) : color0 { float4 colour; colour = tex2D(texCoordSampler, texCoord); colour.r *= strength; return colour; } Aside: It is assumed that strength is in the range 0-1 float4 colourSaturation(float2 texCoord : TEXCOORD0) : color0 { float4 colour; colour = tex2D(texCoordSampler, texCoord); colour.g /= strength; return colour; } Colours will be automatically clamped to a maximum/minimum value of 1 and 0 float4 multiColourEffect(float2 texCoord : TEXCOORD0) : color0 { float4 colour; colour = tex2D(texCoordSampler, texCoord); colour.rg /= strength; return colour; } Any combination of the colour.rgba channels can be modified.

HLSL Effects (colour changes) Other possibly interesting colour effects include: Producing a grey-scale image (potentially combined with additional ‘noisy’ effects to provide an old film visual effect). Inverting the colours in the source image.

HLSL Effects (colour changes) float4 greyscale(float2 texCoord : TEXCOORD0) : color0 { float4 colour = tex2D( texCoordSampler, texCoord); colour.rgb = dot(colour.rgb, float3(0.3, 0.59, 0.11)); return( colour ); } In order to produce a grey-scale the RGB components are not averaged as the eye is not uniformly sensitive to each colour (instead green is weighted strongest, followed by red and finally blue). float4 negative(float2 texCoord : TEXCOORD0) : color0 { float4 colour = tex2D(texCoordtSampler, texCoord.xy); colour.rgb = 1 - colour.rgb; return colour; } The inverse is simply found by subtracting the RGB from 1, with the alpha channel remaining unchanged.

HLSL Effects (scale) A shader can also be used to zoom-in/out on a particular region of a defined texture. float4 scale(float2 texCoord : TEXCOORD0) : color0 { texCoord *= strength; float4 colour = tex2D(texCoordSampler, texCoord); return colour; } As strength ranges from 0 to 1, the texture lookup coordinate is reduced, effectively sampling from a smaller region of the source texture.

HLSL Effects (waves) By using a sinusoidal function it becomes possible to introduce a wave-like pattern of sampling, providing: ‘Heat-haze’ from hot objects. A water effect (either applied to certain ‘water’ textures or the whole scene.

HLSL Effects (waves) float4 water(float2 texCoord : TEXCOORD0) : color0 { float scale = 5.0; float magnitude = 0.01; texCoord += sin( scale * (texCoord + offset) ) * magnitude; float4 colour = tex2D(texCoordSampler, texCoord); return colour; } A new texture lookup coordinate is derived based on the sine of the original offset combined with the varying offset. The scale and magnitude settings control the overall ‘strength’ of the effect. float4 wavy(float2 texCoord : TEXCOORD0) : color0 { float scale = 20.0; float magnitude = 0.01; texCoord += sin( scale * (texCoord.x + offset) ) * magnitude; float4 colour = tex2D(texCoordSampler, texCoord); return( colour ); } This effect is nearly the same as the last one, except only .x component of texCoord is used

HLSL Effects (blur) A blur effect can be added by averaging each pixel’s colour using the colours of the surrounding pixels, e.g. for explosion effects, etc. float4 blur(float2 texCoord : TEXCOORD0) : color0 { float4 colour = tex2D( texCoordSampler, float2(texCoord.x+offset, texCoord.y+offset)); colour += tex2D( texCoordSampler, float2(texCoord.x-offset, texCoord.y-offset)); colour += tex2D( texCoordSampler, float2(texCoord.x+offset, texCoord.y-offset)); colour += tex2D( texCoordSampler, float2(texCoord.x-offset, texCoord.y+offset)); colour = colour / 4; return colour; }

HLSL Effects (sharpen) A simple sharpen effect can be obtained by ‘highlighting’ changes in pixel colour. float4 sharpen(float2 texCoord : TEXCOORD0) : color0 { float sharpenAmount = 25.0; float4 colour = tex2D( texCoordSampler, texCoord); colour += tex2D( texCoordSampler, texCoord - 0.0001) * sharpenAmount; colour -= tex2D( texCoordSampler, texCoord + 0.0001) * sharpenAmount; return( colour ); }

HLSL Effects (emboss/chalk) The emboss technique applies a colour tint based on the rate of colour change in the original image (similar to sharpen). A chalk-like effect can also be obtained by using large ‘sharpen’ values.

HLSL Effects (emboss/chalk) float4 emboss(float2 texCoord : TEXCOORD0) : color0 { float sharpAmount = 50.0f; float4 colour = float4( 0.5, 0.5, 0.5, 1.0); colour -= tex2D( texCoordSampler, texCoord - 0.0001) * sharpAmount; colour += tex2D( texCoordSampler, texCoord + 0.0001) * sharpAmount; colour = (colour.r+colour.g+colour.b) / 3.0f; return( colour ); } Specify the base colour for the effect float4 chalk(float2 texCoord : TEXCOORD0) : color0 { float sharpenAmount = 50.0f; float4 colour = tex2D( texCoordSampler, texCoord); colour += tex2D( texCoordSampler, texCoord - 0.001) * sharpenAmount; colour -= tex2D( texCoordSampler, texCoord + 0.001) * sharpenAmount; return( colour ); } This is the same code as for the sharpen effect, but with larger values

Summary Today we explored: • How HLSL effects can be incorporated into XNA • Introductory forms of pixel shader effects To do: • Complete Question Clinic • If using XNA, decide if you want to introduce custom effects into your game (more effects will be explored tomorrow)

10 . 1. HLSL Effects I