Case Study: Bo children’s book series

1. Case Study:Bo children’s book series Real-time Hand-Painted Graphics for Mobile Games Fabian Di Fiore, Tom Schaessens, Robin Marx, Frank Van Reeth, Eddy Flerackers Expertise Centre for Digital Media Hasselt University, Belgium

2. Introduction

3. Introduction • Many picture books and graphic novels with dynamic and interesting styles • For commercial reasons, publishers create derivative digital products including animation and games • When digitising these books for animation or games • backgrounds and scenes are just static images, which • appear instantaneously or through fading The entertainment industry is asking for alternatives

4. Motivation • Commercial opportunity as Case Study:Bo children's book series • Bo = transparent piglet • aimed at ages 4-6 • teach children how to deal with emotions • multiple traditional painting techniques • Recreate scenes from the book the way that the artist made them • unique recreation of the visual style • to visualise the progression of the story/game(as the player continues, more and more is revealed)

5. Approach • Tools to allow artists to quickly and intuitively create and animate hand-painted scenes • capturing the way an artist creates his picture books • allowing traditional hand-painted strokes to be generated digitally • Real-time animated recreation of hand-painted scenes running on mobile devices • animated recreation of captured scenes (e.g., stroke by stroke) • optimisation for mobile devices • Integration in existing (game) engine • Unity3D

6. Basic Considerations

7. Hand-painted Look • Physical simulation

8. Hand-painted Look • Non-photorealisticRendering (NPR)

9. Hand-painted Look • Raster graphics

10. Hand-painted Look • Raster graphics

11. Hand-painted Look • Trade-off • Accuracy • indistinguishablefrom reality • Performance • speed of recreatingcaptured data • Animatability • ability to animate stroke by stroke • drawingfromstart to finish • Changeability • flexibilitywhendesigning (e.g., movingdrawnstrokes) • Artisticfreedom

12. Mobile GPU Performance • Immediate Mode Rendering (IMR) • < PC graphicsindustry • forwardfacingpolygon -> rendered -> shaded -> textured -> Z-buffer -> frame buffer • (most) androiddevices • Tile-BasedDeferredRendering (TBDR) • < PowerVR architecture • tilebased (parallel) + deferredrendering (hidden surface removal) • all Apple mobile devices

13. Conclusion • Original plan • Raster graphics and TBDR • Bad benchmarking test: ARM + PowerVR architecture • write-to-texture operations not efficient for pixel-basedstrokes • Mesh-basedStrokes • Combination of NPR and raster graphics • Creation: strokes are flat meshesapproximating the contour of brushstrokes • Colouringusing textures: GPU shader program • Allows for fastrendering of meshes on the GPU

14. creating a hand-painted look Implementation

15. Introduction to the Visual Style of Bo • Main characteristics • Rubberpaint roller for background elements • Paper cutstencils for addingdetails • Scannedpaintings for animals and insects • Blending for the final composition

16. Creating a Base Stroke • Sampling of input points (< digital tablet) • Pressure determineswidth • Boundariesdeterminegeometry of brushfootprint • Fading out of paint • Use uv-coordinates as a length-parameter to determinestart and end of the stroke as a shader has no context

17. Rubber Paint Roller • General fading out supported by base stroke • Background • consist of transparant, slowly fading strokes • new parameters: fade offset + fade multiplier • Unique strokes • createwide texture usingtwodollops of paint • for each stroke randomly select narrow band from texture

18. Rubber Paint Roller • Pattern of white dots whenrolling back and forth in a fanning pattern • due to the relief created by the paint itself as thin layers of dried paint on the surface of the roller prevent paint in between from touching the paper • Employ a black and white mask • white representing spots where the paint should adhere, black where it doesn’t • add mask as texture and multiply

19. Grass Brush • Createeffect of individualblades of grass • artist: achieved by ripping of a corner of paper, dipping it in paint and dragging it over the canvas • tool: mask with transparent inside and grass-like outside is stretched over the length of the stroke

20. Other Brushes • Using masks and additional parameters • Oily brush • Stencil roller brush • Stamp brushes

21. content creation pipeline toolset Implementation

22. Brush Editor • Management of brushes, preview canvas…

23. Animation Tool • Painting, timeline, recording functionality…

24. Results

26. Results • 60 strokes • 4500 vertices • 2400 triangles

27. Results • 150 strokes • 3600 vertices • 2300 triangles

28. Discussion • iPad2 • Initially frame rate dropped from 60 fps to 5 fps • due to overdraw of transparent strokes: cover significant amount of the screen and, hence, there is much overlap

29. Discussion • Render-to-texture (RTT) • render to image first i.o. rendering to screen • drawback: background is flattened • RTT for strokes that have been drawn completely and need not to be animated anymore • do this for every 10-20 new strokes • Consistent frame rate of 60 fps

30. Video

31. iTunes App Store: “Bo vindtalles”