Designing Tools Enhance Interactive Experiences & the development process

Designing Tools EnhanceInteractive Experiences& the development process Magy Seif El-Nasr College of IST Penn State University

Interactive Entertainment: Industry Impact

Interactive Entertainment: Industry Impact • Game industry profits: • 2002: $1 billion more than movie industry • 2004: EA reported $1.43 billion quarterly earning • 2005: Nintendo $4 billion and expects to sell at least 2 million Wiis in US alone

Leaders project, ICT Devil May Cry II Virtual Aquarium, NCSU Importance of Game Engines + tools Training Entertainment Education

code Tools

Assembly .radix 16 ;all numbers hexadecimal start: mov #41,@#8264 ;system variable loop: jsr r4,print1 .asciz "HELLO WORLD! " br loop print1:jsr pc,@#1248 ;print string (ROM call) rts r4 C# System.Writeline (“Hello World”);

Learning and Computer literacy Impact Several Researchers believe that “video games provide an easy lead-in to computer literacy” – Cassell 2000 High Schools students and Modding Books on Modding for teens gaining popularity

Interactive Entertainment:Motivation and Learning “I want to learn how to make an awesome video game. I would like to learn everything about technology or at least more than I did” - middle school girl “I want to make my characters talk, build a world, and make interesting stories” - another middle school girl

Impact and Importance ofthe tools Facilitate development: games, training, health therapy application Facilitate learning: media, computing literacy Facilitate Creativity

Two thrusts of my research Tools for building engaging interactive experiences Game Engines + Tools as learning environments

Two thrusts of my research • Focus: visual design and character design • Based on cinematic and theatric theory • Using CI and AI techniques Tools for building engaging interactive experiences Game Engines + Tools as learning environments

My Research • Tools • Lighting • Camera (in progress) • Character (in progress) • Dance (in Progress)

Why Lighting? Lighting is Everything 

Mii • BS in Computer Science • Directed in Theatre • Graphics Design in Advertisement • MS in Computer Science • Studied Psychology and Neuroscience • PhD in Computer Science • 2 years studied acting & lighting design at Northwestern

Why Lighting? Dramatic Tension Visual Focus Style Feel of the space Time of Day Period visibility

Lighting in Traditional Media

Game Lighting Static Lighting Design Manually setting light layout (light maps) Advantages: • realistic • Controllable Disadvantages: • Don’t adapt to variations in the environment • Requires much memory x = Images from Lightmaps (static shadowmaps) article written by Kurt Miller from: http://www.flipcode.com/articles/article_lightmaps.shtml Image from Max Payne

Game Lighting Static + Dynamic Lighting • Manually scripting Lighting effects • Dynamic Character Lighting • Real-time Shadows • Dynamic user control of some lights

Game Lighting Static + Dynamic Lighting

Game Lighting Static + Dynamic Lighting Advantages: • More realistic • Adaptable to change Disadvantages: • Effects are scripted and rely on very careful design • Restricted • Based on realism • Compositional Balancing is done at design time

Problems: No modulation Screenshot from Mission 21 Devil May Cry Saturation Graph for Screenshot

Problems (Blade of Darkness): No adaptation to game Play

Problem 1: unpredictability } Develop a lighting plot or setup based on: • Narrative configuration: • Story • Conflict/dramatic tension • Timing and dramatic progression • Physical Configuration: • Camera orientation and position • Characters positions and orientations Requiring dynamic adaptation depend on user

Two Goals Dynamic Adaptive Lighting: • Better gameplay • Facilitate new Interactive models High-Level authoring of lighting: • Faster prototyping • Exploration of design space

Developing such as a Design Tool is Hard Why? Technical Issues of Dynamic Lighting Artistic control for style low-level balancing of context and lighting parameters but provide continuity

ELE – Expressive Lighting Engine A lighting system that: • Intelligently adjusts lighting in real-time accommodate context and effect • Based on cinematic & theatric theory • Allow artist to controllighting at a high-level

ELE The three subsystems:use optimization to find best solution given context, desired effects, state, and artists’ constraints ELE Artistic Constraints Previous State Current State WAMP (World Action Message Protocol) Allocation Subsystem Game/Rendering Engine Angle Subsystem LAMP (Lighting Action Message Protocol) Color Subsystem

Automatic light allocation • find best allocation given artistic constraints: • Modeling • Depth • Visibility • Visual Continuity • Visual Focus • Low vs. high key

Algorithm for dynamic allocation • Calculate visible area • Divide the zone into overlapping areas • Allocate a number of lights to areas,given that minimize: modeling depth Visual continuity visibility

Layout of lights

Lighting angle Selection • Select azimuth, elevation angles, given artistic constraints : • Visual Continuity • Motivation of direction • Visibility • Modeling • Mood • ELE finds best light angles to meet goals

Automating cinematic lighting designChoosing key light azimuth angle Optimizes: • k is the key light angle, k- is angle previous frame • m is the mood angle desired, V is visibility • v is the cost of deviation from best visibility angle • - is the cost of visual continuity • l is the cost of deviation from realistic direction • m is the cost of deviation from ideal mood angle visual continuity visibility & modeling mood motivation

Evaluation of key light angle visibility & modeling Light Nose line s k Camera From: Millerson’s The technique of lighting for television and Film, 1991 We described these rules as :

Showing character’s facial expressions and gestures ELE: Angle Subsystem

ELE: Angle Subsystem Showing character’s facial expressions and gestures, high tension

Emphasizing mood and mystery ELE: Angle Subsystem Mood Angle = side angle

ELE: Angle Subsystem Mood Angle = side angle Emphasizing mood

Choosing Colors • Color (Hue + saturation + intensity) • Compose colors for different areas on the set

Choosing Colors Adjust colors to accommodate desired artistic constraints: • Depth • Dramatic Intensity • Dramatic focus • low vs. high key setting • Specific author-suggested • Hue, Saturation, Lightness, color Warmth for focus, non-focus, and background • Palette restrictions specifying style + maintain visual continuityand style

Choosing Colors Optimize: depth contrast Palette constraints Visual continuity Artist’s desired color parameters

Choosing Colors Saturation Hue Lightness Warmth

Calculating Color Warmth Based on warmth perception Used linear fit to psychophysical data: Warm Cool

ELE: Color Subsystem Lightness (NF) Contrast (F – NF) Warmth (NF) warm tones, low color contrastfor low tension, realistic colors

ELE: Color Subsystem cool tones, low color contrastfor night scenes,or emphasize character

warm, high intensity contrastfor high tensions scenes ELE: Color Subsystem

Two Goals Dynamic Adaptive Lighting: • Better gameplay • Facilitate new Interactive models High-Level authoring of lighting: • Faster prototyping • Exploration of design space

Designing Tools Enhance Interactive Experiences & the development process