Sound and Electronic Games

1. Sound and Electronic Games Kurt Squire

2. Some of the greatest games of all time • Asteroids • Space Invaders • Pac Man • Super Mario Bros. 3 • Legend of Zelda: Ocarina of Time • Tony Hawk 2 • Baldur’s Gate 2 • Half Life • Thief • Deus Ex • The Sims

3. Sound is important, yet overlooked • Designers, consumers, and critics focus on graphics • People are much more forgiving of poor visuals than audio • Studies show that sound affects perceptions of image quality and peoples’ interpretations of events • Auditory channels work together with visual channels of perception • Sound design not integral to most game designs • Sound designers are frequently brought in as an afterthought. • Audio = 1%-2% of most game budgets, yet (ballpark) 25% of experience

4. On Designing Game Sound What people don’t understand is that you’re not making a movie, but you are attempting to re-create an interactive environment that contains visual and aural elements that combine to create an “experience” that’s totally defined by a player’s actions. Think of your sound designs going way beyond “making sounds for the graphics,” and come up with ways that the sound can be a critical element of the gameplay. The gaming industry is way behind on this concept, and you will see games coming out that use sound as important parts of the gameplay more often. – --Matthew Lee Johnson, Microsoft.

5. Questions Facing ElectronicGame Design • How does sound interact with other elements of the game design in order to support immersion? • How do we design“interactive” sound? • How can we avoid overly repetitive soundtracks? • How can we make sound an integral part of the gameplay? • How can we enable players to play with sound creatively? • How can sound be used to further the medium artistically?

6. Old dogs with good tricks • Music establishing mood and rhythm • (1) (2) • Pac-Man & Frogger • Sound effects creating mood • The joyful glee of Frogger reaching the lilly • The mournful death of Pac-Man • In some cases, when game designers were sound designers resulted more cohesive games • Sound was among the most identifiable aspects of the game

7. Supporting Immersion Through Sound • Sound provides feedback for players’ actions ( ) • Size and characteristics of space • What is this space? (1) (2) • How does it feel to be there? • Music & Effects: The emotional life of the game • Shenmue • Supporting immersion through interactive sound

8. Putting sound in the gameplay • Imbuing objects with meaning • What is this? What is it used for? • Through sound, it gains whole new meaning • How can the game world be affected by user created sounds? • Consistency, not realism within the game world • Critical for creating and sustaining immersion (Poole, 2000) • Exaggeration can be effective • Integrating sound into gameplay • Sound referring to events & time intervals ( ) • Locating objects (or people) in space (racing, 3D) • Players create sound to solve problems & act creatively (Thief) • Requiring players to use rhythm or timing (JGR)

10. Supporting Player Creativity With Sound • Allowing players to make an impact on the game world through sound • Allowing users to create music and sound in game • Loom, Zelda: Ocarina of Time • Opening up the engine: Allowing players to create and modify sounds • The Sims, Baldur’s Gate 2, mod packs

11. Thief

12. Juxtaposing of music & visual imagery • Music can be complementary rather than redundant • Music and sound can provide a powerful subtext in the gaming experience. • The Sims • Unlock the potential of the artistic potential of games through sound and music? • Game music interacting to players’ actions (Black & White) • Using juxtaposition to suggest commentary on players’ behavior

13. Quake

14. Quake Gets Funkier

15. Quake Gets …

16. Stealing musical concepts for gaming • The Importance of rhythm of game play • All things begin with the rhythm (Zelda) • Using music to enhance this rhythm, or play off of it • The importance of silence or no 3 hour guitar solos • Creating dramatic tension • Jamming: Collaborative improvisation between players & producers • If music game design is a collaboration with the player’s activity, how might developers design for this? • Musical models of creative collaborative models • Jazz, rock models of creative partnerships

17. Tensions • Creating an immersive environment vs. user needs (peace and quiet) • How to design for experiences, yet allow users to shape their own experiences • 40 hours of game play = one heck of a lot of music • Unique content Vs. Repetition • Original vs. scored soundtracks

18. Film, games, & Audio Many directors who like to think they appreciate sound still have a pretty narrow idea of the potential for sound in storytelling.  The generally accepted view is that it’s useful  to have "good" sound in order to enhance the visuals and root the images in a kind of temporal reality. But that isn’t collaboration, it’s slavery. 

19. Walkaways • Like film or television, music is a key element in establishing mood. • Visuals and audio work together to support immersion and affect experience. • The most successful games seem to weave patterns of sound together in response to the gameplay. • Sound can be used to give objects meaning, feeling, and emotion • Used cleverly, juxtaposition of sound & music can make artistic statement. • Game designers need to enable game players to be creative with sound. • Should a game be playable without sound? • Restoring the role of the sound engineer to designer