Game Design Considerations for Alternate Controllers Greg LoPiccolo Ryan Lesser Harmonix Music Systems
Who is Harmonix? • Spun out of MIT Media Lab in 1995 • Focused on Interactive Music Innovation • Our Mission: to bring the experience of musical performance to non-musicians • Led to video game development
Why does Harmonix make games using alternate controllers? • By accident! • Mandate to pursue music software • Frequency/Amplitude required • Focus on design innovation • Led to . . .
What is an alternate controller? • Dance pad • Eyetoy / camera • Microphone • Light gun • Drum, Guitar, etc • Steering wheel
Why design games with alternate controllers? • Underappreciated area of game design • Controller has HUGE impact on • Mechanics of player abilities and constraints • Player role expectations • Provides access to new game experiences
Why design games with alternate controllers? • Lowers barrier to entry – more potential players • Conventional controllers canbe intimidating • Physical interactioncontributes to immersive experience
Harmonix Alternate Controller Design • Undertake two simple steps to figure out what game to make • Don’t pre-judge the process, but follow where it leads • Most importantly…
Early prototyping and iterative refinement is CRUCIAL • Build • Test • Revise • Repeat • But that is a different talk . . .
Designing for Alternate Controllers?Use These Two Easy Steps! • Step 1: Identify the desired player experience • Step 2: Evaluate and/or develop the controller
Step 1: Identify the desired player experience • Prerequisite for an effective design • Isn’t this obvious? • Yes, but implicit for most genres
Step 1: Identify the desired player experience • Alternate controllers imply fundamental shifts in player experience. • Think about the experience, NOT the game mechanic
Step 1: Identify the desired player experience • Incomplete understanding of gameplay at beginning of process • Clear understanding of experiential goals helps to evaluate ideas effectively • Easier to identify and kill off dead ends • Provides space for good ideas to flourish
Step 2: Controller Evaluation/Development • Needs to support play experience unachievable with a conventional controller • Alternate controller provides unique data • What opportunities does this data provide? • What limitations does it present?
Step 1: What is the core Karaoke experience? • Singing (drunk) • Performing in front of an audience (drunk) • Singing with other (drunk) people
Step 2: Evaluate Controller • Controller = Microphone • Everyone knows how to use one..which is a huge step forward! • What data does it provide for us to evaluate?
Step 2: Evaluate Controller Good Bad
Step 2: Evaluate Controller • Pitch • Volume • Lyrics (Phonemes)
Step 2: Evaluate Controller • Pitch detection – unique, intuitive, technically feasible • Volume detection ? – too variable to be useful • Phoneme detection? • Computationally expensive • Unforgiving • Time-consuming to develop
Shipping mechanic • Player sings vocal part of familiar song • Game grades on pitch accuracy • No attempt at speech parsing
Conclusion: Karaoke Revolution • Desired player experience = Karaoke performance • Usable controller data = pitch analysis • Outcome: decent (4 sequels and counting)
Step 1: What is the player experience? • Tougher question than for Karaoke Revolution • Possible player experience limitedby (as yet) unknown capabilitiesof controller • On to step 2 --
Step 2: Evaluate/develop controller • Controller system =body + camera + scene analysis code • Scene analysis is highly configurable: • Um, what do we want to accomplish? • Back to step 1!
Step 2: Evaluate/develop controller • What have other Eyetoy games done?
Back to Step 1: what is the player experience? • Mapping limb or body motions to character control could be immersive and intuitive • Control unmediated by handheld controller • Forward to Step 2!
Step 2: Evaluate Controller • What control data can we get from Eyetoy? • 2D Face-tracking • (After lots of work) 2D Hand-tracking • Nothing from feet
Flash of Insight: • Head is firmly attached to body • If body moves, head moves with it • Head tracking = body tracking!
Where does that lead? • Steering, jumping, ducking with head/body • Er, something with hands
Snowboarding? • It’s been done • How about . . .
Control Set • Left-Right Steering • Jump/Duck • Accelerate/Brake • Target-smashing with hands • Trick system gestures
AntiGrav Conclusions • Desired player experience: intuitive physical control of hoverboarding character • Useful controller data: head & hand positions, gestures • Outcome: mixed…works great in ideal setup,but fragile