1. Game Design Considerations for Alternate Controllers Greg LoPiccolo Ryan Lesser Harmonix Music Systems

2. Who is Harmonix? • Spun out of MIT Media Lab in 1995 • Focused on Interactive Music Innovation • Our Mission: to bring the experience of musical performance to non-musicians • Led to video game development

3. Why does Harmonix make games using alternate controllers? • By accident! • Mandate to pursue music software  • Frequency/Amplitude required  • Focus on design innovation • Led to . . .

7. What is an alternate controller? • Dance pad • Eyetoy / camera • Microphone • Light gun • Drum, Guitar, etc • Steering wheel

8. Why design games with alternate controllers? • Underappreciated area of game design • Controller has HUGE impact on • Mechanics of player abilities and constraints • Player role expectations • Provides access to new game experiences

9. Why design games with alternate controllers? • Lowers barrier to entry – more potential players • Conventional controllers canbe intimidating • Physical interactioncontributes to immersive experience

10. Harmonix Alternate Controller Design • Undertake two simple steps to figure out what game to make • Don’t pre-judge the process, but follow where it leads • Most importantly…

11. Early prototyping and iterative refinement is CRUCIAL • Build • Test • Revise • Repeat • But that is a different talk . . .

12. Designing for Alternate Controllers?Use These Two Easy Steps! • Step 1: Identify the desired player experience • Step 2: Evaluate and/or develop the controller

13. Step 1: Identify the desired player experience • Prerequisite for an effective design • Isn’t this obvious? • Yes, but implicit for most genres

14. Step 1: Identify the desired player experience • Alternate controllers imply fundamental shifts in player experience. • Think about the experience, NOT the game mechanic

15. Step 1: Identify the desired player experience • Incomplete understanding of gameplay at beginning of process • Clear understanding of experiential goals helps to evaluate ideas effectively • Easier to identify and kill off dead ends • Provides space for good ideas to flourish

16. Step 2: Controller Evaluation/Development • Needs to support play experience unachievable with a conventional controller • Alternate controller provides unique data • What opportunities does this data provide? • What limitations does it present?

17. Sony PS2 Dual Shock Controller Data

18. Three Game Examples

19. Example 1:

21. Step 1: What is the core Karaoke experience? • Singing (drunk) • Performing in front of an audience (drunk) • Singing with other (drunk) people

22. Step 2: Evaluate Controller • Controller = Microphone • Everyone knows how to use one..which is a huge step forward! • What data does it provide for us to evaluate?

23. Step 2: Evaluate Controller Good Bad

24. Step 2: Evaluate Controller • Pitch • Volume • Lyrics (Phonemes)

25. Karaoke Revolution Microphone Controller Data Raw Input

26. Step 2: Evaluate Controller • Pitch detection – unique, intuitive, technically feasible • Volume detection ? – too variable to be useful • Phoneme detection? • Computationally expensive • Unforgiving • Time-consuming to develop

27. Shipping mechanic • Player sings vocal part of familiar song • Game grades on pitch accuracy • No attempt at speech parsing

28. Karaoke Revolution Microphone Controller Data Derived Input

29. Graphic

30. Conclusion: Karaoke Revolution • Desired player experience = Karaoke performance • Usable controller data = pitch analysis • Outcome: decent (4 sequels and counting)

31. Example 2:

33. Step 1: What is the player experience? • Tougher question than for Karaoke Revolution • Possible player experience limitedby (as yet) unknown capabilitiesof controller • On to step 2 --

34. Step 2: Evaluate/develop controller • Controller system =body + camera + scene analysis code • Scene analysis is highly configurable: • Um, what do we want to accomplish? • Back to step 1!

35. Step 2: Evaluate/develop controller • What have other Eyetoy games done?

37. Back to Step 1: what is the player experience? • Mapping limb or body motions to character control could be immersive and intuitive • Control unmediated by handheld controller • Forward to Step 2!

38. Step 2: Evaluate Controller • What control data can we get from Eyetoy? • 2D Face-tracking • (After lots of work) 2D Hand-tracking • Nothing from feet

39. EyeToy AntiGrav Controller DataRaw Input

40. Flash of Insight: • Head is firmly attached to body • If body moves, head moves with it • Head tracking = body tracking!

41. Where does that lead? • Steering, jumping, ducking with head/body • Er, something with hands

42. Snowboarding? • It’s been done • How about . . .

43. Hoverboarding!

45. EyeToy AntiGrav Controller DataDerived Input

46. Control Set • Left-Right Steering • Jump/Duck • Accelerate/Brake • Target-smashing with hands • Trick system gestures

48. AntiGrav Conclusions • Desired player experience: intuitive physical control of hoverboarding character • Useful controller data: head & hand positions, gestures • Outcome: mixed…works great in ideal setup,but fragile

49. Example 3