The Turing Test for Game AI - PowerPoint PPT Presentation

the turing test for game ai n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Turing Test for Game AI PowerPoint Presentation
Download Presentation
The Turing Test for Game AI

play fullscreen
1 / 39
The Turing Test for Game AI
78 Views
Download Presentation
thiery
Download Presentation

The Turing Test for Game AI

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. The Turing Test for Game AI Bing Gordon Electronic Arts June 2005

  2. “I wish someone would…” • …develop a 'metric' for computer npc ai. • Kind of an eliza scorecard.”

  3. 1. Context Powers of 10… • 1981 Atari VCS 128 bytes • 1986 NES 2K + 2K • 1989 Sega Genesis 64K+64K • 1994 Sony Playstation 2 MB + 512K • 2001 Playstation 2 32MB +4MB • 2006 PSX3/Xbox 2 512MB • 2012? PS4/Xbox 3? 5G?

  4. Context Powers of polish … • 75% of 90+ rated games are hits. • 25% of 90+ rated movies are hits. Source: Metacritic.com

  5. Aspirations Tiger swing Gameface Franchise mode Dynamic sports news Assymmetrical sides Badges Anti-grav gun Auction House Living city Crash mode Freestyle Console online voice and custom maps Powered by Feature Innovation

  6. Quality bars of the near-future • Procedurally generated content • “Emergent” behaviors, collisions • Believable characters • 100x physics • Portable avatars, persistent assets • Communities • Economies and money • Camera POV and LOD drives gameplay • Collaborative and dynamic intelligences

  7. AI could be a “killer app” feature of next gen • Characters: • Awareness • Memory • Complex motives, simple commands • 100x RAM allocation • Procedural responses • Parsers • Stitching • Pathfinding and cover • Must be co-developed with animators! • Game AI must be acted out and seen • Expressions & gestures> pathing

  8. EA Learning We are Influenced by the success of our “Killer Apps” • Pinball Construction Set -- floppy disk • Deluxe Paint -- graphics co-processor • EA Sports – 16-bit processor • Interactive movies – cd-rom • MMO’s – servers • Sims – object-oriented programming • Spore – procedural animation • Etc.

  9. 2. EA Learning EA Sports is a tech story • 16-bit (1990) Sports simulations • Individual ratings • Dynamic simulations • Playstation (1994) 3D Sports • Z-axis passing, tackling • Playstation 2 (2000) Articulation • Skeletal modeling and animation

  10. What EA learned from John Madden • The Oakland Raiders playbook • Matchup strategy • 5 zones of field-position • One Knee Equals Two Feet • Player ratings • All-Madden team

  11. The Madden Test (of game AI) • 1985: “That’s not football!” • 1990: “I’d fire the coach!” • 1995: “What are those guys doing?” • 2000: “Rookie, you’re cut!” • 2005: “That’s the way I designed it!”

  12. Madden Football Genesis23%

  13. Madden 97 Playstation54%

  14. Madden 2001 Playstation 263%

  15. Madden 2005 PS283%

  16. 97 -- Grand Theft Auto III 97 -- Tony Hawk's Pro Skater 3 96 -- Metal Gear Solid 2: Sons of Liberty 95 -- Grand Theft Auto: Vice City 95 -- Grand Theft Auto: San Andreas 95 -- Gran Turismo 3 A-Spec 95 -- Madden NFL 2003 95 -- Tony Hawk's Pro Skater 4 95 -- Devil May Cry 94 -- God of War 94 -- Madden NFL 2002 94 -- Madden NFL 2004 94 -- Burnout 3: Takedown 94 -- NCAA Football 2004 94 -- Virtua Fighter 4 93 -- Pro Evolution Soccer 93 -- SSX 93 -- SSX 3 93 -- World Soccer Winning Eleven 6 International 93 -- NFL2K 3. Reviewers What EA learned from www.metacritic.com

  17. 3. Reviewers AI is not critical…yet • AI cited for 6/20 top PS2 games. • Metal Gear, NFL x 4, Soccer • AI cited for 3/10 top PC games. • Half Life x 2, Civilization

  18. When AI is applauded… • Appropriate npc behaviors • Dynamic adjustments • Satisfying mistakes • Coordinated attacks/retreats • Challenging opponent

  19. When AI is punished… • Too easy • No cover • “Money plays” • Too dumb • Low awareness • “Deer in headlights”

  20. "Do NOT duck inside those doors, men! Honorable warriors need no cover!" Pointless Waste of Time.com

  21. 4. An experiment A Turing Test for Game AI(version 1.0) • Clips from 5 games • Madden, FIFA, Fight Night, Mortal Kombat • Many others could not be tested • “Player or AI controlled?” • App by Karim Mahrous, UCD phd candidate

  22. Format of test • Watch a clip, multiple replays • Player or AI control for specific character. • Rationale: • Pathing • Reaction • Timing • Effectiveness • Orientation • Awareness • 80 respondents

  23. App

  24. Madden 2001

  25. Madden 2001Results

  26. Madden 2005 – Clip 1

  27. Madden 2005 – Clip 1Results

  28. Madden 2005 – Clip 2

  29. Madden 2005 – Clip 2Results

  30. Results are discouraging (so far) • 60% accurate • Clip > game or technology • 0 correlation between rationale, correctness • Possible improvement in “orientation” AI between Madden 01 and 05.

  31. 5. For example Feature metacritic planning for

  32. Metal Gear Solid 2 95 metacritic overall Cited by 4/5 top reviews #2-4 feature mentioned “communication chain” “perceptive reinforcements” “teams of soldiers will find you” “frighteningly intelligent” “Outclassing every game on PS2” Half Life 2 95 metacritic overall Cited by 2/3 top reviews #5 feature mentioned “use cover effectively” “run away from grenades” “actively hunt players down and flank you ” “if you hurt them, they set up traps” “Nothing less than incredible” Competitive quality barEnemy Combat AI

  33. Civilization 2 94 overall Cited by 3/3 top reviews #2-5 feature mentioned “you pay for your mistakes” “clever and resourceful in negotiation” “respects your power in battles” “may act like a bonehead at times” Madden Football 94 overall Cited by 3/6 top reviews #2-6 feature mentioned “Geek out the AI to the viciousness you like” “Players make plays all over the field” “Tough half-time adjustments” “Gone are the massive scores…” “AI is advanced to another level” Hall of fame feature Opponent command AI

  34. Hall of Fame FeaturePathfinding and formations • Rome: Total War • 92 overall • Cited by 2/4 top reviews • #5 feature mentioned • “enjoyable and convincing” • “centered around troop morale” • “moments of divine inspiration” • “on occasion, cretinous”

  35. Recommended Spore AI goals • Half Life 2: enemy hunting AI • Metal Gear Solid 2: communication chains • Civilization 2: city/star combat and diplomacy • Madden Football: “half time adjustments” • Rome Total War: pathfinding and formations • Plus: • Procedural combat • Collaborative filtering

  36. 10 predictions Areas of discovery for next-gen features

  37. My bookshelf • Confessions of an Advertising Man, Ogilvy • Positioning, Ries and Trout • The Fifth Discipline, Senge • How Hits Happen, Farrell • Net Gain, Hagel & Armstrong • The Innovator’s Dilemma, Christenson • The Medici Effect, Frans Johansson

  38. Thanks • bgordon@ea.com