Marc Atkin John Abercrombie

1. Using a Goal/Action Architectureto Integrate Modularity and Long-Term Memory into AI Behaviors Marc AtkinJohn Abercrombie

2. Why Do We Want Better AI? • Bad AI is always noticed • AI often inconsistent between games – why can’t we build on previous work? • The AI is a big part of what makes a game world immersive and fun

3. What’s Stopping Us? • What makes AI complex? • AI has many possible reactions to many situations • Many behaviors are operating on the AI at any given time • Hard to account for all the possible interactions

4. The Basic Idea: Use Modularity • It’s not just for coding anymore! • Produces AI that is easier to understand and debug • Produces AI that it reusable within and between projects • The key point: Design the AI architecture so that it facilitates modularity

5. Part 1: Tyrion

6. Tyrion • An architecture for specifying and executing AI behavior • Implements a language in which to write behavior • Encourages hierarchical, modular design

7. Resources, Goals, and Actions • Goal: A description of adesired world state • Action (or behavior):A method for achieving a goal • Resource:An entity required toperform an action

8. Resource Hierarchy Team Squads Vehicles Driver/Gunners Characters Legs/Arms/Head

9. Resource hierarchy Team Strategic level Squads Tactical level Vehicles Driver/Gunners IndividualUnits Characters Legs/Arms/Head Motor Control

10. Action/Goal Hierarchy Attack Goal Squad Attack From All Sides Attack Goal Character Attack Goal NormalAttack FlankingAttack Legs / Arms MoveTo Goal FireAt Goal MoveTo Fire Weapon

11. Action Structure • Goal satisfied by action • Selection heuristic • Evaluates appropriateness of actionin a given situation • Message callback functions for child actions • Message callback functions for sensors • Body (Unreal script latent code) • Typically executes over a number of game ticks

12. Action Idioms • Simplify writing action bodies • Encapsulate commonly performed tasks • Examples: • WaitForAllGoals • WaitForAnyGoal

13. Execution Model • 10 times a second: • Iterate over every resource: • Order unmatched goals by priority • Find actions that achieve them • Tick every running action • Tick every active periodic sensor;send messages • If an action completes, mark its goal as achieved/failed; send messages

14. Part 2: Building Modular Actions

15. SWAT4 • What is SWAT4? • First person tactical shooter • Player is team leader • Single player – player commands four officer AI’s • Object is to kill or subdue all enemies, and rescue the hostages

16. What Makes Up the SWAT4 AI? • Hierarchical resources for organizing behaviors Squad Tactical Character Individual Motor Control Movement Weapon • Layer behaviors to form more complex behaviors

17. Modularity in SWAT4 • Start from the ground up – Motor Control • The Legs - Movement Resource • Basic Movement – move to a position, move to a particular object, follow another AI • Lower body rotation • The Arms – Weapon Resource • Aiming – aim at a position, to a rotation, at something • Item usage – (i.e. guns, grenades, toolkit, explosives)

18. Modularity in SWAT4 • The Character • Controls arms and legs • Plays special full body animations • The Squad • Controls individual characters, tells them what to do so they can work together

19. Reuse • Tyrion framework encourages tasks to be broken apart into sub-tasks • Higher level behaviors use lower level behaviors to accomplish complicated tasks • Lower level behaviors can be used over and over again by different higher level behaviors • i.e. Movement, aiming, weapon usage

20. Layering • Each behavior on a resource is responsible for controlling everything below it • i.e. Officers move in a formation while aiming at points of interest in the world (Fall in Command) • Using multiple characters in a squad allows group to work together to accomplish more

21. Organizing Behaviors • Use of action idioms • waitForGoal, waitForAllGoals, waitForAnyGoals, waitForResourcesAvailable • Parent action posts child goals to accomplish tasks • on the same resource or one lower in the hierarchy

22. Organizing Behaviors • In SWAT4 the Squad posts child goals on characters (the officers) • Character behaviors usually post child goals on movement and/or weapon resources • Each action can wait for child goals to complete using action idioms

23. Example: Move Squad to a Point • Squad behavior (MoveSquadTo): • Posts MoveTo Goal on each officer • WaitForAllGoals(MoveToGoals) • Character behavior (MoveTo): • Posts Aim Around Goal on weapon resource • Posts MoveToLocation Goal on movement resource • WaitForGoal(MoveToLocation)

24. Stack Up (on a door) • User points at door, gives command • Squad behavior determines points for Officers to stack up at • Each officer locomotes to point given - while lookingaround or aimingat the door ifthey are able 1 1 2 2

25. How Stack Up Works • Squad Behavior (Squad Stack Up) • Post Stack Up goal on each officer • Pass Stack Up Point as destination parameter • WaitForAllGoals(StackUpGoals) • Character Behavior (Stack Up) • Post Aim At Door goal • Post MoveTo(StackUpPoint) goal • WaitForGoal(MoveToGoal) • Result: Movement & Weapon behaviors act independently of each other

26. Move & Clear • Inherits from the Stack Up behavior • Re-use functionality of getting AI’s stacked up on the door • Adds new behaviors: • After finishing stacking up (but not trying the door): • Officer near the door opens it • Another Officer told to start moving as soon as the door is opened • Team then moves through the door into next room to points specified by a designer

27. Breach, Bang & Clear • Inherits from the Move & Clear behavior • Open Door behavior replaced: first officer blows door open using C2 explosives or a breaching shotgun • Second officer throws flashbang grenade into room before entering • Squad organizes characters • Some basic character behaviors are reused across many commands • i.e. deploy grenade, deploy shotgun, or deploy C2

28. Squad StackUp Stack Up Squad Behaviors Squad Move & Clear Move & Clear Squad Breach Bang & Clear Aim At Open Door Character Behaviors Move To Throw Grenade Aim Around Movement & Weapon Behaviors Move To Deploy C2 Use Grenade Deploy Shotgun Move To Move To Inherits From Child Behavior Move To

29. Summary and Results • Framework is key to organizing use of system • Time to prototype and develop AI behaviors reduced • Interesting SWAT4 Stats: • Number of Squad Behaviors: 33 • Number of Character Behaviors: 65 • Number of Weapon Behaviors: 10 • Number of Movement Behaviors: 11 • Number of Months Programming: 16 (May 2003 – September 2004)

30. Part 3: Long-term Memory

31. The Problem of Forgetful AI • “Oh hello – have wemet before?” • State machines don’tremember past events • What’s a natural way to represent task-related memory?

32. Two Types of Memory • Task memory: • What needs to be done • The context of the current action • Represented by the set of goals • Cognitive memory: • Sensor data that has accumulated during the AI’s life time • Represented inside sensors and actions

33. Tribes: Vengeance • First-person shooter • Emphasizes speed and freedom of movement (jetpacks!) • 3rd in the series

34. A Day in the Life of a Grunt • A Grunt AI with three goals: • PatrolGoal (priority 40) • AttackGoal (priority 50, dormant) • DodgeGoal (priority 90, dormant) • AttackGoal will activate when an enemy is sighted • DodgeGoal will activate when a visible projectile will hit the AI

35. Patrol Goal Attack Goal Dodge Goal Patrol is executing 40 legs arms

36. Patrol Goal Attack Goal Dodge Goal Patrol is executing 40 legs Enemy spotted! Sensor wakes up AttackGoal arms X 40 50 Attack executes; posts subgoals for legs & arms legs 50 arms

37. Patrol Goal Attack Goal Dodge Goal Patrol is executing 40 legs Enemy spotted! Sensor wakes up AttackGoal arms X 40 50 Attack executes; posts subgoals for legs & arms legs 50 arms Projectile spotted! Sensor wakes up DodgeGoal X X 40 50 90 Dodge executes; needs legs; arms continue executing Attack! legs 50 arms

38. Patrol Goal Attack Goal Dodge Goal Patrol is executing 40 legs Enemy spotted! Sensor wakes up AttackGoal arms X 40 50 Attack executes; posts subgoals for legs & arms legs 50 arms Projectile spotted! Sensor wakes up DodgeGoal X X 40 50 90 Dodge executes; needs legs; arms continue executing Attack! legs 50 arms X Dodge finishes: legs resume Attack subgoal 40 50 legs 50 arms

39. Seeing the Big Picture • Task memory: An AI’s goal set constitutes the “big picture” of tasks to be accomplished • Failed or interrupted goals hang around • Cognitive memory: actions potentially contain lots of stored data • High level actions have long life span; their data sticks around for that time, too

40. Seeing the Big Picture • Explicit goal representation:AI can reason about remaining tasks • Behaviors can be chosen that achieve multiple goals • Separate rules decide when to remove unachieved goals

41. Part 4: Collaboration

42. Why Collaborate? • Increases productivity • Both working on FPS’s starting at about the same time using the same engine (Unreal) • Development teams based in Boston (SWAT4) and Canberra, Australia (Tribes) • Teams shared code for rendering, networking, and more • Tyrion was an already tested system

43. How to Work Together? • Communication • Shared Codebase • Perforce • Programming (of course!)

44. Communication • Initial technical design document was written in Australia, then discussed via e-mail between offices for weeks • E-mail response time was usually a day (due to time difference) • More complicated work could be accomplished over the phone • An in-person visit helped SWAT4 make greater use of Tyrion’s features

45. Communication: Lessons Learned • We underestimated the amount of time required for effective communication • During design phase, both of us spent about 1/3rd of our time communicating – less later on • We underestimated the amount of time required for “tech support” • Collaboration allowed both teams to identify weaknesses in Tyrion that we would discuss and address

46. Shared Codebase • Shared code between offices to speed development of both Tribes & SWAT4 • One shared codebase, which both projects could integrate code from or to

47. Perforce • Very useful program, allowed for easy collaboration between offices • We didn't have to worry about sharing the code, we got to worry about the code itself!

48. Programming • Marc did most of the implementation • John's prior experience in Unreal helped optimize and streamline the code • We both felt comfortable editing the code in the shared codebase for our own needs • Code today is clean and well commented, although it does contain a few#if IG_SWAT and #if IG_TRIBES3 preprocessor definitions!

49. Programming: Lessons Learned • Would have been nice to have shared a common AI base class • Both games exposed weaknesses that were addressed • Early in SWAT, the idle goal needed to be satisfied by two subclasses of the base Idle Action class • Change was made to system to allow subclasses of actions to satisfy goals • Claiming leaf (motor control) resources issue

50. Programming: Lessons Learned • Inheritance vs Building Blocks • Building blocks: new behaviors are implemented by combining simpler ones • Inheritance: Change an existing behavior by sub-classing  Tyrion didn’t intend for this • Not everyone will use your engine the way you intended • Having multiple ways of accomplishing tasks allows clients to create and innovate • Engine’s strength determined by client’s effective use