Team Member AI in a FPS

1. Team Member AI in a FPS AI Game Programming Wisdom 2 Chapter 3.3

2. Goals • Teammates that back the player up • Teammates that follow instructions • Teammates that survive until end of game • Don’t want: teammates that get in the player’s way or wander off

3. Correct Positioning • Player’s F.O.V. should not be obstructed • Balance between various aspects: • Visible teammates add to feeling of the game • Consider accuracy of the player’s weapon • Should always leave area between player and targets clear

4. Balancing NPC Movement • NPC movement while in player’s F.O.V. should be smooth so as to not detract from game play • Should move to clear player’s F.O.V., but should move as little as possible • Another consideration: stance, could the teammates crouch or crawl past the player? (Again, take into account accuracy of player’s weapon, and lines of fire)

5. Correct Movement • Multiple targets means a greater chance that at least one line of sight is obstructed • Moving out of the way of one target could put a teammate in front of another target • Solutions: • Consider all targets & find the smaller angle that would leave a clear view for player • Consider the player’s heading and move away from it

6. Correct Movement E E T P E E

7. Correct Movement E E T P E E May result in a longer path for teammate, but ensures the teammate will not cross player’s line of sight

8. New Considerations • Target Threat: • Proximity • Level of awareness • Readiness • AI must be able to act to keep the target with the highest threat clear Ex. Teammate might spot a patrol 500m away while there is an enemy just 10m away Full Spectrum Warrior!

9. Guidelines • Maximum straying distance: don’t want the player to have to chase teammates • AI should not take the fun from the player (e.g. Should not be the first around corners, through doors) • If ordered to take the initiative, teammates should behave strategically • Should move at same speed as player but lag behind slightly

10. Guidelines Remove? • Context: teammates should always respect the context and be able to intelligently change between contexts. Ex. In a stealth game teammates would avoid firing on targets unless they were instructed to do so, or unless they were engaged by the target • Can set context for a individual teammates or entire team

11. Correct Behavior • NPC behavior can be defined in rule based terms, focusing on strategic aspects of individual and the entire team • 3 behaviors common to FPSs • Use of cover • Selective firing • Reloading

12. Correct Behavior – Use of Cover Remove? • Use of cover keeps the teammates alive as long as the player Should favor use of cover in path finding algorithms • When entering a new area, teammates should map out possible points of cover • Must take into consideration maximum straying distance

13. Correct Behavior – Selective Firing Remove? • Should only fire when target is in range and in view • Realistic games: fire in short bursts to improve accuracy • Balance accuracy with response time • Low response time could result in AI that appears unaware and less intelligent • Response time should be >= enemy response time to ensure the team survives more often

14. Correct Behavior - Reloading Remove? • Teammates should count the number of rounds used & find safe cover to reload • Staggering firing of teammates ensure only one will need to reload at a given time • While one teammate is reloading, another should be able to keep up the attack

15. Supporting the Player • At least one teammate should cover player’s flanks • Report enemy sightings so player can take action and to help guide player through game • If enemy sighted and team too busy, teammate should engage autonomously • Should concentrate on targets other than the player’s target • Should take into consideration threat level • Might chose target that is farthest from player’s line of sight

16. Table of Actions • A table of actions can help the team function more effectively and give a greater illusion of intelligence

17. The Player is Most Important • Player should be allowed to play the game, supported by the team – team should never hinder game play • NPCs should chose weapon according to player’s weapon • NPCs should defer objects to player • If an NPC does get an object, most needy one should have first choice • NPCs should have different reaction times for distant and near enemies, to allow the player to respond first

18. Rainbow Six Demo

19. Implementation • Layered approach • Four layers • Lower layers take higher priority • Caters to player to improve game play Team Manager Environment Threat Player

20. Implementation – Player Awareness • Primary concern is to avoid player’s line of sight • Player’s position, orientation, current target, and possible targets should all be taken into consideration • Secondary concern is with avoiding other NPCs’ lines of sight • A hierarchical ordering of NPCs solves this

21. Implementation – Threat Awareness • NPCs should be looking for targets using line of sight algorithms • Enemies sighted should be reported • Movement gets deferred to player awareness level when not searching or engaging enemies

22. Implementation - Environment Awareness • When moving into a new area, teamates should identify cover and areas that are impermissible due to danger or exposure • Environment can be used to reduce NPC straying – boundary restrictions

23. Implementation - Team Manager • Shared by all teammates • Contains all the team’s data • Helps coordinate the whole team • NPCs query manager to find out where and how to move (sets context) • Tell NPCs if they can pick up objects • Consults environ. level to get stray dist. • Consults threat level to control rate of fire • Never consider the player (unpredictable)

24. Teammate Availability N = # enemies in covering area O = # enemies in range P = # enemies threatening team Q = supporting another team member (1/0) (Multiply by infinity to make sure busy teammate is never selected) Availability = (1+N) (1+O) (1+P) + (Q*Inf.) Lower numbers indicate greater availability. Set thresholds to determine if a teammate is busy

25. Fire Staggering • Should happen naturally • Teammates should not target and fire at the same enemy • Team manager should monitor ammo levels • Could widen cover area of one NPC to reduce another NPC’s cover area, allowing them to conserve ammo • When one NPC runs out the other can then cover the whole area while the former reloads

26. Applying Goal-Oriented Action Planning to GamesAutonomous decisions to activate behaviors intended to accomplish most relative goals AI Game Programming Wisdom 3.4

27. FEAR! Goal Oriented Action Planning (GOAP) • A decision making architecture that allows NPCs to decide what to do next, and how to do it • Produces less repetitive, less predictable behavior • Can be adapted to fit the current situation • Facilitates authoring, maintaining, and reuse of behaviors • Active goal determines NPC’s behavior through a hard-coded sequence of state transitions

28. GOAP • An agent uses a planner to formulate a sequence of actions that will satisfy some goal • GOAP does not replace the need for FSMs, but greatly simplifies those required • Plan becomes a sequence of actions where each action represents a state transition • Separating the state transition logic from states greatly simplifies the underlying FSM

29. GOAP - Goals • A goal is any condition that an agent wants to satisfy • Agent may have any number of goals • Only one goal is active at any given moment • A goal knows how to calculate its relevance • A goal knows when it has been satisfied • A goal does not include a plan, but merely conditions that must be met

30. GOAP - Plan • A plan satisfying a goal is any valid sequence of actions that move the agent from some starting state to some a state satisfying the goal • Could be multiple plans that satisfy some goal • Only need to find one • The planner can be given hints Ex. Associate costs with actions, look for lowest cost to reach goal

31. GOAP - Action • An action is a single, atomic step that makes an agent do something Ex. GoToPoint, ActivateObject • Could be short or infinitely long Ex. Reload, Attack • An action knows its preconditions and effects • Allows for easy chaining of actions Ex. Reload and Attack

32. GOAP – Plan Formulation • An agent provides a goal to a handler called a planner • The planner searches the space of actions for a sequence that will take the agent from its starting state to a goal state (formulating the plan) • If planner is successful, the agent follows the plan until completion or until another goal becomes more relevant • If another goal activates or plan becomes invalid, agent aborts and formulates another plan

33. GOAP – Plan Formulation Kill Enemy Goal Action Space Sleep Start Idle Draw Weapon Dodge Reload Weapon Go to Cover Attack Activate Object Go to Point Goal State

34. GOAP – Runtime Benefits • An agent can custom fit actions to current surroundings • Agent can dynamically find alternate solutions to problems • Ex. Soldier out of ammo, but laser near by

35. GOAP – Development Benefits • Handling every possible situation for every agent can quickly become difficult to manage • Adding design requirements in GOAP is simply a matter of adding actions and defining their preconditions and effects Ex. Adding TurnLightsOn action, precondition to GoTo action • GOAP helps prevent invalid plans to be formed, where human error could easily contribute to invalid plans in hand-written code

36. GOAP – Variety Benefits • GOAP can be used to create a variety of characters simply by giving the planner different subsets of actions • One character’s actions can easily be replaced with functionally similar, but aesthetically different actions for another character Ex. OpenDoor for regular soldier and brute • Extension: add preconditions to certain actions for mood, energy level, etc.

37. GOAP – Implementation Considerations • Challenges: • Determining best method for search the space of actions • Planner must be able to represent the game world in a clear, compact manner

38. GOAP – Planner Search • Similar to path finding, so the same algorithm can be used: A* • The planner need only implement its own classes for A*’s nodes, map, and goals • A* involves the calculation of the cost of a node and the heuristic distance from a node to a goal • Can search forward or backwards • Forward: exhaustive brute force search • Backwards: more efficient and intuitive

39. GOAP – Planner Search • Node Cost Calculation: sum of costs of actions that take the world to the state represented by the node • A lower cost is preferable • Hueristic Cost Calculation: sum of the number of unsatisfied properties of the goal state

40. GOAP – World Representation • Planner must be able to apply preconditions and effects of action to realize when it has reached the goal state • Only need to represent minimal number of properties of the world state that are relevant to the goal the planner is trying to satisfy • As planner adds actions, goal state grows with the preconditions of actions added

41. GOAP – World Representation • Can construct a list of world property structures for each action taken • Enumerated attribute key • A value • A handle to a subject (the character) • Pass desired properties to the goal state

42. GOAP – World Representation Remove? struct SWorldProperty { GAME_OBJECT_ID hSubjectID; WORLD_PROP_KEY eKey; union value { bool bValue; float fValue; int nValue; } } Ex. KillEnemy goal SWorldProperty Prop; Prop.hSubjectID = hShooterID; Prop.eKey = kTargetIsDead; Prop.bValue = true;

43. GOAP – Final Thought • Context preconditions – something that needs to be true for an action to take place, but the planner will never try to satisfy it Ex. Attack action, must be in range and F.O.V. • Planner must run two functions, one to make a plan and check all preconditions and another to verify all context preconditions are satisfied for the plan

44. GOAP - Optimization • Possible optimizations for GOAP: • Optimizing the search algorithm (which takes the general form of A* but is game specific) • Cache results from previous searches • Distributed plan formulation over several updates • Use of context preconditions to prune search trees.