Building Reactive Characters for Dynamic Gaming Environments

1. Building Reactive Characters for Dynamic Gaming Environments Peter Blackburn and Barry O’Sullivan, IEEE Symposium on Computational Intelligence and Games, 2005. . Presenter : Wei-Shen Tai Advisor : Professor Chung-Chian Hsu 2006/2/15

2. Outline • Introduction • Background • Decision Tree Learning • Integration with Tournament • Evaluation • Conclusions • Comments

3. Motivation • Past “first person shooter” games • Typically provide human players with opponents that simply acted as “cannon fodder” . • Little effort was placed in developing complex behaviours for them. • Intelligent opponents • human-like decision making capabilities. • sophisticated AI is a factor in ensuring a high replay rate for games.

4. Objective • Purely reactive non-player characters (NPC) • Enables developers to incorporate AI in games with minimal development overhead. • Building game opponents in this way yields natural reactive behaviour.

5. Method • Traditional finite-state machine (FSM) architecture • the various states and the transitions from one state to another. • Decision tree • induced from the strategies that human players would employ as a basis for defining the conditions under which each state is reached. • defines the complex relationships between each state giving rise to behaviour by re-evaluating the state (classification) suggested.

6. Link to Unreal Tournament • GameBot • A Programming environment for controlling NPCs. • the client-server architecture developed at the University Of Southern California’s information Science Institute. • JavaBot • Built on top of the GameBot infrastructure and control NPCs easily without having to worry about the underlying network protocols.

7. Evaluation and results • Evaluation way • using decision trees compete against a collection of nine other standard Unreal Tournament NPCs in a death-match context. • Competitive NPC • consistently performing better than both the standard novice and average skilled characters.

8. Introduction • “first person shooter” • Opponents (NPCs) that simply acted as “cannon fodder” . • game players have grown bored due to the lack of sophistication of the opponents. • AI (Artificial Intelligent) • NPCs can evade attacks and plan retreats in order to obtain support or to find a better tactical vantage point.

9. Sophisticated AI • Intelligent opponents • display human-like decision making capabilities. • Purely reactive character • that has skills based upon the strategies a human player would use.

10. FSM and DT • Traditional finite-state machine (FSM) architecture • separating the various states and the conditions that determine the transitions from one state to another. • Decision tree (DT) • induced from the strategies that human players would employ in various gaming scenarios. • defines the complex relationships between each state giving rise to behaviour by re-evaluating the state (classification) suggested.

11. Contributions • Proposed method • building complex, but realistic,behaviour into game characters through theuse of decision trees induced from human strategies; • Cost • reduces the softwaredevelopment burden required to build complex behavioursinto computer games.

12. Past works • “behavioural cloning” • Capture human sub-cognitive skills and incorporate them into a computer program. • The learning program produces a set of rules that can reproduce the skilled behaviour. • Artificial Neural Networks (ANN) • could be used to train an NPC to learn a set of basic commands from an experienced player.

13. DT advantages • Common idea in last two techniques • Training an NPC by monitoring actual game play over a long period of time. • Decision tree • building competitive NPCs based on human-provided responses to a number of gaming scenarios. • with significantly lower training data requirements.

14. Decision tree learning • Approximating discrete-valued functions • make classifications by sorting the features of an instance through the tree from the root to some leaf node. • Constructed in a top-down fashion • by adding a node that performs a test on a feature of the problem that best classifies the set of training examples.

15. DT example Case 1

16. Integration with Unreal Tournament • Objective in the death match game • Several characters compete to achieve a target number of “frags” within a specified time-frame (typically 30 minutes). • A frag is a score computed as the number of opponents that are eliminated by the character.

17. Architecture for Controlling an NPC • GameBot • controlling our NPCs that was first developed at the University Of Southern California’s Information Science Institute. • server-side • provides sensory information about the environment . • For example, the location of other opponents that are currently visible, the positions of obstacles. • client-side • provides the capability to send messages to the server, • thus specifying what action an NPC should take, for example move, turn, shoot, etc.

18. Using DT to control NPCs • Typically JavaBot • implements a simple state changing method to control an NPC. • Decision tree-based architecture • built simple independent states that would be triggered by classifications obtained from the decision tree. These states provide very basic behaviours. • provide our NPC with human-like behaviour and skill.

19. Acquiring strategies from a human player • Captured approach from skilled human player • a form of user interview by creating a questionnaire-based system. • The human selects one of several behavioural actions in response to each scenario.

20. Inducing the decision tree • Training set • the human user response forms a training set from which a decision tree can be induced.

21. Exploiting the Decision Tree • Guide of NPCs • using the decision tree to guide the behaviour of an NPC during the game. • Categorical value converting • Since decision trees do not work on numerical values directly, a preprocessor is required to discretise this data. • For example, for health points a value of 0-40 might be considered ”Low”, 41-75 might be ”Medium” and greater than 75 might be considered to be ”High”. • these values usually need to be considered carefully for each particular game that this technique is being applied too. • “refresh” time period • can also affect the NPC’s performance quite dramatically. • this time again will vary depending on the game context to which this technique is being applied too.

22. Evaluation (1/2) • Environment • a character built using decision trees compete against a collection of nine other standard Unreal Tournament NPCs in a death-match context. • Conditions • varied the number of questions used to build the decision trees used in each case, (5 DTs) • selecting values of 10, 50, 75 and 100, giving us different sized training sets. • varied the skill level of the opponent characters, comparing against both novice and average skill levels.

23. Evaluation (2/2) • Other conditions • Built 5 different decision trees for each training set size and used each in 50 game sessions. • Each game session had a maximum time limit of 30 minutes and a target frag count of 20. • For each game we recorded the score and position of the decision tree-driven character.

24. Evaluation results • Very competitive NPC • consistently performing better than both the standard novice and average skilled characters in Unreal Tournament. • our character is, on the average, placed in the top 3 positions in any configuration.

25. Performance against novice skilled characters

26. Performance against novice average characters

27. Discussion • More sample size, less performance? • The objective is to eliminate as many opponents as possible. • tradeoff survival and the risks in the centre of the battle to be killed or killing another. • induction process with feedback about the performance of the resultant tree. • Skilled opponents make less performance • a slight degradation in performance as we encounter more skilled opponents.

28. Conclusions (1/2) • DT • regards the choice of action that an NPC makes as a classification problem. • easy to build and apply • link these states together in a way that we have shown results in competitive games characters. • save significant amounts of debugging • only building a number of implementations of simple actions that a game character needs to perform.

29. Conclusions (2/2) • Character “personality” • A developer can have some control over the personality that a character can have. • This can be assisted by providing different responses to the scenarios presented on the questionnaire described earlier. • Controlling a team of characters. • The key idea is to build a “commander” decision tree which would determine how they behave as a group. • For example, the team’s task might be to “capture the flag” of an opponent at a specified location. The commander tree initially informs the characters that they must capture the flag (their goal). • Visualization tool and development environment • Developers can use to build and modify decision tree based NPCs.

30. Comments • Fuzzy decision tree • It is feasible to convert numerical data into linguistic terms. • Acquiring strategies from real battle field • Replaces the form of user interview. • It is apt to acquire the respondence from skilled players with various situations in real Unreal Tournament. • Evaluating through real skilled players • Real-human with basic skilled should be better than NPCs.