1 / 12

Creating a Multi-Purpose First Person Shooter Bot with Reinforcement Learning

Creating a Multi-Purpose First Person Shooter Bot with Reinforcement Learning. IEEE Symposium On Computational Intelligence and Games pp. 143-150 Dec. 2008 Authors: M. McPartland and M. Gallagher Presented by Chien-Erh Huang. 1. 2. 3. 4. 5. Introduction. Background. Method. Results.

ciqala
Télécharger la présentation

Creating a Multi-Purpose First Person Shooter Bot with Reinforcement Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Creating a Multi-Purpose First Person Shooter Bot with Reinforcement Learning IEEE Symposium On Computational Intelligence and Games pp. 143-150 Dec. 2008 Authors: M. McPartland and M. Gallagher Presented by Chien-Erh Huang

  2. 1 2 3 4 5 Introduction Background Method Results Conclusions Outline CONCLUSIONS

  3. Introduction(1/2) • Bot artificial intelligence (AI) in first person shooter (FPS) games generally comprise of path planning, picking up items, and combat. • Traditionally, hard-coded methods such as state machines, rule based systems, and scripting are used for bot AI in commercial games. The problem with these methods is that they are static, can be hard to expand, and time is needed to hand tune parameters. • This paper investigates several methods based on reinforcement learning (RL) to create a multi-purpose bot AI with the behaviors of navigation, item collection and combat.

  4. Introduction(2/2) • This paper expands on previous work where low-level FPS bot controllers were trained using RL. The aim of this paper is to use the previously trained controllers and combine them to produce bots with a multi-purpose behavior set. • Three different types of RL will be compared to investigate the differences in statistics and behaviors. ● Hierarchical RL ● Rule based RL ● RL

  5. BACKGROUND(1/2) hierarchical

  6. BACKGROUND(2/2) Sarsa Q=policy e=each s=state a=action δ=variable r=reward =decay parameter =learning rate

  7. METHOD(1/2) arena map maze map • Yellow represents the Bots spawn positions, red and green represent item points.

  8. METHOD(2/2) • All three RL bots were trained for 5000 iterations in both the arena and maze environments.

  9. RESULTS(1/2)

  10. RESULTS(2/2)

  11. CONCLUSIONS • Both the arena and maze maps showed similar trends incombat and navigation statistics indicating the robustness ofthe RL controllers. • The hierarchical RL and rule based RLcontrollers performed significantly better than the flat RLcontroller in the combat and navigation skills. • Thehierarchical RL bot performed best in the shooting accuracyobjective, outperforming all other bots in the experiment. • The rule based RL bot performed slightly better in the otherobjectives than the hierarchical RL bot.

  12. Thank You !

More Related