Evolving Agents in a Hostile Environment

Evolving Agents in a Hostile Environment Alex J. Berry

Overview • Motivation • Background • The Approach • Map Generation • Evolutionary Algorithm • Experiments

Training First Responders • VEnOM Labs is developing a suite to train First Responders • Is the training effective? • How can we make the training more effective?

Goal • To develop a system to allow for friendly and hostile AI agents in the training environment. • To develop a system to create better agents for training First Responders.

Simulation of Adaptive Agents in a Hostile Environment[HW95] • Thomas Haynes • Simple Agents • Mines and Energy • Experiments • Single Agent, Static and Random Environment • Multiple Agent, Static and Random Environment

The Complicator • Aliases: Dr. T, Dr. Tauritz • Input: 2+2=4 • Output:

The Approach • Randomly Generated Grid Environment • Three Types of Agents: • First Responders • Terrorists • Victims • Genetic Programming to Evolve the Agents

Randomly Generated Maps • Any Dimension • Percentage walls • Bit Array to Hold the Data

Demo

Victims Move Randomly Remember Things Forget Things Survive Terrorists Kill Victims Kill First Responders Lay Traps Disguise Themselves Not Get Caught First Responders Help Victims Find and Disarm Traps Survive Catch Terrorists What’s an Agent to do?

Evolutionary Algorithm • Two Agents to Evolve • First Responder • Terrorist • Two Competing Evolving Populations • Genetic Programming for the Evolutionary Implementation

Terminals Current Grid Location (C) Surrounding Grid Locations (S) Rand (R) Non-Terminals If-Then-Else Threat And, Or, Not Victim, First Responder, Terrorist, Trap Valid Move Actions Save Kill Move Place Trap Remove Trap Not (Action) to invert an Action What An Individual Looks Like

Sample Individuals Move

First Responder Victims Helped Terrorists Caught Traps Found Traps Removed Survival Amount of the Map explored Terrorist Kills using Traps Kills on Contact Effective Disguises Amount of Time Survived Genetic Programming Evaluation

Experiments • Static Environment Evolution • Random Environment Evolution • Varying Ratios of First Responders, Victims, and Terrorists • Evolving one Population at a Time

Summary • Looking at agents operating in a hostile environment. • First Responders, Terrorists, and Victims • Evolving first responders and terrorist using genetic programming techniques.

Future Work and Questions • Other Evolutionary Approaches • LCS • GP-LCS Hybrid • Integration into a 3D environment • Playable Human Mode • Representations of Real Buildings • Test effectiveness of adding this to Affective Intensity Experiment • Integration of other types of Traps and sensors to detect those traps

Evolving Agents in a Hostile Environment