Hierarchical Agent Control: A framework for defining agent behavior

1. Hierarchical Agent Control:A framework for defining agent behavior M.S. Atkin, G.W. King, D.L. Westbrook University of Massachusetts Presented by: Omoju Thomas

2. Agents • Viewed as resources for actions • They can be serially reusable, sharable, composite, and so on • Agent • Tank, Mechanized Infantry, Light Infantry, Cavalry, Artillery, and Aviation • Battalion and Divisions (Multi-Agent groups)

3. Sensors • Sensors tie actions to the world • They can be primitive • connecting to the world directly • Abstract • Amalgamating and processing data from multiple sources • They can be shared by actions

4. Sensors • Agents have limited sensors • They have limited knowledge of the battlefield • Agent becomes visible to another • when air reconnaissance spots it • when the agents make visual contact

5. Actions • Actions use agents and sensors • Action sometimes plans • Actions are tasks • They exist in the context of a control hierarchy • Hierarchy grounded in primitive actions • At least 9 actions listed

6. Actions • Move to a location • Occupy a location • Retain a flag • Block • Follow and assume • Follow and support • Forward passage of line • Direct attack • Indirect attack (by artillery)

7. Goals • Drives • Win the game • Capture opponents flags

8. Environment • Abstract Force Simulator • Nearly 3D • Dynamic • Discrete • Non probabilistic

9. Hierarchy Agent Control • Control (actions) hierarchy • Sensor hierarchy • Context hierarchy • Each level communicates via messages

10. Messages • Sensor sends action messages when events occur • Child actions send the parent messages on their status • Parent send children messages • Agents send their actions messages*

11. Action Hierarchy

12. Architecture • Supervenient architecture • Higher levels provide goals and context for lower levels • Lower level provide messages to higher level • Goals down, Knowledge up • Higher level cannot overrule sensory info provided by lower level and like wise

13. Control (Action) Hierarchy • Organized around tasks, not the agent • Lowest level are effectors (turn) • Higher level goals (path planning) • Actions are executed by scheduling in a queue • Queue sorted by action time execution • Scheduled actions executed by realize method

14. Control (Action) Hierarchy • Realize • Reacts to messages coming in from children • Update state • Schedule new child actions if necessary • Send messages up to parent Does not usually complete the action on first call

15. Sensor Hierarchy

16. Sensor Hierarchy • Provides a principled means for structuring the complexity of reading and transforming sensor information in AFS(Abstract Force Simulator) • Grounded in primitives • Current speed • Location of agent on map • Location of terrain features

17. Abstract Sensors • Extend the levels below it • Enemy location info combine, to specifies overall enemy presence • Terrain info combine, specifies passage corridors • Both combined to show enemy vulnerability

18. Abstract Sensors • Used to model psychological factors • Morale • Courage • Factors significantly affected by perception • If battalion believes it is isolated, morale decreases

19. Action-Selection:Context Hierarchy • Mechanism presented as an hierarchical network

20. Context Hierarchy • Actions focused only on primary goal, leads to unintelligent behavior • Scenario • If agent is moving to a destination • Agent attacked • Agent continues to move at the risk of total destruction • Too much persistence is BAD!

21. Context Hierarchy • Context needed • Agent needs a staying alive context Solution: Add conditional statement Problem: Too messy, way too much work! • Have agent satisfy sets of goals

22. Arbitration Mechanism • GRASP(General Reasoning using AbStract Physics) • Least commitment partial hierarchical planner • Conflicting goals • More than one action to satisfy a goal • Relies on a library of plan skeletons • Goals are prioritized • Minimizing resource heuristic

23. Action-Selection contd • Small set of plan combination generated • Plan Evaluation • Simulating each plan • Pick the one with the most favorable future world state • If a plan cannot be completed • Resource unavailable (Agent killed) • The plan is reassigned to another agent • Total re-plan takes place

24. Who decides what next? • GRASP • The planner controls the agents • It initiates actions (goals) using agents as the resource to carry out these goals • To a degree the agents are autonomous • Once they have been assigned their goal, they take it from there • Human player

25. Conclusion • Agent control, planning and sensing all part of same framework • Modular system, supervenience allows for reuse of codelets • Supervenience allows for each level in hierarchy to be dealt with individually • TASKS, NOT THE AGENTS ARE IMPORTANT • Agent just a resource to carry out task

26. References M.S.Atkin, G.W.King, and D.L.Westbrook. Hierarchical agent control: a framework for defining agent behavior. In Proceedings of the Fifth International Conference on Autonomous Agents. Autonomous Agents 2000. G.W.King, M.S.Atkin, and D.L.Westbrook. Tapir:the Evolution of an Agent Control Language