Workshop on High Performance, Fault-Adaptive Large Scale Real-Time Systems Vanderbilt University

1. Workshop on High Performance, Fault-Adaptive Large Scale Real-Time SystemsVanderbilt University The SRTA Agent Architecture as a Basis for Building Soft Real-Time Multi-Agent Systems Victor R. Lesser Computer Science Department University of Massachusetts, Amherst November 15, 2002

2. Acknowledgements • Byran Horling • Dr. Regis Vincent (SRI) • Dr. Tom Wagner (Honeywell Research) • URL:http://mas.cs.umass.edu/~bhorling/papers/02-14.ps.gz

3. Outline • Background/Motivation • EW Challenge Problem • Approach to Soft Real-Time • Approach to Building MAS • SRTA Agent Architecture • Experimental Evaluation • Summary

4. Long Term Motivation • Development of domain-independent techniques (toolkits) for coordinating the soft real-time activities of teams of cooperative agents? • Ease the construction of complex, multi-agent applications that operate in a coherent manner • Avoid reproducing for each application the complex reasoning involved in soft real-time control and in coordinating the activities of agents

5. DARPA EW Challenge Problem: Distributed Sensor Network • Small 2D Doppler radar units • Scan one of three 120 sectors at a time • Commodity Processor associated with each radar • Communicate short messages using radio • Triangulate radars to do tracking

6. Approach to Soft Real-Time: Design-to-Time • “Given a time bound, dynamically construct and execute a problem-solving procedure which will (probably) produce a reasonable answer with (approximately) the time available.” (D’Ambrosio) • Involves elements of planning (deciding what to do) and scheduling (deciding when to perform particular actions).

7. TÆMS: A Domain Independent Framework for Modeling Agent Activities • The top-level goals/objectives/abstract-tasks that an agent intends to achieve • One or more of the possible ways that they could be achieved • abstraction hierarchy (HTN) whose leaves are basic action instantiations, called methods • A precise, quantitative definition of the performance (Qaf’s) • solution quality, cost, time and resource usage.

8. Soft Real Time Control -- Different Paths for Achieving Task -- “BUILD PRODUCT OBJECTS” • Schedule A - Client has no resource limitations; maximize quality • Query-and-Extract-PC-Connection, Query-and-Extract-PC-Mall, Search-and-Process-ZDnet, Query-and-Process-Consumers-Report (Expected Q=55.3,C=2, D=11.5) • Schedule B - Client is interested in a free solution • Query-and-Extract-PC-Connection, Query-and-Extract-PC-Mall, Search-and-Process-Zdnet (Expected Q=33.2,C=0, D=8.4) • Schedule C - Client request an even trade-off between quality, cost and duration • Query-and-Extract-PC-Connection, Search-and-Process-Zdnet (Expected Q=22.4,C=0, D=5.6) • Schedule D - Client wishes to maximize quality while meeting a hard deadline of 7 minutes • Query-and-Extract-PC-Mall, Query-and-Process-Consumers-Report (Expected Q=25.9,C=2, D=6) Examples of Schedules Produced by the Design-To-Criteria (DTC) Scheduler

9. Representing Coordination Patterns Among Agents

10. Approach to Decomposing A Problem into Agents • Sophisticated/Highly Competent Agents • Concurrent goals, goals are time and resource sensitive, goals have varying utilities • Goals have alternative ways of being solved that produce differing levels of utility and consume differing amounts of resources • Not all goals necessarily need to be solved • (Sub)Goals spread across agents are interdependent • Contention for scarce resources • Contributing towards the solution of a higher-level goal • Hard and soft constraints • Sufficient computational/communication resources to do “some” reasoning about coordination • Medium granularity domain tasks

11. Sector Manager Tracking Manager Scanning Agent Tracking Agent What about simpler agents? • Activities of simple,single-threaded agents become the goals of sophisticated agents with dedicated processing resources • Sophisticated agents do the selection, multiplexing, scheduling, coordination and distribution of goals • Contrast with O.S. doing the scheduling without context

12. Approach to Soft, Real-Time Distributed Coordination/Resource Allocation • Structured as a distributed optimization problem with a range of “satisficing” solutions • Adaptable to available time and communication bandwidth • Responsive to dynamics of environment • Organizationally constrained — range of agents and issues are limited • Can be done at different levels of abstraction • Does not require all issues to be resolved to be successful — resource manager agents able to resolve some issues locally

13. Layered Agent Architecture Problem Solver / Negotiation Soft Real Time Architecture Java Agent Framework • Domain analysis and goal formulation • Organization-level resource allocation • Agent-level resource allocation • Constraint discovery and satisfaction • Intra-agent organization and communication • Environmental access points

14. JAF: Java Agent Architecture • Component-based agent design • Attempt to maximize code reuse. Problem Solver Execute Control Communicate Pulse Actions Scan Scheduler Execute Control Communicate Sensor State Log Resource Modeler Scheduler Directory Service Observe • Interfaces are hidden by JAF. • Radsim/ RF communication/ sensor

15. SRTA: Soft Real-Time Agent Architecture • Facilitates creation of multi-resource management agents • Basis of building complex “virtual” agent organizations • Allows for abstract negotiation — maps abstract assignment into detailed resource allocations • Ability to resolve conflicts locally that are not resolved through negotiation These are key to building soft real-time distributed allocation policies

16. Soft Real-Time Control Architecture Schedule Failure Problem solver Periodic Task Controller Negotiation (e.g. SPAM) Commitments/ Decommitments Goal Description/Objective Update Expectations TÆMS Library Schedule failure/ Abstract view Other Agents TAEMS-Plan Network/Objective Learning Update Cache Cache Check Resource Modeler Cache Hit DTC-Planner Resource Uses Linear Plan Schedule Conflict Resolution Module Partial Order Scheduler Schedule Failure Parallel Schedule Multiple Structures Results Parallel Execution Module Task Merging

17. Characteristics of Soft Real-Time Control Architecture • Operates at 50 to 100ms cycle time • Written in JAVA except for Planner in C++ • Uses domain-independent, quantitative representation of agent activities -- TÆMS • Scheduling of multiple activities that have deadlines and are resource sensitive • Can choose among alternative ways of achieving activities that trade off decreased utility for lower resource consumption • Responds to uncertain conditions without the need for complete re-planning/scheduling of activities

18. Addressing Real Time – Direct • Direct technologies - making it possible • DTC (Design-To-Criteria) planner • TÆMS HTN for representing alternative plan options using quantitative information • Create appropriate plan given time, resource costs and quality constraints • Partial order scheduling creates “loose” schedules which can be quickly shifted to real-time constraints. • Avoids constant re-planning • Allows parallel execution and resource usage. • Modeling of some meta-level activities (e.g. negotiation) permit more direct reasoning of time allocation. • We do not model scheduling costs yet. • Learning component discerns actual execution characteristics so future actions can be better modeled.

19. Addressing Real Time – Indirect • Indirect technologies - making it easier • Periodic commitments reduce the need for re-negotiation. • Scheduled caching reduces the need to call DTC • Piecemeal addition and removal of tasks eliminates the need for constant dramatic rescheduling and re-planning.

20. Scheduling • Partial-order Scheduler uses a “sliding” mechanism, coupled with a resource modeler, to quickly shift scheduled tasks. • Action start time uncertainty - real time. • Duration uncertainty. • Commitments have a “window” of time in which the agent can perform them. • Precise action scheduling is left to the discretion of the performing agent.

21. Task 1 Q_min Task Q_max Set-parameters First task to achieve enables3 enables2 Send Results Track Low Track Medium Track High lock RF Sensor Deadline: 3000

22. Enables4 Q_min Task3 Q_min Calibrate Init Task2 Negotiate-Tracking 2 Other tasks to achieve Send-Message 1 Send-tracking-Info Enables4 Enables1 lock1 RF Sensor

23. Reacting to Unexpected Changes Set-Parameters Track-Medium Send-Results Calibrate Init Send Msg Negotiate-Tracking Send-Info-Tracking time 1000 1500 2000 2500 3000 3500 4000 500 Set-Parameters Track-Medium Send-Results Calibrate Init Send Msg Negotiate-Tracking Send-Info-Tracking time 1000 1500 2000 2500 3000 3500 4000 500 Set-Parameters Track-Medium Send-Results Calibrate Init Send Msg Negotiate-Tracking Send-Info-Tracking time 1000 1500 2000 2500 3000 3500 4000 500 • Analogous reactions also take place within the periodic task controller • Slot-based scheduler used to facilitate repetitive actions

24. Meta-Level Costing • Typical scheduling reasons about primitive actions. • This only accounts for some percentage of the agent’s time. • So called meta-level activities (e.g. negotiation, scheduling, planning) use significant resources but are usually not accounted for directly. • Without accountability, these activities can interfere with the actions and commitments currently scheduled over.

25. Meta-Level Costing (cont’d) • To completely reason about all the agent’s actions, we must: • Directly and Indirectly incorporate the activities in plans. • Derive expected costs for these activities. • Use this information when generating schedules. • We are currently using a representation of negotiation activities in some our task structures • Future goal is to more directly account for activities like planning and scheduling.

26. Reducing Scheduling Overhead • Activity parallelism learning • Anticipation of converting linear plan to parallel plan if resources are available • Schedule caching • Adjustable time granularity • Responsiveness vs. meta-level overhead

27. Plan B Original Plan E q_min q_max A B C1 C2 Deadline DTC Schedule A B C1 Deadline Actual Schedule A C1 B Plan With Hints Plan B E q_min q_max A B C1 C2 F F Deadline DTC Schedule B A C2 Deadline Actual Schedule A C2 B Activity Parallelism Learning • The plan is first scheduled… • The schedule is analyzed for parallel actions • These are used to form sets of “hints,” associated with the current resource context • A hint is actually just a mutual facilitates relationship • These hints are later applied to the task structure before planning • The facilitates tells DTC that if A is run first, B will have a duration of zero (or vice versa) • This technique required no changes to DTC • This can result in better plan selection.

28. Schedule Caching • Agents in repetitive environments must frequently address goals which have previously been seen. • Ex: In sensor environment, “Scan-Sector” or “Perform-Track-Measurement” • Results from prior planning can be reused • A key is generated for each task structure • Incorporates method names and expectations, interrelationships, normalized deadlines, etc. • Works correctly with activity parallelism hints • If results exist from a task structure with the same key, that plan is used instead of calling DTC • DTC is an external C++ binary, requiring file reads and writes, so savings are significant

29. 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 15 15 16 16 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Adjustable Time Granularity • The millisecond time line is divided into coarser increments. • Does not significantly degrade the agent’s ability to meet “wall clock” time deadlines • (assuming success within a reasonable grain size) • It does decrease the number of rescheduling events which are needed • Since the agent was already operating effectively at a coarse timeline, this technique has no new drawbacks. Deadlines Deadlines Real Time Real Time 1 2 3 4 5 Perceived Time Perceived Time Actions Actions û û û û ü ü

30. Effects of Schedule Caching • Periodic tasks and methods with deadlines are more achievable • As a result, track updates increase

31. Effects of Changing Granularity Rescheduling attempts decrease, reducing overhead Methods with deadlines and periodic tasks are satisfied more often 30-40 seems ideal; above that, the coarse granularity decreases agent responsiveness

32. Summary • SRTA architecture is a powerful tool for building soft real time agent organizations • Sophisticated soft real time agent control is practical by exploiting a variety of mechanisms • to speed up the planning, scheduling and rescheduling cycles

33. Continue to speed up architecture Simple Planner for time-critical situations Additional work on conflict-resolution strategies Meta-level control component to balance control and coordination costs and domain problem solving Future SRTA Work