Topic 7: Collaborative Distributed Problem Solving

1. Topic 7: Collaborative Distributed Problem Solving application domain type defining a MAS cooperation strategies

2. Managing the interdependencies between the activities of agents. e.g. You and I both want to leave the room. We independently walk towards the door, which can only fit one of us. I graciously permit you to leave first. The Coordination Problem

3. http://media.funmansion.com/funmansion/player/player.php?url=http://media.funmansion.com/content/flv/d4282006/crazy_foreign_traffic.flvhttp://media.funmansion.com/funmansion/player/player.php?url=http://media.funmansion.com/content/flv/d4282006/crazy_foreign_traffic.flv

4. Understand application domain type (requirements) • application characteristics, requirements, constraints • “domain theory” • TOD Task-Oriented Domains • WOD Worth-Oriented Domains • SOD State-Oriented Domains • Define a MAS for the particular system objectives • various aspects • design issues / considerations • interaction situations • Define cooperation strategies / examples • AGV case examples • simple task allocation, task (re)allocation, jointly lifting, traffic mgt, …

5. Understand application domain type- Domain Theory • Application requirements obviously crucial for defining MAS-based software architecture • Domain theory: study of types / domains of MAS • Task Oriented Domains • Agents have tasks to achieve • Task (re)distribution • State Oriented Domains • Goals specify acceptable final states • Side effects • Joint plan and schedules • Worth Oriented Domains • Function rating states’ acceptability • Joint plan, schedules, and goal relaxation

6. 1.1 Task Oriented Domains • “Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994) • characteristics • a set of agents • a set of tasks to achieve • a set of resources • agents can achieve tasks without help or interference from each other • however, agents may benefit by sharing some tasks

7. Task-oriented Domain: Example Imagine you have 3 children, each of whom needs to be delivered to 3 different schools each morning. Your neighbour has 4 children who also need to be taken to school. Delivery of each child is a task. Assume that one of your children and one of your neighbour’s children both go to the same school. It obviously makes sense for both children to be taken together and only you or your neighbour needs to make the trip.

8. Post Office 1 2 a / / c b / / f / e d Task-oriented Domain: Example • Imagine you have 3 children, … • Post men

9. “All female employees making over $50,000 a year.” Common Database “All female employees with more than three children.” 2 1 Task-oriented Domain: Example • Imagine you have 3 children, … • Post men • Database queries

10. 1.2 State-Oriented Domain • State-Oriented Domains • actions lead to changes of system state • System objective ==> attain a particular system state either pre-defined/static, either evolving typical additional requirements (e.g. efficiency, …)

11. 1 2 State-Oriented Domain e.g. slotted blocks world e.g. furniture moving 3 1 2 3 1 2

12. 1.3 Worth-Oriented Domains • Worth Oriented Domains • actions lead to utility: function rating states’ acceptability • System objective ==> maximize utility • Joint plan, schedules, and goal relaxation

13. hole agents tile B A 2 2 5 5 2 obstacle 4 3 2 Worth-Oriented Domains • example: tile world • maximize rewards • example: traffic lights • maximize traffic throughput

14. 2. Define a MASfor particular system objectives • identify various ‘elements’ of MAS • design issues / considerations • interaction situations

15. 2. Defining a MASfor particular System objectives • identify various aspects in defining the MAS • starting fromsystem objectives / requirements / constraints • define MAS for reaching the objective(s) • identify agents • identify individual goals • identify individual skills (actions, comm, capabilities) • identify their relation to resources (insufficient, ...) • identify their cooperation • e.g. task allocation, coordination over resources, ... • identify suitable agent architecture creative task

16. 2. Defining a MASfor particular System objectives • identify various aspects in defining the MAS • design issues / considerations • within application boundaries obviously, i.e. if appl. allows • heterogeneity? • all agents can perform tasks, vs. not • splittable tasks? • specialization? • grouping? • distributed planning? • task distribution • predictiveness (anticipatory vs. in situ) • e.g. for avoiding conflicts over resources • e.g. road load, deadlocks, ...

17. 2. Defining a MASfor particular System objectives • identify various aspects in defining the MAS • design issues / considerations • interaction situations • what is an interaction ? • “dynamic relationship through a set of reciprocal actions” • i.e. stem from actions whose consequences have an influence on the future behaviour of the agents • e.g. • agents helping each other • data exchange between servers • use of a printer by two programs simultaneously

18. Types of interaction situations • components / criteria for categorization of interaction situations types • compatible and incompatible goals • relation to resources • agent capacities / skills

19. Components of interaction situations • compatible and incompatible goals • concurrent goals ? • or contradictory or even opposed ? • incompatible  antagonist situation • achieving one goal  other goal cannot be achieved • compatible  cooperation situation • …

20. resources needed by A resource space resources needed by B Components of interaction situations • … • relation to resources • resources: environmental elements which can be used in carrying out an action • objects, tools, space, time, … • limited resources  conflicts • agents needing the same tool at the same place in the same time • programs sharing a CPU • vehicles on busy roads • conflict resolution • many methods • law of the strongest • negotiation • e.g. insurances and court houses to resolve conflicts of interest due to accident • coordinating actions • devices, regulations, supplementary actions • e.g. traffic lights / highway code / … to avoid conflicts

21. Components of interaction situations • … • capacities of agents in relation to tasks • can an agent carry out a task alone ? or does it need others ? • particular tasks can be carried out either • by a single agents • e.g. moving a block, doing a calculation, reading a file, … • better by several agents / only by several agents • e.g. heavy blocks, different expertise, …  beneficial interactions result can be more than sum of parts

22. Compatible Sufficient Sufficient Independence Indifference Compatible Sufficient Insufficient Simple collaboration Compatible Insufficient Sufficient Obstruction Cooperative Compatible Insufficient Insufficient Coordinated collaboration Incompatible Sufficient Sufficient Pure individual competition Incompatible Sufficient Insufficient Pure collective competition Antagonism Incompatible Insufficient Sufficient Individual conflict over resources Incompatible Insufficient Insufficient Collective conflict over resources Types of interaction situations Resources(each has) Involvement Goals Skills Types of situation

23. Types of interaction situations:Examples • Compatible goals, insufficient resources, sufficient skills • obstruction • e.g. motorway queue • Incompatible goals, insufficient resources, sufficient skills • individual conflict over resources • e.g. only one liter of water, promotion, ... • ...

24. 3. Cooperation strategies - illustrated in AGV case • application domain type • task-oriented domain • defining a MAS • AGV agents, transport agents • individual goals • AGV agents: perform tasks • transport agents: ensure tasks are performed • skills • AGV agents: jobs/actions/communication/… • transport agents: communication/… • resources • warehouse (lanes, stock) • communication medium • cooperation • …

25. 3. Cooperation strategies - illustrated in AGV case (cont.) • … • interaction situations • (in)compatible goals • (in)sufficient skills • (in)sufficient resources

26. 3. Cooperation strategies - illustrated in AGV case (cont.) • (simple) task allocation • CNET • Gradient fields • DynCNET • task (re)distribution • negotiation • resource allocation - traffic management • CNET • collision avoidance • coordination protocol • hull projection • joint action - jointly lift a (heavy) pallet • joint intentions • batch order management • team formation • distributed planning

27. 3.1 Simple task allocation • situation a task consists of - a pick job (pick up pallet) - a move job (move towards destination) - a drop job (place pallet on target location) a task can be executed by any AGV a task needs to be assigned / allocated to an AGV characteristics “delayed commencement” dynamic communication overhead?

29. Simple task allocation (cont.) • situation • collaboration approaches • “Classic approach”: agents coordinate byexchanging messages Contract Net DynCNet • Exploit the agent environment Gradient Fields

30. Simple task allocation (cont.) • situation • collaboration approaches 1. Contract Net transport agent issues request for bids (local broadcast) if no response: stronger signal AGV agents can place bids based on current location (distance) other tasks battery level … transport agent assigns task to best bid characteristics …

31. Simple task allocation (cont.) • situation • collaboration approaches 1. Contract Net … characteristics communication overhead limited does not take dynamics/opportunities into account no rejection of assigned task no re-allocation if better-suited AGV becomes available issues how should an AGV agent deal with multiple concurrent requests? bid? value of bid? consequences? ok approach for indiviual tasks, what for large numbers of tasks? one task at a time

32. Simple task allocation (cont.) • situation • collaboration approaches 2. Dynamic Contract Net (DynCNET) contract net, but taking into account dynamics characteristics communication overhead can take dynamics/opportunities into account rejection of assigned task re-allocation if better-suited AGV becomes available issues how should an AGV agent deal with multiple concurrent requests? bid? value of bid? consequences? ok approach for indiviual tasks, what for large numbers of tasks? one task at a time

33. New Task Becomes Available

34. New AGV Becomes Available

35. DynCNET = Dynamic Contract Net

36. Simple task allocation (cont.) • situation • collaboration approaches 3. Gradient Fields coordination through the environment • physical environment • Restricts how agents can exploit the environment • agent environment • virtual environment layer • enables agents to share information and coordinate their behavior characteristics cfr. DynCNET reduced complexity of AGV agent!

37. Agent Environment for Agents

38. Exploit the Agent Environment Task Assignment (Field-Based Approach)

39. Exploit the Agent Environment Task Assignment

40. Task Allocation experiments -Test Setting AGV System • AGV system “fish handling” (real scenario) • 134 x 134 m; 56 pick/drop locations • 14 AGVs; 0.7m/s; load manipulation 5s • 140 transports/hour (standard test profile) • Tests • Communication load • Average waiting time • Stress test: perform fixed number of transports as quick as possible (e.g. handle the arrival of a truck with loads)

41. Test Results Communication Load Average Waiting Time

42. Finished Transports in Stress Test

43. Two Qualities: Flexibility and Robustness • Flexibility (adapt to changes in the environment) • FiTA: implicitly represented in the fields • DynCNET: explicit points of choice • Robustness (to message loss) • FiTA: graceful degradation • DynCNET: requires additional functionality

44. 3.2 Task (re)distribution • situation AGV agents have several tasks assigned to them a set of AGV agents could redistribute tasks amongst each other for better results e.g. after using several CNET protocols, the following assignment exists: AGV1: task 1 (NE corner), task 2 (SW corner) AGV2: task 3 (NE corner), task 4 (SW corner) redistributing the tasks could seriously enhance task execution time AGV1: task 1 (NE corner), task 3 (NE corner) AGV2: task 2 (SW corner), task 4 (SW corner)

45. Task (re)distribution (cont.) • situation • typical situation for negotiation in Task-Oriented Domains …

46. Task (re)distribution (cont.) Task-Oriented Domains (TODs) Defined • a TOD is a triple<T, Ag, c>where • Tis the (finite) set of all possible tasks • Ag ={1,…,n} is the set of participating agents • c =Ã(T) ú+ defines the costof executing each subset of tasks • an encounteris a collection of tasks<T1,…,Tn>where TiÍTfor each i ÎAg

47. Deals in TODs • given encounter <T1, T2>, a dealis an allocation of the tasks T1ÈT2 to the agents 1 and 2 • the costto i of deal d=<D1, D2> is c(Di), and will be denoted costi(d) • the utilityof deal d to agent i is:utilityi(d) = c(Ti ) – costi(d) • the conflict deal, Q, is the deal <T1, T2> consisting of the tasks originally allocated.note that utilityi(Q) = 0 for all i Î Ag • deal dis individual rationalif it weakly dominates the conflict deal

48. Negotiation • ”The process of several agents searching for an agreement” • Reaching consensus ”Rules of Encouter” by Rosenchein and Zlotskin, 1994

49. Complexity of Negotiations • Some attributes that make the negotiation process complex are: • Multiple attributes: • Single attribute (price) – symmetric scenario. • Multiple attributes – several inter-related attributes, e.g. buying a car. • The number of agents and the way they interact: • One-to-one, e.g. single buyer and single seller. • Many-to-one, e.g. multiple buyers and a single seller, auctions. • Many-to-many, e.g. multiple buyers and multiple sellers.

50. Negotiation Components • Any negotiation setting will have 4 components: • Negotiation set: represents the space of possible proposals that agents can make • Protocol: defines the legal proposals that agents can make • Collection of strategies: (one for each agent) determines what proposals the agent will make • Rule: to determine when an agreement has been reached