Software Agent - MAS: multi-agent systems-

1. Software Agent- MAS: multi-agent systems-

2. Outline • Definition • Issues and elements of MAS • MAS architectures • Coordination • Collaboration • Several issues in designing competitive MAS • Applications • MAS research direction • Summary

3. Multi-agent Systems • MAS as seen from distributed AI • A loosely coupled network of entities that work together to find answers to problems that are beyond the individual capabilities or knowledge of each entity • A more general meaning • systems composed of autonomous components that exhibit the following characteristics: • each agent has incomplete capabilities to solve a problem • there is no global system control • data is decentralized • computation is asynchronous • A multi-agent system contains a number of agents… • …which interact through communication… • …are able to act in an environment… • …have different “spheres of influence” (which may coincide)… • …will be linked by other (organizational) relationships

4. Overview of MAS • Aspectsof multi-agent systems • Cooperative vs. competitive • Homogeneous vs. heterogeneous • Macro vs. micro • Interaction protocols and languages • Organizational structure • Mechanism design / market economics • Learning • Types of MAS • Cooperative MAS • Distributed problem solving: Less autonomy • Distributed planning: Models for cooperation and teamwork • Typical (cooperative) MAS domains • Distributed sensor network establishment • Distributed vehicle monitoring • Distributed delivery • Competitive or self-interested MAS • Distributed rationality: Voting, auctions • Negotiation: Contract nets

5. Intelligent Agents Intelligent Agents Intelligent Agents Intelligent Agents Intelligent Agents Intelligent Agents Intelligent Agents Intelligent Agents Intelligent Agents Comparison with Traditional Approaches Traditional Software Client Server Function(Parameters) Return(Parameters) Agents Blackboard Message Reply A14<MAS>-4 • Traditional • Client-server • Low-level messages • Synchronous • Can not do the job! • Agent breakthroughs • Peer-to-peer topology • Blackboard coordination model • Encapsulated messaging • High-level message protocols

6. Main Points in MAS MAS researchers develop communications languages, interaction protocols, and agent architectures that facilitate the development of multi-agent systems MAS researcher can tell you how to program each ant in a colony in order to get them all to bring food to the nest in the most efficient manner, or how to set up rules so that a group of selfish agents will work together to accomplish a given task MAS researchers draw on ideas from many disciplines outside of AI, including biology, sociology, economics, organization and management science, complex systems, and philosophy

7. Key Elements of MAS • A coordination mechanism supported by a common agent communication language and protocol • A collaboration mechanism supported by agent community architecture (including agent and interaction architecture) to support the organization goal • A shared ontology • Popular MAS architectures • Object Manager Group (OMG) • Foundation for Intelligent Physical Agents (FIPA) • Knowledgeable Agent-oriented System (KAoS) • Open Agent Architecture (OAA) • General Magic group

8. MAS Architectures (1) • OMG’s Model • Composed of agents and agencies that collaborate using general patterns and policies • Agents are characterized by: capabilities, type of interaction and mobility • Agencies support: • concurrent execution of agents • security • agent mobility • FIPA’s Model • Agents • Agent Platform (AP) • Directory Facilitator (DF) • Agent Management System (AMS) • Agent Communication Channel (ACC) • Agent Communication Language (ACL)

9. MAS Architectures (2) • KAoS’s Model • An Open Distributed Architecture for Software agents • Defines various agent implementations • Uses conversation policies to elaborate on agent-to-agent communication • OAA Model

10. MAS Architectures (3) • General Magic’s Model • A commercial agent technology for electronic commerce • Views MAS as an electronic marketplace • The marketplace is modeled as a network of computers supporting a collection of places that offer services to mobile agents • The mobile agents: • can travel, meet other agents, create connections to other places • they have authority • Zeus: a MAS development toolkit

11. MAS Architectures (4) Other Agent Systems User Task(GeoScript) Reply Query agent Geo-Agents Administrator UI Agent Query agent Exchange registry Pass task Reply Query agent Facilitator Query agent Coordinate Coordinate Task Agent Domain (Service) Agent Control/Reply Domain (Service)Agent Task Agent Retrieve Collaborate Collaborate Data sources Geo-Agents (GIS agents) Architecture

12. Coordination • Coordination: a process to manage dependencies among activities • Three aspects of coordination • Activity aspect • What activity to execute? • When an activity should be executed? • Model to coordinate distributed tasks: Statecharts, Flowcharts, Process algebra, Lotos, SDL, Estelle … • Conversation (state) aspect • What is the structure of the conversation among the coordinating entities? • FSM, Petri-Nets, State Transition Diagrams • Implementation aspect • How to implement distributed software systems where software components coordinate their actions

13. Coordination KQML • (ask-all /* message layer */ • :content "price(IBM, [?price, ?time])“ • /* content layer */ • :receiver stock-server • /* communication layer */ • :languagestandard_prolog • :ontology NYSE-TICKS • :sender me) • Knowledge Query and Manipulation Language (KQML) is both a message format and a message-handling protocol to support run-time knowledge sharing among agents • KQML comprise a substrate on which to develop higher-level models of inter-agent interaction such as contract nets • KQML is a coordination mechanism from the conversation aspect • KQML contains an extensible set of performatives, which defines the permissible speech acts agents may use • Example performative:

14. Coordination KQML: Types of Performatives Basic informative performatives: tell, deny, … Database performatives: insert, delete, … Basic responses: error, sorry, … Basic query performatives: ask-one, ask-all, evaluate,… Multi-response query performatives: stream-all, … Basic effectorperformatives: achieve, … Generator performatives: standby, ready, next, … Capability-definition performatives: advertise Notification performatives: subscribe Networking performatives: register, forward, pipe, broadcast, … Facilitation performatives: broker-one (all), recommend-one (all), recruit-one (all)

15. agent agent Bid Contract Collaboration • Collaboration refers to cooperative effort among agents to reach a single goal by exchanging knowledge built upon the underlying coordination mechanism • Example mechanism: Contract Net Protocol (CNP) • Negotiation as a collaboration mechanism • Negotiation on how tasks should be shared • A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning) • An agent may subcontract another agent to perform a (sub)task.

16. Collaboration Phase 1: Task Announcement - The contractor agent publicly announces a task. - Potential candidates evaluate the task according to their won skills and availability. Phase 2: Submission of Bids / Proposals - Agents that satisfy the requiremenst, i.e., are able to perform the task, send their bid / proposal to the contractor.

17. Collaboration Phase 3: Selection - The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates. • Phase 4: Contract awarding • A contract is established between the contractor and the selected candidate. • - A privileged bilateral communication channel is established between the two agents.

18. Several Issues in Designing Competitive MAS Distributed rationality Pareto optimality Stability

19. Competitive MAS Distributed Rationality • Techniques to encourage/coax/force self-interested agents to play fairly in the sandbox • Voting: Everybody’s opinion counts (but how much?) • Auctions: Everybody gets a chance to earn value (but how to do it fairly?) • Contract nets: Work goes to the highest bidder • Issues: • Global utility • Fairness • Stability • Cheating and lying

20. Competitive MAS Pareto Optimality Which solutions are Pareto-optimal? Y’s utility Which solutions maximize global utility (social welfare)? X’s utility • S is a Pareto-optimal solution iff •  S’ ( x Ux(S’) > Ux(S) →  y Uy(S’) < Uy(S)) • i.e., if X is better off in S’, then some Y must be worse off • Social welfare, or global utility, is the sum of all agents’ utility • If S maximizes social welfare, it is also Pareto-optimal (but not vice versa)

21. Competitive MAS Stability • If an agent can always maximize its utility with a particular strategy (regardless of other agents’ behavior) then that strategy is dominant • A set of agent strategies is in Nash equilibrium if each agent’s strategy Si is locally optimal, given the other agents’ strategies • No agent has an incentive to change strategies • Hence this set of strategies is locally stable • Prisoner’s dilemma • Pareto-optimal and social welfare maximizing solution: Both agents cooperate • Dominant strategy and Nash equilibrium: Both agents defect

22. Development of MAS • Define the organization of the MAS according to the problem specification (or solution structure) • Decide the coordination mechanism • Select a MAS implementation framework, e.g., Zeus, that supports the coordination mechanism • Implement the collaborative mechanism which support the MAS organization • Implement shared ontology • Implement each task agent (including customizing associated communication module) • Customize middle agents • Facilitators • Mediators • Brokers • Matchmakers and yellow pages • Blackboards

23. Applications of MAS • Advanced Manufacturing Management Systems • Agents as representatives of machines, users, business processes, etc. • Intelligent Information Search on Internet • Some agents may show learning capabilities (learn the preferences of their users, ..) • Intelligent security enforcement on Internet • Agents are representative of sensors or IDSs • Shopping Agents in Electronic Commerce • With search, price comparison, and bargaining capabilities • Multi-agent auction in E-commerce • Distributed Surveillance • For information search or to look for special events informing their users of relevant news • Distributed Signal Processing • For problem diagnosis, situation assessment, etc. in the network • Distributed Problem Solving • Collaborative design, scheduling, and planning

24. MAS Research Directions Agent Organizations Multiple (human and/or artificial) agents Goal-directed (goals may be dynamic and/or conflicting) Affects and is affected by the environment Has knowledge, culture, memories, history, and capabilities (distinct from individual agents) Legal standing is distinct from single agent Q: How are MAS organizations different from human organizations?

25. MAS Research Directions Organizational Structures • Exploit structure of task decomposition • Establish “channels of communication” among agents working on related subtasks • Organizational structure: • Defines (or describes) roles, responsibilities, and preferences • Use to identify control and communication patterns: • Who does what for whom: Where to send which task announcements/allocations • Who needs to know what: Where to send which partial or complete results

26. MAS Research Directions Communication • Communication models • Theoretical models: Speech act theory • Practical models: • Shared languages like KIF, KQML, DAML • Service models like DAML-S • Social convention protocols • Communication strategies • Connectivity (network topology) strongly influences the effectiveness of an organization • Changes in connectivity over time can impact team performance: • Move out of communication range  coordination failures • Changes in network structure  reduced (or increased) bandwidth, increased (or reduced) latency

27. MAS Research Directions Learning in MAS • Emerging field to investigate how teams of agents can learn individually and as groups • Distributed reinforcement learning • Behave as an individual, receive team feedback, and learn to individually contribute to team performance • Iteratively allocate “credit” for group performance to individual decisions • Genetic algorithms: Evolve a society of agents (survival of the fittest) • Strategy learning: In market environments, learn other agents’ strategies

28. MAS Research Directions Adaptive Organizational Dynamics • Potential for change: • Change parameters of organization over time • That is, change the structures, add/delete/move agents, … • Adaptation techniques: • Genetic algorithms • Neural networks • Heuristic search / simulated annealing • Design of new processes and procedures • Adaptation of individual agents

29. Summary • “Agent” means many different things • Different types of “multi-agent systems”: • Cooperative vs. competitive • Heterogeneous vs. homogeneous • Micro vs. macro • Lots of interesting/open research directions: • Effective cooperation strategies • “Fair” coordination strategies and protocols • Learning in MAS • Resource-limited MAS (communication, …) • Next lecture • Communication & Platform