Multi-Agent Coalition Formation for Long-Term Task or Mobile Network

1. Multi-Agent Coalition Formation for Long-Term Task or Mobile Network Hsiu-Hui Lee and Chung-Hsien Chen

2. Proposal • Propose a new architecture which integrates case-based reasoning, negotiation, and reinforcement learning to improve the coalition formation process • Suit for executing long-term task or for accomplishing a task in high mobility networks

3. Ubiquitous and mobile networks • Ubiquitous networking system was designed by A group of researchers at AT&T Laboratories Cambridge • Several devices that have network capability communicate each other to achieve a common goal • E.g. locate a person at a building, connection to a personal computer via several devisors

4. Notations

5. Problem

6. Case-based reasoning • Use to obtain the past coalition case • Fuzzy match mechanism -> find similar task in the past • If similar case found: • Sending looking up request to peer agents who are belong to the solution set in the past • If found resources: negotiate • If not enough resources found among peers • Broadcast requests to search agents who have resources

7. Leaving Rate • The leaving rate of peer agents indicates the probability that peer agents disappear

8. Negotiation • Processes in continuous rounds • Each round: the agent makes a proposal and send it to the peer agents • The peer agent checks the proposal whether it can be accepted or not • Strategies: • Linear strategy: dropping to its limitation steadily • Tough strategy: dropping to its limitation immediately when deadline approaches

9. Negotiation… • The linear strategy -> low leaving rate agents • Increases the successful probability of negotiating • Tough strategy -> high leaving rate agents • More agents with low leaving rate and lesser agents with high leaving rate

10. Negotiation… • About rewards • Closer to the idle value -> higher probability to agree • Partially formed coalition doesn’t has enough resources, but the system has • Agent leave an acting coalition -> fail execute task

11. Reinforcement learning • machine learning mechanism • An agent perceives the current state to takes action • Agent collect experience for better coalition formation • For a given goal the computer learns how to achieve the goal by trial-and-error • They don’t use this method

12. Temporal difference learning • Learning rate – higher, more experience • Remove reward made by uncertain agents • Similar task in the past