Agent-Based Decentralised Control of Complex Distributed Systems

1. Agent-Based Decentralised Control of Complex Distributed Systems Alex Rogers School of Electronics and Computer Science University of Southampton acr@ecs.soton.ac.uk http://users.ecs.soton.ac.uk/acr/

2. Contents • Agent-Based Decentralised Control • Cooperative Systems • Local Message Passing Algorithms • Max-sum algorithm • Graph Colouring • Wide Area Surveillance Scenario • Competitive Systems • Game Theory • Mechanism Design • Eliciting Effort in Open Information Systems • Decentralised Energy Systems

3. Electronics and Computer Science • 5* for Electrical and Electronic Engineering • 5* for Computer Science • 100 academic staff • 36 professors • 150 research fellows • 250 PhD students • Research grant income: • £15 million per annum • £10 million from UK Research Councils

4. Intelligence, Multimedia and Agents Research Group • Design and application of computing systems for complex information and knowledge processing tasks • Agent-Based Computing • Digital Libraries • Decentralised Information Systems • E-Business Technologies • Grid and Distributed Computing • Human Computer Interaction • Web Science • Knowledge Technologies • Trust and Provenance

5. Contents • Agent-Based Decentralised Control • Cooperative Systems • Local Message Passing Algorithms • Max-sum algorithm • Graph Colouring • Wide Area Surveillance Scenario • Competitive Systems • Game Theory • Mechanism Design • Eliciting Effort in Open Information Systems • Decentralised Energy Systems

6. Agent-Based Decentralised Control • Multiple conflicting goals and objectives • Discrete set of possible actions Agents

7. Agent-Based Decentralised Control • Multiple conflicting goals and objectives • Discrete set of possible actions Sensors

8. Agent-Based Decentralised Control • Multiple conflicting goals and objectives • Discrete set of possible actions • Some locality of interaction Agents

9. Agent-Based Decentralised Control • Multiple conflicting goals and objectives • Discrete set of possible actions • Some locality of interaction Maximise Social Welfare: Agents

10. Agent-Based Decentralised Control • Cooperative Systems • All agents represent a single stakeholder • We have access to these agents (closed system) • We can design the strategies that the agents adopt and the mechanisms by which they interact • Competitive Systems • Agents represent multiple stakeholders • We can not directly influence the strategies of the agents (open system) • We can only design the protocols and mechanisms by which they interact

11. Cooperative Systems No direct communication Solution scales poorly Central point of failure Decentralised control and coordination through local computation and message passing. • Speed of convergence, guarantees of optimality, communication overhead, computability Central point of control Agents

12. Landscape of Algorithms Optimality Complete Algorithms DPOP OptAPO ADOPT Message Passing Algorithms Sum-Product Algorithm Iterative Algorithms Best Response (BR) Distributed Stochastic Algorithm (DSA) Fictitious Play (FP) Greedy Heuristic Algorithms Communication Cost

13. Sum-Product Algorithm Find approximate solutions to global optimisation through local computation and message passing: A simple transformation: allows us to use the same algorithms to maximise social welfare: Factor Graph Variable nodes Function nodes

14. Graph Colouring Graph Colouring Problem Equivalent Factor Graph Agent function / utility variable / state

15. Graph Colouring Utility Function Equivalent Factor Graph

16. Max-Sum Calculations Variable to Function: Information aggregation Function to Variable: Marginal Maximisation Decision: Choose state that maximises sum of all messages

17. Graph Colouring

18. Optimality

19. Communication Cost

20. Robustness to Message Loss

21. Hardware Implementation

22. Wide Area SurveillanceScenario Dense deployment of sensors to detect pedestrian and vehicle activity within an urban environment. Unattended Ground Sensor

23. Energy Constrained Sensors Maximise event detection whilst using energy constrained sensors: • Use sense/sleep duty cycles to maximise network lifetime of maintain energy neutral operation. • Coordinate sensors with overlapping sensing fields. duty cycle time duty cycle time

24. Energy-Aware Sensor Networks

25. Future Work • Continuous action spaces • Max-sum calculations are not limited to discrete action space • Can we perform the standard max-sum operators on continuous functions in a computationally efficient manner? • Bounded Solutions • Max-sum is optimal on tree and limited proofs of convergence exist for cyclic graphs • Can we construct a tree from the original cyclic graph and calculate an lower bound on the solution quality?

26. Contents • Agent-Based Decentralised Control • Cooperative Systems • Local Message Passing Algorithms • Max-sum algorithm • Graph Colouring • Wide Area Surveillance Scenario • Competitive Systems • Game Theory • Mechanism Design • Eliciting Effort in Open Information Systems • Decentralised Energy Systems

27. Competitive Systems • Controlling open competitive systems is much more difficult • Global credit crisis • Key challenges • Understanding the emerging macroscopic properties of a system of selfish competitive agents • GAME THEORY • Designing protocols and ‘rules of the game’ such that these macroscopic properties are desirable • COMPUTATIONAL MECHANISM DESIGN

28. Game Theory • For a given ‘game’ • What action should a rational player take? • What is the equilibrium action of all players? • Nash equilibrium A Beautiful Mind: Genius and Schizophrenia in the Life of John Nash Sylvia Nasar Faber and Faber

29. Nash Equilibrium • Two strategies s1 and s2are in Nash equilibrium if: • under the assumption that agent iplays s1, agent jcan do no better than play s2; and • under the assumption that agent jplays s2, agent ican do no better than play s1. • Neither agent has any incentive to deviate from a Nash equilibrium

30. Nash Equilibrium NE 1 3 2 4

31. Computational Mechanism Design • Mechanism design concern the analysis and design of systems in which the interactions between strategic, autonomous and rational agents leads to predictable global outcomes. • Design interactions to ensure the system has desirable and predictable Nash equilibrium • Computational mechanism design • Limited communication • Incomplete information • Bounded computation

32. Nash Equilibrium NE 1 3 2 4

35. First Price Auction • Desirable properties • Efficiency • Allocation • Item assigned to the highest bidder • Payment • Pay bid ( ) • Bidding strategy • Shade bid • Bayes Nash

36. Second Price (Vickrey) Auction • Desirable properties • Efficiency • Allocation • Item assigned to the highest bidder • Payment • Pay second bid • Bidding strategy • Bid true valuation • Dominant strategy

37. Open Information System • Information buyer requires a prediction of an uncertain • Tomorrow’s temperature • Requires certain minimum precision or “certainty” • Identify cheapest provider • Make prediction of precision of at least θ0 • Truthfully report this prediction to buyer • Ensure provider’s utility is positive in expectation c(θ) θ c(θ) θ θ0 c(θ) θ

38. Two Stage Mechanism • Two stage Mechanism: • Ask information producers to declare their costs • Ask cheapest producer to make measurement and reward him with a payment using a ‘strictly proper scoring rule’ calculated from the second lowest cost • Payment is made once the event is verified • Desirable system wide properties • Dominant strategy to truthfully declare costs • Information buyer can always identify cheapest supplier • Dominant strategy to commit effort and truthfully reveal prediction

39. Challenges • Solution concepts • Mechanisms with dominant strategy solutions are rare • How do we automate the design process? • Decentralised Mechanisms • Remove need for a central auctioneer • Payment Free Mechanism • Non-transferable utility • Induce cooperative behaviour through reciprocity • Iterated Prisoner’s Dilemma • Trust and reputation models • Match making mechanisms to pair producers and buyers

40. Contents • Agent-Based Decentralised Control • Cooperative Systems • Local Message Passing Algorithms • Max-sum algorithm • Graph Colouring • Wide Area Surveillance Scenario • Competitive Systems • Game Theory • Mechanism Design • Eliciting Effort in Open Information Systems • Decentralised Energy Systems

41. 2016 Zero Carbon Home Wireless Sensors Appliances Flywheel Storage Micro-CHP Plug-in Hybrid

42. Energy Exchange

43. Research Questions • How to coordinate energy use and make optimal energy trading decisions within the home to minimise energy consumption / costs? • Load management through smart appliances • Predicting load (occupancy, activity, weather conditions) • Understanding and learning thermal characteristics of home • Price prediction in external and local markets • Optimal use of storage devices • Optimal decisions to buy electricity / use CHP

44. Research Questions • What protocols and trading mechanisms generate desirable system wide properties? • Stable, predictable and low prices • Minimise CO2 emissions through flattening demand One day

45. Publications Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices using the Max-Sum Algorithm. In: Proceedings of the Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2008), pp. 639-646, Estoril, Portugal. Papakonstantinou, A., Rogers, A., Gerding, E. and Jennings, N. (2008) A Truthful Two-Stage Mechanism for Eliciting Probabilistic Estimates with Unknown Costs. In: Proceedings of the Eighteenth European Conference on Artificial Intelligence (ECAI 2008), pp. 448-452, Patras, Greece. R. K. Dash, N. R. Jennings, and D. C. Parkes. (2003) Computational Mechanism Design: A Call to Arms. IEEE Intelligent Systems, pages 40–47.

46. Questions Thank you for your attention.