Decentralized vs. Centralized Economic Coordination of Resource Allocation in Grids

1. Decentralized vs. Centralized Economic Coordination of Resource Allocation in Grids T. Eymann, M. ReinickeAlbert-Ludwigs-University, Freiburg (DE) O. Ardaiz, P. Artigas, L. Díaz de Cerio, F. Freitag, R. Messeguer, L. Navarro, D. RoyoUniversitat Politècnica de Catalunya, Barcelona (ES) CATNET project – Open Research, Evaluation(3/2002-3/2003)

2. Problem and objective • Problem: Provisioning services • Requiring (huge amount of) resources • From large number of computers • CDN, Grid and P2P • Objective: evaluation of decentralized mechanism for resource allocation, based on economic paradigm: Catallaxy. • (compare against a centralized mechanism using an arbitrator object) • Methodology: simulation • Network simulator (javasim)+ application network (catnet)

3. Resource infrastructures • Content distribution networks, • Grid • Peer-to-Peer • Application networks on top, run in multiple resource locations • Example: word-processor requiring service for creation of PDF files • Client: Look for nearest/cheapest svc. Instance • Network: always provide svc, optimize provisioning costs and network communication Service control, resource allocation

4. Service control+resource allocation • Decentralized economic coordination • Price generation and negotiation • Trading resources and services • Regulation of supply and demand in large and complex systems • Catallaxy

5. Catallaxy Basics (1) • Catallaxy is an alternative word for “market economy” (Mises and Von Hayek of the Neo-austrian economic school) • “Fundamentally, in a system in which the knowledge of the relevant facts is dispersed among many people, prices can act to co-ordinate the separate actions of different people in the same way as subjective values help the individual to co-ordinate the parts of his plan.” (Friedrich A. von Hayek, The Use of Knowledge in Society, 1945) • “The Market” as a technically decentralized, distributed, dynamic coordination mechanism • Adam Smith’s “invisible hand” • Hayek’s “spontaneous order” • Walras’ “non-tâtonnement process”

6. Spontaneous order of the participants „Unplanned result of individuals' planful actions“ (Hayek) Constitutive Elements of the Catallaxy Access to a Market Knowledge about scarceness of resources is transported through price information Constitutional Ignorance Self-interest and autonomy of participants Ability to choose between alternative actions Institutions and Evolution "Institutions are frictions which, like frictions in mechanical systems, by restricting movement may make controlled movement possible.” (Loasby 2000, p. 299). Implementation of Norms, Rules, Objects Learning Dynamic Co-Evolution Income expectations and price relations stabilize development How does Catallaxy avoid chaos and achieve order?

7. Catallactic Information Systems – Internal model • Self-Interest • Individual goals of the agent can be formalized (e.g. profit maximization) • Agent attempts to prognose future world state • Actions effect environmental state in order to achieve goals • Choice • Agent can choose between diverse alternatives • Agent can rank alternatives according to prognosed goal approximation • Environment is worth-oriented domain (cf. Rosenschein/Zlotkin) • Constitutional Ignorance • No agent can exactly prognose a future market state („future is blind“) • No agent can exactly prognose a „best strategy“ (always historically bound) • You never step twice in the same river (Heraklit) • Strategy is sophisticated trial and error procedure at best • Requires adaptive and learning strategy • Learning procedures are based on subjective past experiences

8. Consequences for Application Development • Application must be a Worth-Oriented Domain • Application Domain needs common value denominator (money) • Requires “money vs. Goods“ exchange • However: if the application domain already uses money, it can be directly modelled

9. Consequences for Application Development • Agent-based solution is always inferior to analytical optimization • Catallaxy is inverse scalable • Works better, the larger the network is • Information • The more information is available, the more accurate are the choices • The more agents, the more information exists • Computation • Computation is fully parallel (no central bottleneck) • Solution always exists in the system (no non-allocated resource)

10. What we could expect? • Catallaxis good for certain situations: • Load balancing • Large systems: inherent cost of global/up-to-date state information for resource allocation where autonomous and decentralized algorithms work well • Adaptive to changes: in demand, topology, location and number of resources evolutionary learningself-organisation (specially for non-uniform systems with “hot spots”) • Centralized/de-centralized systems may have oscillatory behaviour  “constitutional ignorance” • Centralized: tragedy of state info overload with scale; • Decentralized: tragedy of commons

11. Catallactic Information systems

12. The Catnet network simulator • Client: computer program at host, requests service • Service Copy: instance of service, hosted in a resource computer • Resource: host computer with limited storage and bandwidth

13. We are measuring... • Social Welfare: the sum of all utilities over all participants in a given timespan. • Utility = Benefit - Cost, basic utility function per participant. • Resource Allocation Efficiency (RAE): • [Marketing] "fill rate", the ratio of matched transactions divided by the number of all proposals. (#"accepts“/#"proposals“) • Comm.Cost= #messages * #hops • Response time

14. Our goal: compare baseline/Catallactic Communicationcost ResourceAllocationEfficiency BW utilization Reactiontime System • Quasi-static • Very dynamic • Low node density • High node density • Dynamics: change: %node disconnection time (SC?) • Node density: many small nodes / few large nodes SWF B B B B ~ C C C C C B B B B ~ C C B C C /

15. Message Flows (Baseline)

16. Message Flows (Catallactic)

17. Scenarios • Appropiate scale: 10th or 100th or 1000th nodes … • Change (dynamics): • Movement / failure, creation (R) • Change of demand (C) • Location of demand (which clients) • Characteristics (many, including temporal distribution) • Density: • Fragmentation of resource capabilities • Same global amount of resources: P2Pmany small PC, CDNfew large servers /

18. Demand • From several clients • At the same time, at different times • Requests with different price/priority • Rate: #requests/second  distribution in time, space. • Deterministic, random

19. Catallactic better than Centralized Very-dynamic (Many disconn.) Very-dynamic (Many disconn.) Dynamics Dynamics E2.3 E2.3 Mobile, ad-hoc, overloaded networks Mobile, ad-hoc, overloaded networks Exp. 1 E1.2 E1.3 Exp. 1 E1.2 E1.3 E2.2 E2.2 Fixed networks Fixed networks Exp. 2 Exp. 2 Quasi-static (few disconn.) Quasi-static (few disconn.) Node density (1/concentration) Node density (1/concentration) low high low high CDN P2P CDN P2P Grid Grid

20. Network of 105 nodes, 75 Clients, 105 ResourcesResource Density/#SC: 0/5, 1/25, 2/75 500 Client requests for service, during 10, 50, 75, 100, 125, 150 seconds Density/#SC: 0/5 1/25 2/75 RAE (%) Dynamics C/B C/B C/B C/B C/B C/B C/B C/B C/B Ongoing work

21. Conclusions • Initial simulation results prove that a decentralized, economic model works better in certain situations. • “Better” is a combination of factors (SWF) • Promising: • Large scale • Dynamic • Saturation • Resource allocation