Market Based Control of Complex Computational Systems

1. Market Based Control of Complex Computational Systems Nick Jennings nrj@ecs.soton.ac.uk

2. The Complex Systems Challenge Building software that operates effectively in environments that: • Have no centralised control • Are highly interconnected • Are in constant state of flux • Are highly unpredictable • Involve multiple, individually-motivated actors

3. Pervasive Computing Autonomic Computing Peer-to-Peer eCommerce Semantic Grid Semantic integration OGSA uses WS standards “Brain meets Brawn” The Complex Systems Landscape Web Services Semantic Web Service description Service discovery Service composition Flexible interoperation & reasoning in heterogeneous environments Agent Based Computing Grid Computing Robust, large scale open systems Autonomy Rich interactions

4. (Jennings, 2000 & 2001) Electronic institution Organisational relationship Agent Interaction Environment Sphere of influence The Computational Model • Entities offer services in an institutional setting • Entities connect to services • Service discovery • Service composition • Service procurement • Entities enact services • Flexible & context sensitive service delivery

5. “encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives” Agents as Service Providers & Consumers

6. “encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives” Agents as Service Providers & Consumers • control over internal state and over own behaviour

7. “encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives” Agents as Service Providers & Consumers • control over internal state and over own behaviour • experiences environment through sensors and acts through effectors

8. “encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives” Agents as Service Providers & Consumers • control over internal state and over own behaviour • experiences environment through sensors and acts through effectors • reactive: respond in timely fashion to environmental change • proactive: act in anticipation of future goals

9. Agree appropriate service contracts Service composition Service procurement Fixed price offerings Catalogues Dynamic pricing Negotiations Auctions Economic efficiency Historical precedent Negotiation as de facto Form of Interaction

10. permissible participants e.g. buyers, sellers & third parties interaction states e.g. accepting bids, auction closed events causing state transitions e.g. bid, time out, bid accepted valid actions bid, ask, propose, accept, reject, counter-proposal, critique reward structures who pays & who gets paid for what Computational Service Economies (Dash et al., 2003) Mechanism Design “rules of the game”

11. shaped by interaction protocol decision making employed to achieve trading objectives from very simple to very complex maximise benefit to self (self interest) and/or to group (social welfare) permissible participants e.g. buyers, sellers & third parties interaction states e.g. accepting bids, auction closed events causing state transitions e.g. bid, time out, bid accepted valid actions bid, ask, propose, accept, reject, counter-proposal, critique reward structures who pays & who gets paid for what Computational Service Economies (Dash et al., 2003) Mechanism Design Agent Strategies “rules of the game” “how to succeed in the game”

12. shaped by interaction protocol decision making employed to achieve trading objectives from very simple to very complex maximise benefit to self (self interest) and/or to group (social welfare) permissible participants e.g. buyers, sellers & third parties interaction states e.g. accepting bids, auction closed events causing state transitions e.g. bid, time out, bid accepted valid actions bid, ask, propose, accept, reject, counter-proposal, critique reward structures who pays & who gets paid for what Computational Service Economies (Dash et al., 2003) Mechanism Design Agent Strategies Game theory analyses interactions to determine likely outcomes and equilibria “rules of the game” “how to succeed in the game”

13. The Market-Based Control Project • Market-Based Control (MBC): • paradigm for controlling computer systems using economically-inspired techniques • Market mechanisms used to: • generate and predict emerging system properties, • although decisions are made independently by local agents that each have their own aims and objectives • a market is a self-organising system, directed by mechanism • The proposition: • MBC is good for effectively controlling and managing complex, adaptive, distributed computational systems

14. Objectives • Develop and evaluate core MBC technologies • Automated mechanism design • Automate design of market mechanisms to achieve a desired set of global goals • Adapt to a changing environment and changing (priority of) objectives • Predict and automate design of agent strategies • Apply MBC solutions to design and manage complex, distributed computational systems

15. Project Applications • Utility data centres • MBC to allocate computational resources & achieve a robust, scalable service • Distributed content delivery within p2p networks • MBC to regulate sharing of content • Decentralised control and scheduling of multiple robots • MBC to provide incentives for cooperation and to achieve global goals

16. Research Highlights • Competing sellers in online auctions • Strategies for bidding in multiple auctions • Empirical game theory to select mechanisms and strategies for complex markets • Adaptive auctions

17. Research Highlights • Competing sellers in online auctions • Strategies for bidding in multiple auctions • Empirical game theory to select mechanisms and strategies for complex markets • Adaptive auctions

18. Often strong competition among sellers in online auctions • How many eBay auctions yesterday? • 10 • 100 • 1000

19. Often strong competition among sellers in online auctions • Seller’s choice of mechanism & auction parameters affect buyer’s choice of seller • How should bidder choose between auctions/sellers? • How should a seller set its parameters? • Focus on seller’s reserve price & sealed-bid auctions

20. Model of Competing Sellers • Set & announce Reserve Price Seller Seller Seller Mediator Auction Auction Auction • Set & announce Auction Fees Buyers • Select seller • Bid in auctions

21. Shill Bidding • Competing sellers reduces optimal reserve price and expected revenue (compared to isolated auctions) • Avoid by shill bidding: • Seller disguised as buyer to bid in own auction. • Illegal and undesired, but hard to detect • But mediator can use auction fees to deter it • Use Evolutionary Simulationto: • Evaluate effectiveness of different types of auction fees in deterring shill bidding • Measure market efficiency

22. Results with Auction Fees Fraction of auctions won by shill bids Allocative efficiency CP= closing price RD = difference between reserve and closing prices

23. Observations • Competition among sellers affects choice of mechanism and auction parameters • Important to take competition into account when designing mechanisms and bidder strategies • Sellers can decide to shill bid in order to improve profits • Mediator (such as eBay) can deter shill bidding and increase efficiency by setting appropriate auction fees

24. International Competition • Made proposal to have new game in the Trading Agents Competition Foundation • TAC Market Design • “Reverse” Trading Agents Competition • Design mechanisms with varying: • Clearing policy • Information revelation policy • Auction fees

25. Research Highlights • Competing sellers in online auctions • Strategies for bidding in multiple auctions • Empirical game theory to select mechanisms and strategies for complex markets • Adaptive auctions

26. simultaneous sequential hybrid Bidding in Multiple Auctions • Different start/finish times • Simultaneous, sequential, or hybrid • Heterogeneous: • N single-unit auctions • 1st/2nd price sealed bid, English or Dutch • Each can have different number of bidders • Multiple items Find optimal best response

27. Heuristic Strategies • Setting too complex to analyse theoretically and find optimal strategies • Heuristic strategies: • Choose the thresholds • Single auction dominant strategy (DOM) • Equal threshold (EQT) • Choose the auction • Exhaustive search (ES) • Knapsack utility approximation search (KS) • Trade-off between speed and complexity

28. Heuristics close to optimal for this restricted case • EQT better than DOM • KS much more computationally efficient than ES

29. Research Highlights • Competing sellers in online auctions • Strategies for bidding in multiple auctions • Empirical game theory to select mechanisms and strategies for complex markets • Adaptive auctions

30. Empirical Game Theory • Game Theory is a mathematical theory which underpins auction- and mechanism-design • very powerful and, at least in theory, can tell us what are the optimal mechanism and strategies. • But some markets too complex to analyse in practice using game theory. • too many participants and too many possible moves. • Evolutionary methods do not always converge on robust strategies • Empirical Game Theory: • emerging field combines principled game-theoretic analysis together with computer simulation methods. • amenable to automation, so it may be used by agents themselves to decide on market mechanisms.

31. Empirical Game Theory • Analysing strategies in Double Auctions • Find payoffs for strategies by repeated simulations • Find mixture of these “pure” strategies that constitute a evolutionarygame-theoretic equilibrium

32. Research Highlights • Competing sellers in online auctions • Strategies for bidding in multiple auctions • Empirical game theory to select mechanisms and strategies for complex markets • Adaptive auctions

33. Fixed bid increment Discrete Bid English Auctions

34. Research Questions • What effect do these discrete bid levels have on the auction properties? • How should the auctioneer determine the discrete bid levels to use in any situation in order to maximise his revenue?

35. m [ ( ) ] [ ( ) ] m l l F F 1 1 ¡ ¡ º º X i + i 1 ¡ h i e e X [ ( ) ( ) ( ) ] l l l l E P P P 1 2 3 + + £ ¤ £ ¤ ( ) ( ) c a s e c a s e c a s e l l l l E F F = 1 1 l l i i i i ¡ ¡ ¡ ; ; ; = i i i i 1 1 + + 0 º ( ) ( ) m l l F F : : : ¡ i 0 i i 1 + = i 0 = Discrete bid levels implemented Bidders’ valuation distribution Mean number of bidders Calculating Auction Revenue (David et al., 2005) • We calculate the auction revenue by considering the probability of these three cases: • Gives the final result: • We can optimise this expression (analytically or numerically) to find the optimal discrete bid levels .

36. Optimal Bid Levels • Uniform bidders’ valuation distribution Bid increment decreases Reserve price increases

37. Optimal discrete bid levels Fixed bid increment Optimal discrete bid levels Fixed bid increment Fixed bid increment Optimal discrete bid levels Optimal Bid Levels • Increases expected revenue. • Decreases expected auction duration. • Increases expected auction efficiency.

38. Learning Auction Parameters • To calculate optimal discrete bid levels we must know: • The bidders’ valuation distribution. • The number of participating bidders. • Typically we do not know these parameters. • However, we can use Bayesian Machine Learning to estimate these parameters – online.

39. Auction Closing Price Parameter Estimates Optimal Bid Levels Auction Learning Auction Parameters (Rogers et al., 2005) Prior Knowledge

40. Bayesian Machine Learning • Bayesian machine learning is attractive for this application: • Makes use of our knowledge of how the auction closes. • Allows us to incorporate prior knowledge or experience. • Makes efficient use of the sparse training data (observations of auctions). • Computationally efficient (no need to maximise multi-dimensional functions).

41. Learning the Number of Bidders

42. Learning the Number of Bidders

43. Conclusions • MBC prima facie candidate for controlling complex, distributed computational systems with autonomous self-interested components: • Computational game theory / Mechanism design • Evolutionary algorithms / Machine learning • Decision theory • Ongoing research and goals: • design of mechanisms and strategies for MBC • gain understanding of and predict dynamic properties of market-based computational systems • develop formal representation and tools • Ultimate goal: automated mechanism design

44. Partners http://www.iam.ecs.soton.ac.uk/projects/mbc.html