Tuomas Sandholm

1. Expressiveness in mechanisms and its relation to efficiency:Our experience from $40 billion of combinatorial multi-attribute auctions, and recent theory Tuomas Sandholm Professor Computer Science Department Carnegie Mellon University Founder, Chairman, Chief Scientist CombineNet, Inc.

2. Outline • Practical experiences with expressiveness • Domain-independent measure of expressiveness • Theory on how it relates to efficiency • Application of the theory to sponsored search • Expressive ad (e.g., banner) auction that spans time

3. Sourcing before 2000 Pros Cons • Manual negotiation • Expressive =>win-win • Implementable solution • Unstructured, nontransparent • Sequential =>difficult, suboptimal decisions • 1-to-1 =>lack of competition • Basic reverse auction • Structured, transparent • Simultaneous negotiation with all suppliers • Global competition • Bidding on predetermined lots is not expressive =>~ 0-sum game • Lotting effort • Small suppliers can’t compete • Unimplementable solution • Bidding complexity & exposure

4. Expressive commerce • Expressive bidding • Expressive allocation evaluation

5. Expressive bidding • Package bids of different forms • Conditional discount offers of different forms (general trigger conditions, effects, combinations & sequencing) • Discount schedules of different forms • Side constraints, e.g. capacity constraints • Multi-attribute bidding – alternates • Detailed cost structures • All of these are used in conjunction • Don’t have to be used by all

6. Benefits of expressive bidding Pareto improvement in allocation • Finer-grained matching of supply and demand (e.g. less empty driving) • Exposure problems removed => better allocation & lower cost • Capacity constraints => suppliers can bid on everything • No need to pre-bundle => better bundling & less effort • Fosters creativity and innovation by suppliers • Collaborative bids => lower prices andbetter supplier relationships

7. Academic bidding languages unusable (in this application) • OR [S. 99] • XOR [S. 99] • OR-of-XORs [S. 99] • XOR-of-ORs [Nisan 00] • OR* [Fujishima et al. 99, Nisan 00] • Recursive logical bidding languages [Boutilier & Hoos 01] Fully expressive

8. Expressive allocation evaluation • Side constraints • Counting constraints • Cost constraints • Unit constraints • Mixture constraints • … • Expressions of how to evaluate bidder and bid attributes

9. Example of expressive allocation evaluation

10. Benefits of expressive allocation evaluation • Operational & legal constraints captured => implementable allocation • Can honor prior contractual obligations • Speed to contract: months  weeks • $ savings begin to accrue earlier • Effort savings

11. Clearing (aka. winner determination) problem • Allocate (& define) the business • so as to minimize cost (adjusted for buyer’s preferences) • subject to satisfying all constraints • Even simple subclass NP-complete & inapproximable[S., Suri, Gilpin & Levine AAMAS-02] • We solve problems ~100x bigger than competitors, on all dimensions: • > 2,600,000 bids • > 160,000 items (multiple units of each) • > 300,000 side constraints • > 1,000 suppliers • Avg 20 sec, median 1 sec, some instances take days • Speed & expressiveness: huge competitive advantage

12. CombineNet events so far • > 500 procurement events • $2 million - $1.6 billion • The most expressive auctions ever conducted • Total transaction volume > $40billion • Created 12.6% savings for customers • Constrained; Unconstrained was 15.4% • Suppliers also benefited • Positive feedback (win-win, expression of efficiencies, differentiation, creativity) • Un-boycotting • They recommend use of CombineNet to other buyers

13. Applied to many areas Chemicals Aromatics Solvents Cylinder Gasses Colorants Packaging Cans & Ends Corrugated Boxes Corrugated Displays Flexible Film Folding Cartons Labels Plastic Caps/Closures Shrink/Stretch Film Ingredients/Raw Mat. Sugars/Sweeteners Meat/Protein TransportationAirfreightOcean FreightDrayTruckload Less-than-truckload (LTL)BulkSmall ParcelIntermodal3PLs Industrial Parts/Materials Bulk Electric Fasteners Filters Leased Equipment MRO Pipes/Valves/Fittings/Gauges Pumps Safety Supplies Steel Marketing Media buy Corrugate Displays Printed Materials Promotional Items Technology Security Cameras Computers Services Pre-press Temporary Labor Shuttling/Towing Warehousing Medical Pharmaceuticals Medical/surgical supplies Miscellaneous Office Supplies

14. Broader trend toward expressiveness

15. Amazon.c & New Egg o offer bundles of items (ca. 2000)

16. Facebook increases expressiveness of privacy control (2006) “…we did a bad job of explaining what the new features were and an even worse job of giving you control of them…. This is the same reason we have built extensive privacy settings — to give you even more control over who you share your information with.”

17. CD+Tunes adds option for users to rent movies (2007)

18. Checked baggage Airlines charge extra for baggage, food & choice seats (2008)

19. Prediction/insurance markets becoming more expressive

20. Is more expressiveness always better? • Not always for revenue! Expressive mechanism: vi( )? vi ( )? vi ( )? An inexpressive mechanism: vi ( )?

21. Is more expressiveness always betterfor efficiency? • And what is expressiveness, really? [Benisch, Sadeh & S. AAAI-08]

22. What makes a mechanism expressive?A straw man notion Item bid auction Combinatorial auction $5 $2 $5 $2 $6 Expression space 2 Expression space 3

23. What makes a mechanism expressive? Prop: Dimensionality of expression space does not suffice Proof intuition [based on work of Georg Cantor, 1890]: a b Expression space 3 Expression space 1 Work on informational complexity in mechanisms [Hurwicz, Mount, Reiter 1970s…] puts technical restrictions that preclude such mappings Mapping

24. Our notion: Expressive mechanisms allow agents more impact on outcome An agent’s impact is a measure of the outcomes it can choose between by altering only its own expression $X $Y $4 $6 $Y A C B D $4 $6 $X

25. Uncertainty introduces the need for greater impact A,A A,C C,C $Y $X $Y A,B A,D C,D $4 $6 $4 B,B B,D D,D $6 $X

26. Uncertainty introduces the need for greater impact… A,A A,C C,C $Y $X $Y $7 A,B A,D C,D B,B B,D D,D $7 $3 $3 $X

27. Uncertainty introduces the need for greater impact… A,A A,C C,C $Y $X $Y $7 A,B A,D C,D $4 $6 $4 B,B B,D D,D $7 $3 $3 $6 $X • 10 outcome pairs but only 9 regions • In this example the impact vector B,C can’t be expressed

28. Expressive mechanisms A,A A,C C,C $Y $Y Z=0 $Z A,B A,D C,D A,A A,C C,C $Y $X $Y $Z B,B B,D D,D $7 $3 $X $X Some Z>0 A,B A,D C,D $4 $6 A,A Extra Region $Y • In combinatorial auction all 10 pairs can be expressed A,B B,C B,B B,D D,D B,B B,D D,D $X $X • Our measure of expressiveness for one agent (semi-shattering): how many combinations of outcomes can he choose among • Not just for combinatorial allocation problems because outcomes can be anything • Captures multi-attribute considerations as well

29. An upper bound on a mechanism’s best-case efficiency • We study a mechanism’s efficiency when agents cooperate • It bounds the efficiency of any equilibrium • It allows us to avoidcomputing equilibrium strategies • It allows us to restrict our analysis to pure strategies only ? ?

30. Theorem: the upper bound on efficiency for an optimal mechanism increases strictly monotonically as more expressiveness (# of expressible impact vectors) is allowed (until full efficiency is reached) Proof intuition: induction on the number of expressible impact vectors; each time this is increased at least one more efficient outcome is allowed

31. Theorem: the upper bound on efficiency for an optimal mechanism can increase arbitrarily when any increase in expressiveness (# of expressible impact vectors) is allowed Proof intuition: construct preference distributions that ensure at least one type makes each combination of outcomes arbitrarily more efficient than any others

32. The bound can always be met Theorem: for any outcome function, there exists at least one payment function that yields a mechanism that achieves the bound's efficiency in Bayes-Nash equilibrium Proof intuition: if agents are charged their expectedimposed externality (i.e., the inconvenience  that they cause to other agents in the potentially inexpressive mechanism), then making expressions that maximize social welfare is an optimal strategy for each agent given that the others do so as well

33. Application to sponsored search [Benisch, Sadeh & S. Ad Auctions Workshop 2008]

34. Heterogeneous bidder preferences Prototypical value advertiser Bidder utility Prototypical brand advertiser Rank percentile

35. Mechanisms we compared Google, Yahoo!, Microsoft, … Our proposal Rank 1 $5 2 Premium mechanism Fully expressive mechanism Inexpressive mechanism $4 3 4 $5 $4 $3 $2 1 1 $4 2 2 Expressiveness 3 3 4 4

36. Inexpressive mechanism Premium mechanism Fully expressive mechanism

37. Expressive ad (e.g., banner) auctions that span time, and model-based online optimization for clearing [Boutilier, Parkes, S. & Walsh AAAI-08]

38. Prior expressiveness • Typical expressiveness in existing ad auctions • Acceptable attributes • Per-unit bidding (per-impression/per-clickthrough (CT)) • Budgets • Single-period expressiveness (e.g., 1 day) • Most prior research assumes this level of expressiveness

39. Campaign-level expressiveness • Advertising campaigns express preferences over a sequence of allocations • Minimum targets: pay only if 100K impressions in a week • Tiered preferences: $0.20 per impression up to 30K, $0.50 per impression for more • Temporal sequencing: at least 20K impressions per day for 14 days • Substitution: either NYT ($0.90) or CNN ($0.50) but not both • Smoothness: impressions vary by no more than 20% daily • Long-term budget: spend no more that $250k in a month • Exclusivity • Additional forms of expressiveness • Advertiser’s choice of impression/CT/conversion pricing (or combination) • Target audience (e.g., demographics) rather than indirectly via web site properties

40. Bidder 1: 45.45k Bidder 2: 4.55k t0 t1 t2 Bidder 1: 9.09k Value of optimization under sequential expressiveness • Bidder 1: bids $1 on A, $0.50 on B, budget $50k • Bidder 2: bids $0.50 on A, budget $20k • Traditional first-price auction: $52.3k revenue Supply of A 50k 0 . . . . . . Supply of B 10k 10k

41. t0 t1 t2 Value of optimization under sequential expressiveness • Bidder 1: bids $1 on A, $0.50 on B, budget $50k • Bidder 2: bids $0.50 on A, budget $20k • Optimal allocation: $70k revenue Bidder 2: 40k Bidder 1: 10k Supply of A 50k 0 . . . . . . Supply of B 10k 10k Bidder 1: 80k

42. Stochastic optimization problem • Advertising channels C • Supply distribution of advertising channels PS • Set of campaigns B • Spot market demand distribution PD • Time horizon T • Can be modeled as Markov Decision Process (MDP) • But how do we make it scale?

43. Scalable optimization with real-time response • Huge number of possible events => infeasible to compute full policy contingent on all future states • Cannot reoptimize policy in real time at every event • Optimize-and-dispatch architecture [Parkes and S., 2005] • Periodically compute policies with limited contingencies (e.g., stop dispatching when budget reached) • Dispatch in real time • Policy form: xti,j - fraction of channel i allocated to campaign j at time t • Optimize over coarse time periods (e.g., minutes, hours) • Tradeoff between optimization speed and optimality • Finer-grained in near-term, coarse-grained in long-term

44. Channels • A channel is an aggregation of properties (web pages or spots on them) • Constructed automatically based on campaigns • Lossless aggregation: two web pages are in the same channel if indistinguishable from the point of view of bids • Example: • Bid 1: NY Times (NYT) • Bid 2: Medical article (Med) • Channels: (NYT ∧ Med), (NYT ∧ ¬Med), (¬ NYT ∧ Med) • Non-NYT pages grouped together, non-Med pages grouped together • We can also perform lossy abstraction to avoid exponential blowup

45. Algorithms for stochastic problem • Infeasible to solve the MDP • Huge state space – cross product of individual campaign states • High-dimensional continuous action space • Our approaches: • Deterministic optimization • Online stochastic optimization

46. Deterministic optimization • Replace uncertain channel supply with expectations • Formulate the problem as a mixed-integer program (MIP) • Solving a MIP is much faster than an MDP • Our winner determination algorithms can solve very large problems [S. 2007] • Solutions may be far from optimal if supply distributions have high variance • Does not adequately account for risk • Can be mitigated by periodic reoptimization

47. Sample-based online stochastic optimization [van Hentenryck & Bent 06] • Compute only nextaction, rather than entire policy • Informed by what we might do in the future • Recompute at each time period • Sample-based • Solve w.r.t. samples from distributions • Extremely effective when good deterministic algorithms exist • Requires that domain uncertainty is exogenous • Distribution of future events doesn’t depend on decisions • Roughly true for advertising: allocation of ads should have little effect on supply of channel

48. Time t Action Value xt,1 f(xt,1) xt,2 f(xt,2) xt,3 f(xt,3) Choose xt,i that maximizes f(xt,i) λKt+4 λKt+2 λKt+1 λKt+3 λ1t+1 λ1t+2 λ1t+3 λ1t+4 λ2t+2 λ2t+3 λ2t+4 λ2t+1 λKT λ1T λ2T ... ... ... λKt λ2t λ1t . . . . . . xt,n f(xt,n) REGRETS algorithm[Bent & van Hentenryck 04] Sampleλ1 Optimal solution Sampleλ2 Optimal solution . . . SampleλK Optimal solution

49. Lower bound on Q-values for action xt at time t λKt+4 λKt+3 λKt+2 λKt+1 λ2t+2 λ2t+4 λ2t+3 λ1t+4 λ2t+1 λ1t+2 λ1t+1 λ1t+3 λKT λ2T λ1T ... ... ... . . . λKt λ2t λ1t REGRETS algorithm [Bent & van Hentenryck 04] Sampleλ1 Optimal solution Sampleλ2 Optimal solution . . . SampleλK Optimal solution

50. REGRETS doesn’t apply to ad auctions • Requires set of possible first-period decisions to be small • Our dispatch policies are continuous • Even a discretization of our continuous decision space would be huge: dimensionality = |C||B||Discretization|