Mechanism design with correlated distributions

1. Mechanism design with correlated distributions Michael Albert and Vincent Conitzer malbert@cs.duke.edu and conitzer@cs.duke.edu

2. Impossibility results from mechanism design with independent valuations • Myerson auction is revenue optimal for independent valuations • This is an impossibility result in disguise! • Myerson auction doesn’t always allocate the item, and it doesn’t always charge the bidders valuation • Bidder’s virtual valuationψ(vi)= vi - (1 - Fi(vi))/fi(vi) • The bidder with the highest virtual valuation (according to his reported valuation) wins (unless all virtual valuations are below 0, in which case nobody wins) • Winner pays value of lowest bid that would have made him win • Combined with the revenue equivalence theorem, we have an impossibility result. • The impossibility result is: we can’t efficiently allocate an item and maximize revenue at the same time. • More than that, we have to give some of the utility to the bidders because they have private information.

3. Why should we care about maximizing revenue? • Auctions are one of the fundamental tools of the modern economy • In 2012 four government agencies purchased $800 million through reverse auctions (Government Office of Accountability 2013) • In 2014, NASA awarded contracts to Boeing and Space-X worth $4.2 billion and $2.6 billion through an auction process (NASA 2014) • In 2014, $10 billion of ad revenue was generated through auctions (IAB 2015) • The FCC spectrum auction, currently in the final round, expects to allocate between $60 and $80 billion worth of broadcast spectrum It is important that the mechanisms we use are revenue optimal!

4. Do current techniques get us “close enough”? • Standard simple mechanisms do very well with large numbers of bidders • VCG mechanism revenue with n+1 bidders ≥ optimal revenue mechanism with n bidders, for IID bidders (Bulow and Klemperer 1996) • For “thin” markets, must use knowledge of the distribution of bidders • We use the distribution to set the reserve price for a Myerson auction • Thin markets are a large concern • Sponsored search auctions with rare keywords or ad quality ratings • Of 19,688 reverse auctions by four governmental organizations in 2012, one third had only a single bidder (GOA 2013)

5. What if Types are Correlated? • This result is for all possible distributions over bidder valuations • Specifically, the impossibility of efficient allocation and revenue maximization must encompass the case where the agents types are independent. • This is unlikely to hold in many situations • Oil drilling rights • Sponsored search auctions • Anything with a common value component (like similar inputs) • Under correlation, we can break this impossibility result • Cremer and McLean (1985, 1988), Albert, Conitzer, Lopomo (2016)

6. Example: Divorce arbitration • Outcomes: • Each agent is of high type w.p. .2 and low type w.p. .8 • Preferences of high type: • u(get the painting) = 11,000 • u(museum) = 6,000 • u(other gets the painting) = 1,000 • u(burn) = 0 • Preferences of low type: • u(get the painting) = 1,200 • u(museum) = 1,100 • u(other gets the painting) = 1,000 • u(burn) = 0 Distribution under independent valuations H L H .2*.2 = .04 .8*.2 = .16 L .2*.8 = .16 .8*.8 = .64 Maximum Utility = 5,728 Maximum Expected Revenue = 4,320

7. high low Perfectly Correlated Distribution high low 0 .2 0 .8 Maximum Social Welfare = 12,000*.2 + 2,200*.8 = 4,160

8. high low Clarke (VCG) mechanism high low Both pay 5,000 Husband pays 200 Wife pays 200 Both pay 100 Expected sum of divorcees’ utilities = (12,000-10000)*.2 + (2200-200)*.8 = 2000 Expected Revenue = 10,000*.2 + 200*.8 = 2,160

9. high low Mechanism with Perfect Correlation high low Both pay nothing Both pay nothing Both pay nothing Both pay nothing Expected sum of divorcees’ utilities = (12,000)*.2 + (2200)*.8 = 4,160

10. high low Maximum Revenue with Perfect Correlation high low Both pay $6000 Both pay nothing Both pay $1100 Both pay nothing Expected sum of divorcees’ utilities = (12,000 – 12,000)*.2 + (2200-2200)*.8 = 0 Expected Revenue = 4160

11. high low Clarke (VCG) mechanism + side payments high low Husband pays 200 Both pay 5,000 & husband pays 1,000, Wife pays 1,000 & both pay 1,000 Both pay 100 Wife pays 200 & husband pays 1,000, Wife pays 1,000 & both pay 1,000 Expected Revenue = 4160 Expected sum of divorcees’ utilities = (12,000 – 12,000)*.2 + (2200-2200)*.8 = 0

12. How much correlation do we need to maximize revenue? • Need to look at ex-interim individually rational (IR) mechanisms: • For now we will use dominant strategy (ex-post) incentive compatible: • Nearly any correlation will do! In fact, for bidders with two types each, any correlation at all will do! • We can do this with a Groves mechanism!

13. high low Slightly Correlated Valuations L H H .15 .05 high low L .15 .65 Husband pays 200 Both pay 5,000 Husband receives 23,300 Both pay 106,300 Wife pays 106,300 Both pay 100 Wife pays 200 Wife receives 23,300 Both receive 23,300 Husband pays 106,300 Maximum Expected Utility = 5,630 Expected revenue = 5,630

14. Cremer-McLean Condition

21. Can we do better than Cremer-McLean? • The Cremer-McLean condition is sufficient, but not necessary • While the condition is generic for two (or more) bidders with the same number of types, is this always going to be the case? • What if we really have an external signal that we are using to condition payments, so that there is only one bidder? • Ad auctions with click through rates of related ads • Prices of commodities that are used as part of the production process • What if we don’t know the distribution well? • Maybe we want to “bin” the other bidders bids in order to estimate a smaller distribution • What is both necessary and sufficient?

28. Necessary and Sufficient Condition for Ex-Interim IR and Dominant Strategy IC Full Revenue

29. Why restrict ourselves to Dominant Strategy IC? • While dominant strategy IC is sufficient to give us a generic condition when there are sufficient bidders, we’ve already seen that is not necessarily the case. • Can we relax the necessary conditions if we consider BNE incentive compatibility? • This gives us the ability to have multiple lotteries over the external signal.

39. Necessary and Sufficient Condition for Ex-Interim IR and BNE IC Full Revenue

40. A whirlwind tour of other interesting results in this area

41. Impossibility results from mechanism design with independent valuations • Myerson-Satterthwaite Impossibility Theorem [1983]: • We would like a mechanism that: • is efficient, • is budget-balanced (all the money stays in the system), • is BNE incentive compatible, and • is ex-interim individually rational • This is impossible! ) = x ) = y v( v(

42. Sufficient Conditions for Strongly Budget Balanced Mechanisms with Correlated Distributions • For Interim IR and BNE IC mechanisms, it is possible to construct an efficient and budget balanced mechanism! (Kosenok and Severinov 2008) • Given the following conditions: • The Cremer-McLean condition holds. • For any distribution over bidder types that is not the true distribution, it is impossible for at least one bidder to replicate their conditional distribution under the fake distribution, , by strategically misreporting their own type. • This ensures that the mechanism designer always has someone who he knows isn’t lying to give the excess payments to. • Both of these conditions are generic for three bidders, i.e. any random distribution will satisfy these conditions with probability 1.

43. Correlated vs Interdependent Values • So far, we have been discussing correlated valuations: • Mineral rights problem • However, valuations may instead be interdependent: • Common resale value • Suppose that there are two bidders bidding for a lawnmower, each receiving an independent signal, , and the winning bidder can sell the lawnmower to the losing bidder at half his value. • , and the valuation for each bidder is . • Then bidders can’t report their valuations only their signals! • Signals are independent but values are not! Can’t extract full surplus (doesn’t satisfy the Cremer-McLean Condition). • What is the optimal mechanism in this example? • However, signals can also be correlated, and we recover the Cremer-McLean result.

44. Correlated Mechanism Design with Ex-Post Mechanisms • Note that BNE mechanisms have a few undesirable properties: • May require the bidder to pay more than his valuation for the item • What if the bidder is risk averse? • Requires that the bidder knows the true distribution in order to reason about his best report • Is susceptible to forming coalitions (like peer prediction with scoring rules) • We can sidestep these issues if we use ex-post mechanisms • Always guarantees positive and maximal utility for reporting truthfully • Bidder doesn’t need to know the distributions to know that he would not be better off mis-reporting • Any member of a coalition always has a weakly dominant strategy to deviate and tell the truth • Note that we can do arbitrarily bad by using ex-post vs interim mechanisms • See Albert, Conitzer, and Lopomo (2016)

45. 1-Lookahead Auction (Generalized English Auction) • The 1-Lookahead auction is a simple ex-post mechanism • Run a Japanese auction until only one person is in the room, bidder i. • The time where everyone else leaves tells you their valuation • When only one person is left in the room, compute . • Calculate the optimal reserve price for bidder igiven, . • If bidder i’svaluation is larger than the reserve price, bidder i pays either the second highest bid or the reserve price, whichever is higher. • Otherwise, the item is not allocated. • For general correlated valuation settings, this is a 2-approximation to the optimal ex-post mechanism (Ronen 2001). • For symmetric distributions, this is the optimal mechanism (Lopomo 2000). • For interdependent valuations, this is also optimal with a couple of additional assumptions (Lopomo 2000).

46. Can we do anything prior free? • All of the mechanisms that we have discussed have relied strongly on the assumption that we know the prior distribution • What if we don’t know the prior and we can’t estimate it? • The Dhangwatnotai et. al. [2010] single sample mechanism • Elicit reports from bidders . • Choose a reserve bidder uniformly at random, denote his report by • Find the highest bidder whose report is above the reserve report. If this bidder doesn’t exist, don’t allocate the item. • Charge the highest bidder the value that he would have had to report to win the item (like in the Myerson auction). • This simple prior-free mechanism is a constant factor approximation to the optimal revenue of an ex-post IR and IC mechanism for both correlated and interdependent valuations (Roughgarden and Talgam-Cohen 2013). • This requires a few assumptions – symmetric, matroid, affiliated

47. Open questions • What is the optimal mechanism for distributions that do not satisfy the Albert-Conitzer-Lopomo condition? • Conjecture that with affiliation this becomes something like a Myerson auction where the reserve price is a menu of lotteries • What is the optimal mechanism that is weakly strategyproof for coalitions without side payments? • What is the computational complexity of calculating this mechanism? • What is the optimal prior-independent mechanism • Single sample is a constant factor approximation, but it’s not necessarily the best approximation • Will require restrictive assumptions