The Potential of Prediction Markets

1. The Potential of Prediction Markets Robin Hanson Economics, George Mason University Chief Scientist, Consensus Point C.M.U. Computer Science, Feb. 15, 2011

2. ?? Professions Academia Bet Odds Media Polls P = .25 P = .25 What do “we” believe?

3. “Pays $1 if Obama wins” Will price rise or fall? sell E[ price change | ?? ] buy price sell Lots of ?? get tried, price includes all! buy Buy Low, Sell High

4. Current Event Prices 2-10% Julian Assange indicted in US by Apr. 2011 9-13% US or Israel overt strike on Iran by 2012 5-11% Magnitude 9 earthquake anywhere by 2012 10-15% Supersymmetric particle seen by 2012 11-12% Palin is Republican nominee in 2012 59-60% Obama is re-elected in 2012 30-37% US Sup. Court bans med. mandate by 2013 11-30% Japan says it has nuke by 2013 5-14% China war act on Taiwan by 2013 15-30% Higgs Boson seen by 2014 20-33% Cap & trade for US emissions by 2014 41-47% Nation using Euro drops it by 2015 InTrade.com

5. Beats Alternatives • Vs. Public Opinion • I.E.M. beat presidential election polls 709/964 (Berg et al ‘08) • Re NFL, beat ave., rank 7 vs. 39 of 1947 (Pennock et al ’04) • Vs. Public Experts • Racetrack odds beat weighed track experts (Figlewski ‘79) • If anything, track odds weigh experts too much! • OJ futures improve weather forecast (Roll ‘84) • Stocks beat Challenger panel (Maloney & Mulherin ‘03) • Gas demand markets beat experts (Spencer ‘04) • Econ stat markets beat experts 2/3 (Wolfers & Zitzewitz ‘04) • Vs. Private Experts • HP market beat official forecast 6/8 (Plott ‘00) • Eli Lily markets beat official 6/9 (Servan-Schreiber ’05) • Microsoft project markets beat managers (Proebsting ’05) • XPree beat corp error, 3.5 vs 6.6%

6. NFL Markets vs Individuals Average of Forecasts Servan-Schreiber, Wolfers, Pennock & Galebach (2004) Prediction Markets: Does Money Matter? Electronic Markets, 14(3). 1,947 Forecasters

7. “Prediction Market Accuracy in the Long Run” Joyce Berg, Forrest Nelson and Thomas Rietz, Jan. 2008.

8. Iowa Electronic Markets vs. Polls “Accuracy and Forecast Standard Error of Prediction Markets” Joyce Berg, Forrest Nelson and Thomas Rietz, July 2003.

9. Iowa FluMarkets Clinical Infectious Diseases 2007:44 (15 January)

10. Used By Hundreds of Firms • Sales - HP, Google, Nokia, XPree, O’Reilly, Best Buy • Deadlines - Siemens, Microsoft, Misys • Pick Project - Qualcomm, GE, Lily, Pfizer, Motorola, Intercontinental Hotels • Unknown - Novartis, GSK, Motorola, ArcelorMittal, Corning, Dentsu, Masterfoods, Thomson, Yahoo, Abbott, Chrysler, Edmunds, InfoWorld, FritoLay, Erickson, IHG, NBC, HVG, RAND, SAIC, SCA, TNT, Cisco, General Mills, Swisscom

11. Advantages Incentives Self-Selection Correct Biases • Numerically precise • Consistent across many issues • Frequently updated • Hard to manipulate • Need not say who how expert when • Issue is not experts vs. amateurs • At least as accurate as alternatives

12. Requirements Use: • Questions really want answered now • Will eventually know answer (or parts) • Suspect not getting frank info via usual channels • Don’t mind participants knowing best estimates • People who have or can get key info • Their time is the main cost • Little penalty for invite many don’t know • Incentives to entice participation • Money, attention, influence can legally offer • Valued when questions answered • Many fast questions, status quo estimates Validate:

13. To Evaluate Institutions Institution A Institution B When They Use Similar Inputs Compare Quality Of Outputs

14. Collective Forecasting Forecasts On Requested Topics User Contributions User Scores Engagement

15. Collective Forecasting Questions Consensus What exactly is my influence? What exactly are my incentives? My Forecast My Score How exactly do I express my opinion? Truth

16. Editing Interface Is Transparent If my edit increases the consensus chance of true state, I win. If decreases, I lose. I directly change the consensus Consensus My Edits My Score Truth

17. Issues Input: Contributions Output: Forecasts, Scores What questions can ask? How account for value? Use or validate system? Should adjust outputs? Who let see outputs? Sabotage & manipulation Legal, P.R. risks? • What info can express? • How account for costs? • Who let in where? • Enough Incentives • T-shirts enough? • Zero-sum scoring? • Limit Costs • Awkward Interface • Wait for offer accept • Retribution

18. Possible Issues For Today Problems Promises Accounting Governance • Info Leaks • Lying Advisors • Sabotage • Manipulation • Expressiveness

19. Info Leaks • Prices may inform rivals, alarm public • Traders now don’t know prices at order • Deal via: Limit orders, market orders, … • Can extend this to show less about price • Hide prices for a time delay after trade? • Orders can say how respond to price history • Trusted traders see actual prices?

20. Lying Advisors • E.g., WorldCom, other accounting scandals • Participants in, advisors out • Participants contribute directly to forecast • Advisors try to persuade participants • Advise via shared interest, track record • Forecasting institution seems irrelevant • But if advisors can bet, more reason to lie? • Let advisors show bet stake • Expect advisors to participate, not talk?

21. Sabotage • Ever trading profits from sabotage? • Not: 9/11, ’82 Tylenol, ’02 PaineWebber • Yes: hacker extortion, life insurance kill • Hard to match willing capital & skilled labor • Limit amount of “negative” holdings • Let investigators see trading records

22. Manipulation

23. No desire to bias Higher than expected Want lower price Want higher price Lower than expected A Graphical Model of Bias Private Info on Asset Value Joint distribution of private info Expected value Private Info on Desire to Bias

24. Counter-balanced by anti-bias example Then this is how price should vary with net orders Buy 3 Buy 2 Assume net orders monotone in asset info + bias desire × price effect Buy 1 An example of successful bias No net sale Sell 1 Sell 2 Simple Bias Story Private Info on Asset Value Private Info on Desire to Bias

25. The Game Traders pick Manipulator sees Traders see Traders, Manipulator pick Market Maker sees Market Maker picks All get payoffs Time Effort Bias Clues Quantity Total Price

26. Lab Data Hanson, Oprea, Porter JEBO, 2005 • 12 subjects, value = 0,40,100 • Each clue like “Not 100”. • 6 manipulators, get bonus for • higher price • Manipulators bid higher • Others accept lower • Prices no less accurate

27. 8 traders, Value = 0,100 • Each Prob(Clue=V) = 2/3 • 4 manipulators, bonus for price to hidden target 0,100 • 5 judges see prices, predict • Manipulators bid toward target • Prices and judges predictions • no less accurate R. Oprea, D. Porter, C. Hibbert, R. Hanson, D.Tila 2006

28. Expressiveness

29. $1 if A p(A) $1 $1 if A&B p(A&B) $1 $ x E[x] $1 $1 if A&B p(B|A) $1 if A $ x if A E[x|A] $1 if A Estimates from Prices • Probability • Expected value • Joint prob. • Conditional pr. • Conditional EV

30. Useful Conditional Estimates E[ Revenue | Switch ad agency? ] E[ Revenue | Raise price 10%? ] E[ Project done date | Drop feature? ] E[ Project done date | Add personnel? ] E[ Stock price | Fire CEO? ] E[ Stock price | Acquire firm X? ]

31. Factors Might Influence Sales E[Sales|Factor] P[Factor] • Economy recovers fast? • Competitors introduce new version? • We do big promotion? • We lower prices? They lower prices? • We add distribution channel? • We add feature F? They add feature F? • Our defect rate very low?

32. $1 if Sales Up & New Leader P(SU | NL) Change Leader Decision Markets P(NL) $1 if New Leader $1 Compare! P(SU | not NL) $1 if Not New Leader P(not NL) $1 if Sales Up & Not New Leader

33. $1 if Lifespan Up & Single Payer P(L | S) Compare! P(L | not S) $1 if Not Single Payer $1 if Lifespan Up & Not Single Payer P(not S) Single Payer Decision Markets P(S) $1 if Single Payer $1

34. 2008 US President Example From InTrade.com

35. Politimetrics.com US President Decision Markets

36. Legal permission Outcome Measured Aggregate-enough Linear-enough Conditional-enough Decision Distinct options Important enough Enough influence Public credibility Traders Enough informed Decision-insiders Enough incentives Anonymity Prices Intermediate-enough Can show enough Decision Market Requirements

37. Win Place Show All outcomes Yoopick Facebook Application Combo Betting Show Win Place Not Not Not

38. Sport Finals Tickets Ticket if Greece in Finals Greece v. Croatia

39. Imagine Combo Dashboard Ave. Worth: $12,459 Coming soon from Consensus Point

40. Ask For Detail Ave.Worth: $12,459 Them B Ship Date 2009 2010 J F M A M J J A S O N D

41. Make An Edit Ave. Worth: $12,459 If We Have Autozoop, you gain $53 But if We Don’t Have It You lose $78. OK?

42. Make an Assumption Scenario: 15% Ave. Worth: $10,724

43. Add 2nd Assumption Scenario: 2.3% Ave. Worth: $10,982

44. Edit As Before Scenario: 2.3% Ave. Worth: $10,724 If we have Autozoop, you gain $53 But if we don’t have it You lose $78. OK?

45. Represent p(s), $(s,i) Add/settle var, Add/take $ Browse E[x|A] & E[$|A] & history of changes For each E[x|A], show max/min/indifferent $ edits Allow edit of many E[x|A] Update D$(s,i) = b*D log(p(s)) Avoid money pump errors Edit structure? Compute Tasks D A C G F B E H

46. Environments: Goals, Training (Actually: X Z Y ) Case A B C 1 1 - 1 2 1 - 0 3 1 - 0 4 1 - 0 5 1 - 0 6 1 - 1 7 1 - 1 8 1 - 0 9 1 - 0 10 0 - 0 Sum: 9 - 3 Same A B C A -- -- 4 B -- -- -- C -- -- -- • Want in Environment: • Many variables, few directly related • Few people, each not see all variables • Can compute rational group estimates • Explainable, fast, neutral • Training Environment: • 3 binary variables X,Y,Z, 23 = 8 combos • P(X=0) = .3, P(X=Y) = .2, P(Z=1)= .5 • 3 people, see 10 cases of: AB, BC, AC • Random map XYZ to ABC

47. Experiment Environment (Really: W V X S U Z Y T ) Case A B C D E F G H 1 0 1 0 1 - - - - 2 1 0 0 1 - - - - 3 0 0 1 1 - - - - 4 1 0 1 1 - - - - 5 0 1 1 1 - - - - 6 1 0 0 1 - - - - 7 0 1 1 1 - - - - 8 1 0 0 1 - - - - 9 1 0 0 1 - - - - 10 1 0 0 1 - - - - Sum 6 3 4 10 - - - - Same A B C D E F G H A -- 1 2 6 -- -- -- -- B -- -- 7 3 -- -- -- -- C -- -- -- 4 -- -- -- -- D -- -- -- -- -- -- -- -- … • 8 binary vars: STUVWXYZ • 28 = 256 combinations • 20% = P(S=0) = P(S=T) = P(T=U) = P(U=V) = … = P(X=Y) = P(Y=Z) • 6 people, each see 10 cases: ABCD, EFGH, ABEF, CDGH, ACEG, BDFH • random map STUVWXYZ to ABCDEFGH

48. Combo Market Maker Best of 5 Mechs 3 subjects, 7 prices, 5 minutes 6 subjects, 256 prices, 5 minutes

49. KL(prices,group) 1- KL(uniform,group) MSR Info vs. Time – 255 prices 1 % Info Agg. = 0 0 5 10 15 Minutes -1

50. Accounting