Decision Markets

1. Decision Markets Robin Hanson Department of Economics George Mason University

2. “Pays $1 if Bush wins” Will price rise or fall? sell E[ price change | ?? ] buy price sell Lots of ?? get tried, price includes all! buy Buy Low, Sell High (All speculation is “gambling”!)

3. Today’s Prices 10-12% GMU wins DC NCAA bracket 60-62% Bird Flu in US by Sept 30 17-20% Hamas Recognize Israel by ‘07 11-13% Bin Laden caught by end ‘07. 3-7% Soc. Sec. private account bill by end ‘06. 44%  Hillary Democrat Pres. Nominee in ‘08. 47-49% Democrat President in ‘08. TradeSports.com

4. In direct compare, beats alternatives • Vs. Public Opinion • I.E.M. beat presidential election polls 451/596 (Berg et al ‘01) • Re NFL, beat ave., rank 7 vs. 39 of 1947 (Pennock et al ’04) • Vs. Public Experts • Racetrack odds beat weighed track experts (Figlewski ‘79) • If anything, track odds weigh experts too much! • OJ futures improve weather forecast (Roll ‘84) • Stocks beat Challenger panel (Maloney & Mulherin ‘03) • Gas demand markets beat experts (Spencer ‘04) • Econ stat markets beat experts 2/3 (Wolfers & Zitzewitz ‘04) • Vs. Private Experts • HP market beat official forecast 6/8 (Plott ‘00) • Eli Lily markets beat official 6/9 (Servan-Schreiber ’05) • Microsoft project markets beat managers (Proebsting ’05)

5. Growing # Corporations Trying

6. Inputs Outputs Prediction Markets Theory Foul Play For Same Compare! Status Quo Institution

7. Not Experts vs. Self-chosen Amateurs • Forecasting Institution Goal: • Given same participants, resources, topic • Want institution with accurate forecasts • Separate question: who let participate? • Can limit who can trade in market • Markets have low penalty for add fools • Hope: get more info from amateurs?

8. $1 if A p(A) $1 $1 if A&B p(A&B) $1 $ x if A E[x|A]*p(A) $1 $ x E[x] $1 $1 if A&B p(B|A) $1 if A $ x if A E[x|A] $1 if A Estimates from Prices

9. School Voucher Decision Advice $1 if Scores Up & Voucher P(V) $1 if Voucher P(SU | V) $1 Compare! P(SU | not V) $1 if Not Voucher $1 if Scores Up & Not Voucher P(not V)

10. Decision Market Applications E[ Test scores | School vouchers?] E[ SA civil war | US moves troops out? ] E[ Electricity consume | Price by hour ] E[ Sea level | CO2 Treaty adopted ] E[ firm stock price | fire CEO? ] Market costs start high, not depend on topic, so do big value questions first.

11. Typical Problems • Legal Barriers • Moral/Culture Concerns • Not really want to know • Hard to find precise related events • Can not get participation for cheap • Not enough events to validate, learn

12. Concerns • Self-defeating prophecies • Decision selection bias • Price manipulation • Inform enemies • Share less info • Combinatorics • Moral hazard • Alarm public • Embezzle • Bozos • Lies • Rich more “votes” • Risk distortion • Bubbles

13. Ask the Right Questions • Cost independent of topic, but value not! • Seek high value to more accurate estimates! • Relevant standard: beat existing institutions • Where suspect more accuracy is possible • Suspect info is withheld, or not sure who has it • Prefer fun, easy to explain and judge • Prefer can let many know best estimates • Not fear reveal secrets, use fear to motivate • Avoid inducing foul play

14. Eight Design Issues • How avoid self-defeating prophecies? • How handle billions of possible combos? • What if bad guys lose $ to mislead us? • What if bad guys gain $ by give us info? • How not alarm public, inform enemies? • Price can mislead if deciders know more. • Will markets induce people to lie? • Will markets help employees embezzle?

15. The Fuss: Analysts often use prices from various markets as indicators of potential events. The use of petroleum futures contract prices by analysts of the Middle East is a classic example. The Policy Analysis Market (PAM) refines this approach by trading futures contracts that deal with underlying fundamentals of relevance to the Middle East. Initially, PAM will focus on the economic, civil, and military futures of Egypt, Jordan, Iran, Iraq, Israel, Saudi Arabia, Syria, and Turkey and the impact of U.S. involvement with each. [Click here for a summary of PAM futures contracts] The contracts traded on PAM will be based on objective data and observable events. These contracts will be valuable because traders who are registered with PAM will use their money to acquire contracts. A PAM trader who believes that the price of a specific futures contract under-predicts the future status of the issue on which it is based can attempt to profit from his belief by buying the contract. The converse holds for a trader who believes the price is an over-prediction – she can be a seller of the contract. This price discovery process, with the prospect of profit and at pain of loss, is at the core of a market’s predictive power. The issues represented by PAM contracts may be interrelated; for example, the economic health of a country may affect civil stability in the country and the disposition of one country’s military may affect the disposition of another country’s military. The trading process at the heart of PAM allows traders to structure combinations of futures contracts. Such combinations represent predictions about interrelated issues that the trader has knowledge of and thus may be able to make money on through PAM. Trading these trader-structured derivatives results in a substantial refinement in predictive power. [Click here for an example of PAM futures and derivatives contracts] The PAM trading interface presents A Market in the Future of the Middle East. Trading on PAM is placed in the context of the region using a trading language designed for the fields of policy, security, and risk analysis. PAM will be active and accessible 24/7 and should prove as engaging as it is informative. Became:

17. PAM Press Of 500+ articles, these gave more favorable PAM impression: Article: later in time, more words, mentioned insider, news (not Editorial) style, not anonymous Publication: finance or science specialty, many awards, many readers

18. Academia Expert Panel News Media Web & Blogs Elections Rumor Mills Speculative Markets Can ask direct question? What $ per question? How clear answer? How precise, accurate? How fast/often updates? How open/equalitarian? Incentive: truth or what folks want/expect hear? Info Institutions & Features

19. Cosmo constant > 0 Cancercuredby 2010 Science 291:987-988, February 9 2001

20. Hollywood Stock Exchange Science 291:987-988, February 9 2001

21. Fourteen professional handicappers Estimated using 146 races, tested on 46 races Track Odds Beat Handicappers Figlewski (1979) Journal of Political Economy

22. Economic Derivatives Market Wolfers & Zitzewitz “Prediction Markets” (2004) Journal of Economic Perspectives

23. NFL Markets vs Individuals Average of Forecasts Servan-Schreiber, Wolfers, Pennock & Galebach (2004) Prediction Markets: Does Money Matter? Electronic Markets, 14(3). 1,947 Forecasters

24. Iowa Electronic Markets vs. Polls “Accuracy and Forecast Standard Error of Prediction Markets” Joyce Berg, Forrest Nelson and Thomas Rietz, July 2003.

25. Iowa Electronic Markets “Accuracy and Forecast Standard Error of Prediction Markets” Joyce Berg, Forrest Nelson and Thomas Rietz, July 2003.

26. Beating official forecasts • Drug sales • ~12 traders • ~14 days Source: eLilly & NewsFutures

27. Theory I - Old • “Strong Efficient Markets” is straw man • No info - Supply and Demand • Assume beliefs not respond to prices • Price is weighted average of beliefs • More influence: risk takers, rich • Info, Static - Rational Expectations • Price clears, but beliefs depend on price • No trade if not expect “noise traders” • Price not reveal all info • More influence: info holders

28. Theory II - Market Microstruture • Info, Dynamic – Game Theory • Example – Kyle ’85 • X - Informed trader(s) – risk averse • Y - Noise trader – fool or liquidity pref • Market makers – no info, deep pockets • If many compete, Price = E[value|x+y] • Info markets – use risk-neutral limit • If Y larger, X larger to compensate more info gathered, so more accuracy!

29. Theory III – Behavioral Finance • Humans are overconfident • Far more speculative trade than need • Mere fact of disagreement shows • Overconfidence varies with person, experience, consequence severity • Implications • Price in part an ave of beliefs? • Adds noise to price aggregates? • Prices more honest than talk, polls, …

30. Aumann in 1976 Re possible worlds Common knowledge Of exact E1[x], E2[x] Would say next For Bayesians With common priors If seek truth, not lie Since generalized to Impossible worlds Common Belief A f(•, •), or who max Last ±(E1[x] - E1[E2[x]]) At core, or Wannabe Symmetric prior origins We Can’t Agree to Disagree

31. Explanation: We Self-Deceive • We biased to think better driver, lover, … “I less biased, better data & analysis” • Evolutionary origin: helps us to deceive • Mind “leaks” beliefs via face, voice, … • Leak less if conscious mind really believes • Beliefs like clothes • Function in harsh weather, fashion in mild • When see our self-deception, still disagree • So at some level we accept that we not seek truth

32. Rafi Eldor and Rafi Melnick (2004) “Financial markets and terrorism” European Journal of Political Economy 20:367–386

33. Terror Concept: Red Teams • Key: want enough events to validate tech, let participants see their ability • Red Team tries to get items onto planes • Vary: item, demographic, airport, time • Chosen by team, or at random • Forecast chances of red team success • Given variation, or changed policy, budget • Invite others to also forecast, compare

34. 1. Self-Defeating Prophecies • Example • Speculators learn of possible airport attack • Market price of attack chance rises • Airport officials see price, close airport • Speculators are punished for their insight • Solution: predict outcome given choices • Predict terror attack given airport open

35. 2. Useful attack forecasts distinguish: • Thousands of possible locations • Hundreds of possible times • Dozens of methods of attack • Dozens of terrorism demographics • Dozens of policy responses • Many combinations for each attack • More combinations for many attacks

36. Policy Analysis Market • Every nation*quarter: • Political stability • Military activity • Economic growth • US $ aid • US military activity • & global, special • & all combinations

37. Return to Focus ? Trade IQcs4 IQcs4 < 85 85 03 03 SAum3 105-125 03 Update Payoffs: If & Ave. pay Select New Price 65% Max Up 95.13% +$34.74 -$85.18 -$19.72 Buy 10% Up 68.72% +$2.74 -$3.28 -$1.07 You Pick 65 % +1.43 -2.04 +0.34 Saudi Arabian Economic Health No Trade 62.47% $0.00 $0.00 $0.00 125 30 15 10% Dn 56.79% -$2.61 +$2.74 -$1.12 65 70 Sell Exit Issue 48.54% -$15.34 +$26.02 -$6.31 35 40 100 94 100 Max Dn 22.98% -$120.74 +$96.61 -$22.22 < 85 25 35 35 30 10 10 75 1 2 3 4 1 2 > 03 03 03 03 04 04 ? Return to Form Execute a Trade If US military involvement in Saudi Arabia in 3rd Quarter 2003 is not between 105 and 125, this trade is null and void. Otherwise, if Iraq civil stability in 4th Quarter 2003 is below 85, then I will receive $1.43, but if it is not below 85, I will pay $2.04. Abort trade if price has changed? Execute Trade Scenario

38. Simple Info Markets Market Scoring Rules Scoring Rules opinion pool problem thin market problem 100 .001 .01 .1 1 10 Best of Both Accuracy Estimates per trader

39. Laboratory Tests • Joint work with John Ledyard (Caltech), Takashi Ishida (Net Exchange) • Caltech students, get ~$30/session • 6 periods/session, 12-15 minutes each • Trained in 3var session, return for 8var • Metric: Kulback-Leibleri qilog(pi /qi) distance from market prices to Bayesian beliefs given all group info

40. Mechanisms Compared • Survey Mechanisms (# cases: 3var, 8var) • Individual Scoring Rule (72,144) • Log Opinion Pool (384,144) • Market Mechanisms • Simple Double Auction (24,18) • Combined Value Call Market (24,18) • MSR Market Maker (36,17)

41. Environments: Goals, Training (Actually: X Z Y ) Case A B C 1 1 - 1 2 1 - 0 3 1 - 0 4 1 - 0 5 1 - 0 6 1 - 1 7 1 - 1 8 1 - 0 9 1 - 0 10 0 - 0 Sum: 9 - 3 Same A B C A -- -- 4 B -- -- -- C -- -- -- • Want in Environment: • Many variables, few directly related • Few people, each not see all variables • Can compute rational group estimates • Explainable, fast, neutral • Training Environment: • 3 binary variables X,Y,Z, 23 = 8 combos • P(X=0) = .3, P(X=Y) = .2, P(Z=1)= .5 • 3 people, see 10 cases of: AB, BC, AC • Random map XYZ to ABC

42. KL(prices,group) 1- KL(uniform,group) MSR Info vs. Time – 7 prices 1 % Info Agg. = 0 0 5 10 15 Minutes -1

43. Experiment Environment (Really: W V X S U Z Y T ) Case A B C D E F G H 1 0 1 0 1 - - - - 2 1 0 0 1 - - - - 3 0 0 1 1 - - - - 4 1 0 1 1 - - - - 5 0 1 1 1 - - - - 6 1 0 0 1 - - - - 7 0 1 1 1 - - - - 8 1 0 0 1 - - - - 9 1 0 0 1 - - - - 10 1 0 0 1 - - - - Sum 6 3 4 10 - - - - Same A B C D E F G H A -- 1 2 6 -- -- -- -- B -- -- 7 3 -- -- -- -- C -- -- -- 4 -- -- -- -- D -- -- -- -- -- -- -- -- … • 8 binary vars: STUVWXYZ • 28 = 256 combinations • 20% = P(S=0) = P(S=T) = P(T=U) = P(U=V) = … = P(X=Y) = P(Y=Z) • 6 people, each see 10 cases: ABCD, EFGH, ABEF, CDGH, ACEG, BDFH • random map STUVWXYZ to ABCDEFGH

44. KL(prices,group) 1- KL(uniform,group) MSR Info vs. Time – 255 prices 1 % Info Agg. = 0 0 5 10 15 Minutes -1

45. Combo Market Maker Best of 5 Mechs

46. D A C G F B E H Overlapping Patches • Simple all-combo MSR per patch • Allow trade only if vars in same patch • Is Markov network, much research re update • Exact update rule possible, but only if net is tree • But any predictable error allows money pump • Monte Carlo sampling update solves this? • Arbitrage neighbors is simple, robust, not Bayes • Adapt tech to share assets across patches • How change or add patches?

47. 3. Manipulation • Lie (WorldCom) – hits all forecast institutions • Mix-it-up – slow reveal trade to max $ of info • Mislead – Terrorists lose $ to raise price error • Even if works, still useful if can calibrate accuracy • 0.1% prob. gain 0.1% of $500B/yr pays $1M PAM • Field, Lab agree: on ave manip. fails • Any trade not to info is “noise” trade • More noise trading => more accuracy! • Less error, more harm? Use price/harm linear

48. Simple Manipulation Model Kyle Style Market Microstructure Game Theory Market maker Manipulator Informed trader Noise trader Equilibrium

49. Lab Experiments Confirm Joint with David Porter, Ryan Oprea • 12 subjects, V=0,40,100 • Each clue like “Not 100”. • Manipulator = price bonus • Manipulators bid higher • Others accept lower • Prices no less accurate

50. 4. Moral Hazard • Ever trading profits from sabotage? • Not: 9/11, ’82 Tylenol, ’02 PaineWebber • Yes: hacker extortion, life insurance kill • Hard to match willing capital & skilled labor • Terrorism futures differ: • Good: amounts of money very small • We’d pay $10 to know where robber hits next • Bad: individuals influence events more?