Prediction Markets

1. Prediction Markets Leighton Vaughan Williams Professor of Economics and Finance Nottingham Business School Nottingham Trent University Leighton.Vaughan-Williams@ntu.ac.uk

2. How Well Do Markets Aggregate Information? • How wise is the crowd?

3. Galton’s Ox • In 1906, Sir Francis Galton (1822-1911), the English explorer, anthropologist and scientist, visited the West of England Fat Stock and Poultry Exhibition, where he came across a competition in which visitors could, for sixpence, guess the weight of an ox. • Those who guessed closest would receive prizes. • 800 people entered.

4. Galton’s crowd • “Many non-experts competed, like those clerks and others who have no knowledge of horses, but who bet on races, guided by newspapers, friends, and their own” (Brief paper by Galton in ‘Nature’, March 1907). • Reference: • F. Galton, Vox Populi, Nature, 75, • March 7, 1907.

5. Galton’s findings • Galton added the contestants’ estimates and calculated the average of the estimates. • Using the mean, the crowd had guessed that the ox (slaughtered and dressed) would weigh 1,197 pounds. In fact, the ox weighed 1,198 pounds. • The median estimate was 1,207 pounds, not as close but within 1% of the correct weight.

6. Treynor’s Jelly Beans Experiment • Jack Treynor, in a classic experiment, asked his class of 56 students to guess the number of jelly beans in a jar. The mean guess was 871. • The actual number was 850. Only one student guessed closer. • Reference: Jack Treynor (1987), ‘Market Efficiency and the Bean Jar Experiment’, Financial Analysts Journal, 43, 50-53. • See also: Kate Gordon’s seminal study of 200 students estimating the weights of items. The group (average) result was 94.5% correct: only 5 students were better than this. • Kate H. Gordon (1921), ‘Group Judgements in the Field of Lifted Weights’, Psychological Review, 28 (6), November, 398-424.

7. Webinar on Forecasting Excellence and Prediction Markets, Sept. 15, 2007. • Joe Miles, a mathematician employed at ‘eyepharma’ (a company offering services to the pharmaceutical industry) gave a presentation, with the following key points. • 1. He relayed the results of a ‘M&Ms in a jar’ experiment he had conducted with a large group of conference delegates at a pharmaceutical forecasting conference earlier that year. The estimates ranged from 381 to over 40,000! The median estimate was 1,789. The actual number was 1,747, just 2.4% off. The middle estimate was closer than any individual estimate. • 2. He relayed the results of an experiment conducted by ‘eyetravel’, a sister company, at a hotel industry conference. Delegates were asked to estimate the average price of a hotel room in Amsterdam that day. Estimates ranged by a factor of three, but the average estimate was just 0.5% off (Mean estimate = 117.8 Euro: Actual price = 118.4 Euro.

8. What destroyed the space shuttle ‘Challenger’? • On January 28, 1986, the space shuttle Challenger lifted off from its launch pad at Cape Canaveral. Seventy-four seconds later, it blew up. Within minutes, investors started dumping the stocks of the four major contractors who had participated in the Challenger launch: Rockwell International, which built the shuttle and its main engines; Lockheed, which managed ground support; Martin Marietta, which manufactured the ship's external fuel tank; and Morton Thiokol, which built the solid-fuel booster rocket. Within minutes, trading in Thiokol was suspended and by the end of the day, Thiokol's stock was down nearly 12 percent. By contrast, the stocks of the three other firms each fell a little but soon started to creep back up, and by the end of the day had fallen only around 3 percent. The market was right. Six months later and after an extensive investigation, Thiokol was held liable for the accident. The other companies were exonerated

9. How do you find a missing submarine? • On the afternoon of May 27, 1968, the submarine USS Scorpion was declared missing with all 99 men aboard. It was known that she must be lost at some point below the surface of the Atlantic Ocean within a circle 20 miles wide. This information was of some help, of course, but not enough to determine even five months later where she could actually be found. • The Navy had all but given up hope of finding the submarine when John Craven, who was their top deep-water scientist, came up with a plan which pre-dated the explosion of interest in prediction markets by decades. He simply turned to a group of submarine and salvage experts and asked them to bet on the probabilities of what could have happened. Taking an average of their responses, he was able to identify the location of the missing vessel to within a furlong (220 yards) of its actual location. The sub was found!

10. What are Prediction Markets?

11. Betting on the outcome • Betting markets aggregate all available information to produce best estimate, not least because those who know, and are best able to process the information, bet the most. Based on the ‘Efficient Markets Hypothesis’, the idea that markets accurately incorporate all relevant information.

12. Prediction markets v. Betting markets • The essential difference between prediction and betting markets is not an issue of structure. • Rather, prediction markets, as usually termed, are distinct from betting markets in the purpose to which they are put. • For example, when betting markets are used explicitly to forecast the outcome of any event, whether it is the World Cup or a rowing regatta, they are essentially acting as prediction markets. • Even so, the term ‘prediction markets’ often implies that the markets are being used to produce information externalities that can inform business and policy decisions.

13. The Hayek Question • How does one effectively aggregate disparate pieces of information that are spread among many different individuals, information that in its totality is needed to solve a problem? • Hayek’s answer was that market prices are the means by which those disparate pieces of information are aggregated. • “The mere fact that there is one price for any commodity ... brings about the solution which ... might have been arrived at by one single mind possessing all the information which is, in fact, dispersed among all the people involved in the process.” • Source: F.A. Hayek, ‘The Use of Knowledge in Society’, American Economic Review, 35, 4, Sept. 1945: 520.

15. Speed of the market in processing new information • Obama price spiked one day in August, 2008, despite the only obvious news being a relatively poor opinion poll. • Why?

16. Warp Speed Market – Saddam capture or neutralize • Date: 13 December, 2003 • Market moves from about 20 to 100. • Next day: News of Saddam capture announced by US.

18. Predicting the outcome of rowing regattas • Jed Christiansen (2007) reports on markets set up to predict the outcome of rowing regattas in the UK. Despite the small number of participants, and the absence of any incentives other than the challenge of getting it right, the predictions of the rowing events were highly accurate. • Christiansen puts the success of the experiment down to the effects of community and uniqueness, which encouraged motivated participation. • Source: Christiansen, J.J. (2007), Prediction markets: Practical experiments in small markets and behaviours observed’, Journal of Prediction Markets, 1, 17-41.

19. Polls or markets? • Predicting the winner of an election!

20. Early prediction markets • The earliest data we have from prediction markets are those from organized markets for betting on the US Presidential election between 1868 and 1940. • Although there are reports that these markets date back to the election of George Washington, and even before, the market in 1868 seems to be the first we would call a prediction market in that its data was used to inform the public about the likelihood of a particular candidate winning and may have been used by financial asset traders. • As an example, the New York Times reported that between $500,000 and $1 million was wagered on the Curb Exchange (the fore-runner to the AMEX) in one day on the 1916 election and that “oil stocks were almost forgotten”. The total amount wagered in these markets in 1916 was $165 million (at 2002 prices). • In this period between 1868 and 1940, the market failed to predict the winner on just one occasion.

21. AN EARLY BRITISH PREDICTION MARKET • Brecon and Radnor By-Election, 1985. • Mori v. Ladbrokes

22. ELECTION EVE • MORI: Labour to win by 18%. • LADBROKES: • Liberal candidate: 4 to 7 • Labour candidate: 5 to 4

23. WINNERS: • The Liberal candidate. • Those who ignored MORI and backed the market favourite.

24. Bush v. Gore, 2000IG Index v. Rasmussen

25. Outcome forecasts • IG Index: • 265-275 Bush • 265-275 Gore • Rasmussen: Bush by 9%

26. Opinion Polls v. markets • Opinion polls, like all market research, provide a valuable source of information, but they are ONLY ONE source of information. • Other information includes: • 1. Local canvass returns • 2. On-the-ground inside information • 3. Forecasting models • 4. Opinions of professional ‘pundits’ (‘experts’) • 5. Focus groups • Betting markets aggregate all the available information

27. Producing an optimal forecast • Because those who know the most, and are best able to process that information, tend to bet the most, this drives the market to produce an optimal forecast at any point in time. • Moreover, unlike polls, which are snapshots of opinion, betting markets are all about forecasting the eventual outcome. • Since the advent of zero-tax low-margin betting exchanges, the accuracy of these markets have improved yet further.

28. US Presidential Election 2004 • INTRADE state-by-state predictions: 50 out of 50.

29. British General Election, 2005 • Predicted Labour majority to within a handful of seats.

30. US Senate 2006 Intrade: All correct

31. US Presidential Election 2008 • Prediction markets • INTRADE state-by-state predictions: 49 out of 50 (called Missouri wrong). • BETFAIR state-by-state predictions: 49 out of 50 (called Indiana wrong). • Statistical Modelling Using Weighted Polling Data • FIVETHIRTYEIGHT predictions: 48 out of 50 (called Missouri and North Carolina wrong).

32. Indiana • Polls closed at 11.30 pm (UK time) in Indiana, a key state which McCain almost certainly needs to win to secure the Presidency. • McCain favourite to win Indiana on Betfair. • 11.45 pm: Obama becomes favourite to win Indiana, attracting significant sums to win from traders. • By this time CNN was calling just 1% of precincts in Indiana. • So what caused the shift to Obama on Betfair? • In retrospect, it seems that professional traders had latched on to the detail in the few published results. • Importantly, this shows the power of prediction markets in assimilating and processing new information very rapidly.

33. Early Precinct Results • Stueben: Kerry 34%, Obama 42% • De Kalb: Kerry 31%, Obama 38% • Knox: Kerry 36%, Obama 54% • Marshall: Kerry 31%, Obama 50% • Only the most well-informed had accessed these results by 11.45, and knew what they meant, i.e. a big swing from Republican to Democrat since 2004, but Betfair traders were among them. Minutes later, the swing was confirmed in Vigo County. By 12.20, Obama was shorter than 1 to 2 on Betfair.

34. The election was called at 4am, but Betfair watchers knew before midnight! • At 4am, California was declared, giving Obama the final few electoral votes required to win the Presidency. • At 2.30am Ohio was called by most news networks. • Before midnight, the knowledge that Indiana was going to Obama, or at least that McCain would at best claim a small win there, was enough to indicate to Betfair watchers that the election was all but over. At 12.23 am, McCain was available at 25 to 1. • Meanwhile, Fox News declared that Indiana was over-polling for Obama because it shares a border with his home state of Illinois! • It was well past 3am when Fox News called the election for Obama. • Betfair 1, Fox News 0.

35. 2010 UK General Election • It was the debates that lost it for the Conservatives!!! • Before the first debate, the markets all predicted a Conservative overall majority. • After the first debate, none of the markets ever predicted anything other than a hung parliament! • While the polls swung all over the place, the markets barely flickered after that first debate in predicting a hung parliament with the Conservatives the largest party with somewhere between 300 and 320 seats.

36. Could prediction markets have prevented 9/11? • The 9/11 Commission Report stated the problem like this: "The biggest impediment to all-source analysis - to a greater likelihood of connecting the dots - is the human or systemic resistance to sharing information. • "What was missing in the intelligence community ... was any real means of aggregating not just information but also judgements. In other words, there was no mechanism to tap into the collective wisdom of National Security nerds, CIA spooks, and FBI agents. There was decentralization but not aggregation ...“ (James Surowiecki, ‘The Wisdom of Crowds’) • Can the market can help achieve this? Some people within the US Department of Defence had been working on just such an idea for several months when al Qaeda struck. Indeed, in May 2001 the Defense Advanced Research Projects Agency (DARPA) had issued a call for proposals under the heading of 'Electronics Market-Based Decision Support' (later 'Future Markets Applied to Prediction' (FutureMAP).

37. FutureMAP (cont.) • The remit prescribed for FutureMAP was to create market-based techniques for avoiding surprise and predicting future events. It was not long, however, before the US media and key members of the Congress began to train their guns on the idea of such a market. After all, it isn't difficult to portray the market as no more than a forum for eager traders to profit from death and destruction. The populist arguments won the day and DARPA was forced to cancel the project. While most of the arguments against the market were emotional rather than intellectual, there was nevertheless some genuine intellectual concern as to how effective it would be likely to be.

38. Was Stiglitz right? • In particular, Prof. Joseph Stiglitz argued in an article published in the Los Angeles Times on 31 July 2003 ('Terrorism: There's No Futures in It'), that the market would be too "thin" (i.e. there would be too little money traded in the market) for it to be a useful tool for predicting events meaningfully. His argument was based on work he had previously published showing that markets can never be perfectly efficient when information is costly to obtain. The cost of obtaining and processing this information is, by implication, likely to act as a significant disincentive particularly in the context of a thin market (and hence low rewards). • But is it obviously the case that a properly constructed market, populated by suitably motivated (and perhaps screened) players can be viewed in this way?

39. Can prediction markets be used to study climate change? • 1) A properly constructed market might encourage climate change analysts to become more specific in their forecasts, and would encourage the development of new modelling techniques. 2) The markets could help to provide an assessment of the tangible impact upon climate change of various policies under consideration by governmental and international bodies.3) The market could potentially help to establish a price for carbon. 4) The markets could help to price in new information more quickly. 5) The market would help businesses and governments to hedge against both the dangers of climate change, and against costs of addressing it. • There could be a series of contracts and perhaps options on, for example, temperature, CO2 emissions, precipitation, and tropical storms which expired at various intervals.

40. Can prediction markets help us make flight plans? • Volcanic ash cloud • BA Strike • Can we construct a prediction market which can amass the collective wisdom of the informed crowd to help us plan our future schedule?

41. Prediction Markets in Public Authorities • A notable feature of public policy in the UK over the past decade has been the imposition by central governments of performance targets as a means of evaluating the performance of local public organisations. • Targets cover a huge range of activities ranging from those specific to health or education to those relating to more general local authority performance. • Targets are used as a means of evaluating performance, improving standards and allocating resources. The significance of achieving or not achieving particular targets can be very high for local politicians as well as senior managers in local authorities and health organisations in terms of both resources, public image. At the same time, it is extremely difficult for politicians and central managers to be aware of, let alone to process, the complex streams of information that are available

42. PMs in public authorities (cont.) • Within this context, prediction markets offer a potentially valuable tool that may be used to synthesize the specific knowledge of those directly involved with implementing policy at a lower level. The specific nature of targets relating to, e.g., waiting list times, educational outcomes, are both specific and quantifiable and, hence, ideal candidates for operating a trading market. Taking the example of health care targets, the numbers of people involved from nurses, doctors to administrators further suggest that the operation of markets in this context is feasible. • The value of the information provided by prediction markets will come primarily from the advance warning that politicians and managers will be given of weak performance in particular areas. This has the potential to improve resource allocation to make it more likely that key targets are met.

43. PMs for Public Policy Decisions • Example: Should policy A or policy B be undertaken to reduce waiting lists? • Current waiting list for an appointment at the eye clinic = 30 days. • Contract pays £1 for the length of the waiting list in days. And currently trades at £30 pounds. • Participants in the market can BUY the contract at £30 if they think the waiting list will increase and SELL if they think it will decrease. • E.g. If they SELL at £30 and the waiting list decreases to 25 days, they will £5 (30-25). But if the waiting list increases to 35 days, they lose £5. • By comparing the ‘Waiting list with policy A’ contract with the ‘Waiting list with policy B’ contract, the policy maker has gained information on what the ‘market’ thinks about the relative impacts of introducing policy A and policy B on the length of the waiting list. • If a policy is not implemented, the contract is declared void.

44. Using the power of prediction markets for disease surveillance • http://iehm.uiowa.edu/iehm/index.html • “Reporting speed is one of the most import aspects of any surveillance program; for seasonal influenza even two weeks advance notice can have dramatic results on the effectiveness of vaccinations. • Although there are many existing strategies for gathering opinions about the future trends of infectious diseases, the resulting data are often difficult to interpret using standard epidemiological methods. Prediction markets, on the other hand, are well known for their ability to quickly collect and summarize information. • The Iowa Electronic Health Markets is a research project at the University of Iowa exploring the use of prediction markets as a tool for disease surveillance. By combining the strengths of prediction markets with the knowledge of our trading community from around the world, our hope is that these markets will report future infectious disease activity quickly enough to be clinically useful”.

45. Limitations of crowd wisdom • Can the crowd predict the lottery numbers? • If not, why not? • Because lottery numbers are drawn randomly, no model or individual or crowd or other means of aggregating information can predict them because random numbers are by definition unpredictable. • If the lottery numbers were, for whatever reason, not drawn randomly, however, we have a different issue.

46. Is Manipulation Bad for Prediction Markets? • Robin Hanson and Ryan Oprea, of George Mason University and the University of California, Santa Cruz respectively, co-authored a paper title, 'A Manipulator Can Aid Prediction Market Accuracy‘. A perspective on its basic message is offered by Alex Tabarrok at Marginal Revolution. Tabarrok was considering the impact of the clear attempt by at least one determined trader to manipulate one of the US election betting markets in favour of Senator John McCain. In particular, the John McCain contract was bought in the markets systematically every morning by one US-based trader for sizeable sums. In consequence, it was possible to arbitrage between McCain (on Intrade) and Obama (on Betfair) for a few weeks in the run-up to Election 2008. • How much of a danger, Tabarrock asks, does this sort of activity pose for the whole concept of prediction markets? Not much, he argues, instead offering support for Hanson and Oprea's finding that manipulation can actually improve prediction markets, for the simple reason that manipulation offers informed investors a free lunch.

47. Manipulation (cont.) • "In a stock market", Tabarrok writes, "... when you buy (thinking the price will rise) someone else is selling (presumably thinking the price will fall) so if you do not have inside information you should not expect an above normal profit from your trade. But a manipulator sells and buys based on reasons other than expectations and so offers other investors a greater than normal return. The more manipulation, therefore, the greater the expected profit from betting according to rational expectations.“ • For this reason, investors should soon move to take advantage of any price discrepancies thus created within and between markets, as well as to take advantage of any perceived mispricing relative to fundamentals. Thus the expected value of the trading is a loss for the manipulator and a profit for the investors who exploit the mispricing. Moreover, the incentive the activity of the manipulator gives for others to become informed, and to trade on the basis of this information, is valuable in itself in improving the efficiency of the market.

48. Worth manipulating? • Tabarrok offers the additional observation that, considerations of predictive accuracy aside, there is one even more important lesson to be learned from the activities of the manipulators: "...that prediction markets have truly arrived when people think they are worth manipulating". • But have they? What does the corporate sector think?

49. HOW CAN COMPANIES USE PREDICTION MARKETS? • To take an example, a manufacturer of aero engines will seek good forecasts of future orders from plane manufacturers, which in turn will be contingent upon orders from airlines. Forecasts of future airline orders will be greatly assisted by the collation of a range of information from those involved in each of these sectors. • It is important that the questions posed are in a form which is unambiguous and which can ultimately be quantified. This requires an assessment of who should be involved in responding, and ensuring that each of these contributors has an equivalent understanding of the meaning of what is being asked, and that these answers can usefully be pooled. The set-up will vary depending on the diversity of contributors, both geographically and functionally. There is also the issue of incentives and the number of markets to run, as well as the length of these markets and how often new markets should be introduced. • But in principle markets should be able to help aggregate information.

50. But what’s the evidence? Can prediction markets actually help internal company forecasting? • There is in fact plenty of published research showing how internal prediction markets have helped improve the ability of commercial organisations to structure and implement internal prediction markets to assist in forecasting. • to predict key business variables • e.g. when will a product launch, what will be the unit sales? • broader-based prediction markets are a useful mechanism for predicting market-wide outcomes, e.g. box office receipts for a new film, success of a new video game, property prices.