Let’s try and step on this canard before it grows wings … Oliver Kamm is quoting some writer at Fortune saying something that ain’t true about election betting markets.
The reputation of exit polls was perceptibly if unfairly damaged by the US presidential election. But, as a writer in Fortune magazine points out, another predictor was unambiguously accurate. This was the electronic predictions market: the various websites allowing punters to place bets on the electoral outcome.
As anyone who was watching the CT Election Night Special will know, this just isn’t true. The election markets, on the big day, were more or less exactly as bad at providing us with predictive information as were the exit polls. I think that we may have been the only place recording the intraday fluctuations on the prediction markets (which were massive), so maybe it’s important to summarise the facts.
The IEM was predicting a Kerry victory on the morning of the election; Tradesports wasn’t in the morning but started to do so during the day. When the exit poll data was published on Slate at 1600ET, both Tradesports and IEM reacted to it imediately; the price of the Bush contracts collapsed (about half an hour later, so did the Tradesports and IEM websites, under the load). As late as 1830ET, the markets had squarely plumped for Kerry, prompting me to write the following comment on our trader-talk thread:
Bush now down to 17+5 mid price on IEM. Surely to God he’s got more of a chance than that? Tradesports has him at 29 bid. For what it’s worth, Newsnight just suggested that Kerry’s lead in the early exit polls was attenuating as more information came in …
The Bush contracts recovered during the evening as it became clearer and clearer that the exit polls were wrong (by 2109ET, I was writing “Bush back up to 40% on Tradesports; anyone know anything we don’t?”, but there is simply no support in the intraday record for the contention that the markets had any information marginal to that contained in the exit polls.
The IEM and Tradesports contracts certainly did get it right by saying for most of the contract life that Bush would win, but so did the polls. Bush was ahead in the polls throughout the race, both nationally and in a preponderance of swing states; predictions of a Kerry victory like Ruy Teixeira’s were entirely based on a belief that don’t knows would break against the incumbent, that high turnout favoured Kerry, or various other ways of rationalising away the evidence in front of one’s eyes (a tendency we warned about). The best election predictions, as far as I can tell, came from Stuart “Dr Pollkatz” Thiel’s state-level poll correlation approach. As far as the question of “Collective Wisdom” is concerned, I stand by my assessment posted at 2:38am UK time (2138 ET), at which point I seem to remember that I was rather drunk:
“Bush keeps nudging up you know; more like 45% now. This election was 50/50 and remains 50/50, and we shouldn’t have got carried away by the “Wisdom” of fucking “Crowds of Halfwits Who Read Slate”. I still think that in a 50/50 jump ball, Bush wins by 20 electoral votes. Dark pessimism etc.”
{ 17 comments }
Matthew 12.17.04 at 9:53 am
Are there any examples of these kinds of markets where opinion polls have not been available? I suppose either because a) it’s a long time before the election (perhaps something like the London Mayor?) or b) because they are banned, e.g. in the last week of the Spanish election, or similarly (I think) in France.
dsquared 12.17.04 at 10:42 am
Well, they do them on the Oscars and they work reasonably well, but I don’t know whether there are polls on the Oscars or not.
Nick 12.17.04 at 11:16 am
Betfair’s markets were doing about the same during the night – from what I recall Kerry came in to about 1.25 at one point, while Bush was out past 5.
abb1 12.17.04 at 11:36 am
I lost 200 bucks on the IEM, dammit. I still can’t believe 59 million Americans can be so stupid. It was a scam, I demand a recount.
Angry Moderate 12.17.04 at 1:44 pm
The other fact that is strangely neglected is that the betting markets rely highly on the polls, while the polls don’t (yet!) rely on the betting markets. So the betting markets are the wisdom of the crowd absorbing and slightly correcting on the margin the polling information. I’d like to run a (near impossible) experiment where a group of people were denied all polling information but granted all other information and see how they did in an elections market.
Rob 12.17.04 at 2:12 pm
Ugh…I am feeling that people are really going to misunderstand what the “wisdom of crowds” provides. It can give us the best prediction of future events under given circumstances. But that doesn’t mean it is even a good prediction. Just compare the betting lines on NFL games to the final scores. Here we have a shorterm prediction where the outcome distribution is finite and the risks are fully understood and how many games actually end up pushes in a given week?
Giles 12.17.04 at 3:32 pm
Just to through a bit of conspiracy in – doesnt anyone else think the market was ramped.
Both the polls and markets correctly predicted a Bush victory 2 days before hand. Then the market tailed down – perhaps because some one was doing some tactical selling (having gained the infomration on how the market would respond the month before when there was that sudden downward spike due to a big sell).
Then on the day of the election “selected raw figures” that are favorable to a Kerry victory are leaked to the dimmer end of the Democratic blogosphere -e.g. wonkette. A panic ensues and Blogfeld buys Bush at 20 and makes a huge profit the next day.
And since these markets seem to be relatively unregulated, Blofeld has little fear of being pursued by the SEC.
Further on both sides of the spectrum, there’s little willingness to want to look into the matter. The right doesnt want to admit its markets aren’t purfect and the left doesnt want to admit that they were dupped.
Anyway I think we should be told. Ba!
Bruce Cleaver 12.17.04 at 4:25 pm
Rob brings up a great point – the markets are quite imperfect, albeit better than other methods. A one-eyed man in the land of the blind.
The analogy for sports betting has an important twist to it – I think the job of the oddsmakers is to get 50% of the bettors to vote on Team A, and 50% to vote on Team B, not necessarily to predict which team will actually win and by how much. They are predicting what the market will predict, so to speak, in order to minimize their risk and maximize their vigorish.
Rob 12.17.04 at 4:26 pm
Uhh, giles–the netowrks based their news reports on the exact same exit polls. The exit polls themselves would have to be manipualted to make what, $35k at most?
Giles 12.17.04 at 5:03 pm
Rob I don’t think the exit polls themselves were manipulated – just that some one put “selected” and put outlier raw figures into circulation and that then everyone just ran with them. Who could do that – maybe just some data compiler at Zogby/Gallop who was meant to be putting it all together for the definitive poll, he selected a few outliers and emailed them out. And once things got going the balancing process broke down. And yes, if it was someone low down, 35K would be enough to take this risk. After all the only liability that they face is getting the sack from zogby/gallop or whoever.
Winner 12.17.04 at 5:23 pm
Five p.m. Monday I submitted an entry to an election contest requiring predictions of the presidential winner in each state. Making my predictions, I relied heavily on the odds quoted at Tradesports. In only one instance, Wisconsin, did I, on the strength of my reading of the survey evidence, go against Tradesport’s odds.
I predicted 49 of 50 state outcomes correctly. The one I missed was Wisconsin.
Similar remarks apply to the Senatorial races.
It’s true that when the exit poll news broke, the odds tilted heavily in Kerry’s favor. This is an iffy business, and the Best Evidence at ten a.m. on Tuesday was the exit polls. The crowd responded appropriately (as for a while did in-the-know White House insiders).
Sebastian Holsclaw 12.17.04 at 6:52 pm
It is an interesting case of markets reacting to bad information. Is this a well understood area? (It obviously isn’t for me). It is pretty clear that if you had taken the predictions of the toy-markets at one day before the election, they were very accurate. But choosing that time is arbitrary and may be influenced by our current knowledge of the outcome. We should have determined the cut-off point before hand.
nick 12.18.04 at 9:26 pm
I think we can establish a New Rule of Lazy Journalism:
‘Electronic markets will always be deemed to have predicted results more accurately, regardless of whether or not they actually did.’
Justin Wolfers 12.19.04 at 2:13 pm
Let me make two very simple points about the response of markets on election day:
1. Sensible markets respond to information. If the election betting markets didn’t respond to the “news” in the exit polls, then I think we should indict them. (Unless one thought that the markets already knew which way all Americans were voting, then a “random” sample of such information is clearly news.) As such, efficient prediction markets should have responded to this news. And they did.
2. The remaining question is whether they responded appropriately. On this, it is harder to be sure. So how much should the markets have responded to this news? As always, when assessing the “wisdom of crowds” it is important to assess it relative to some alternative predictive institution. Two comparisons seem useful here. First, the S&P 500 moved almost in lock-step with Tradesports throughout election day. But you might object that this is simply comparing the foolishness of crowds with the foolishness of crowds. So the only other predictor I can think of comes from the standard statstical approach (which pollsters and others use). I believe that there were 13,660 people surveyed in the exit polls, although I don’t know how many had been polled in the early leaked numbers. This suggests a “standard error” on the exit poll of +/- 0.5%, or a 95% “confidence interval” around the polling estiate of +/- 1%. That is, given standard statistical methods (and the fact that it was not yet clear that the sample was demographically unbalanced), you would have thought that there was not a hope in hell that Bush could have won. So by this benchmark (which is really the next best predictive institution), the markets did pretty well in giving this news very little weight, as Bush only fell as far as 25-30 on election night. Personally, I market this up as one for the markets.
(For those who are interested, Eric Zitzewitz and I present data on the performance of the markets every ten minutes through election night in a forthcoming paper “5 Open questions About Prediction Markets”. In that paper we analyse the performance of the markets in somewhat greater depth, and raise questions about when and where we might expect the markets to fail.)
John Quiggin 12.19.04 at 9:13 pm
“Unless one thought that the markets already knew which way all Americans were voting, ”
But, a lot of the discussion thus far has implied something pretty much like this. At least, it has implied that the markets had collective access to the kind of information that would produce tight priors, unlikely to be radically revised in the light of exit polls.
Justin Wolfers 12.19.04 at 9:51 pm
John Quiggin’s response is interesting. It may adequately describe some of the excess hype in the media, but I don’t think that it describes any of the serious academic research on prediction markets at all…
In my own writing, I’ve argued that prediction markets serve three purposes, providing incentives for
1. Truthful revelation;
2. Information gathering;
3. An algorithm for information aggregation
No serious economist has ever said that they aggregate all information. Indeed, I’m also quite interested in figuring out which information aggregation mechanisms perform best in a variety of settings. (It won’t always be markets… Witness what happens to the odds on a horse race when Kerry Packer decides he wants to bet.)
But John raises a very interesting approach to thinking about the wisdom of markets. The new information raised by the exit polls produced a change in beliefs. Presumably one merely needs to invert Bayes’ Rule to back out just how tight the priors of the markets were prior to the exit polls. One could then compare this with the degree of confidence that other predictive institutions might allow.
That very strong (mis-)information (knowing how 10,000 people voted) led to a moderate revision of priors suggests that the market had already aggregate a fair bit of information. But this is clearly a glass half-full view, and I’m more than happy to hear that others view the same glass is half-empty. As long as we are looking at the same glass…
Kaus Hackula 12.23.04 at 11:07 pm
hey, I heard that if you get a crowd of several thousand people to throw darts at a board, and average the results, you get nearly a perfect bullseye.
Comments on this entry are closed.