In the weekend edition of the Australian Financial Review (reproduced here), Justin Wolfers writes about a betting market on the Iraqi election turnout, run by the Irish betting exchange Tradesports. The bet turned on whether turnout would exceed 8 million and was roughly even money before voting began. The price of the contract rose sharply on early reports of turnouts over 70 per cent, then fell back again when to around even money when it became clear these reports had little basis. The final official turnout was about 8.4 million.
Readers will recall that something very similar happened in the US election when early exit polls favored Kerry. Modifying an old aphorism to say that “two striking observations constitute a stylised fact”, I think we can now say pretty safely that political betting markets display the wisdom of crowds who read blogs.
In economic terms, we need to look at the implications for the efficient markets hypothesis, which comes in various levels of strength. The weak form, that you can’t predicted prices on the basis of their own past movements is well confirmed, and not of much interest here. The semi-strong form is that markets make the best possible estimate, given available public information. This, I think, is still open for debate. Obviously markets react to the news, but in these two instances they appear to have over-reacted. So, it seems likely that in a market with new information arriving continuously, we would replicate the stockmarket finding of excess volatility.
Finally, there’s the strong form of the hypothesis which is that markets make the best use of all available information, public or private. This has clearly failed, or else been shown to be irrelevant. Either there was no information anywhere to suggest that the early reports were wrong in these cases, or there was no useful private information which markets failed to incorporate in prices1.
Now, let’s look back at the most controversial proposal for use of betting markets, the idea of ‘terrorism futures’. Since most of the interesting information here is private (the public gets color-coded alerts and that’s about it), the claim that a market in terrorism futures would provide useful information depends on the strong form of the efficient markets hypothesis. The (stylised) fact is that strong form efficiency doesn’t hold in political markets, and therefore that terrorism futures would provide no useful information.
1 In a strict sense, the private information obviously existed. People knew whether and how they had voted, and a perfectly aggregating information market should have been perfectly informed about the votes that had been cast, and well-informed about votes that people intended to cast.
What if Osama used the terrorism futures market to help fund his activities? We’d get some more warning while he got more money. Kind of a super efficient way to pay Osama’s henchmen for ratting out their plans.
Has the strong form of the efficient market hypothesis really failed? In the election case, the markets were making use of information which was falsely passed off as accurate.
One less strong form of the EM Hypothesis says that the people who possess more accurate information, or more correctly evaluate the available information, will be the ones to make money from the market. Anyone who disbelieved the exit polls, and bought Bush at 30, made money on election day. (I made $90 on a $250 investment on Tradesports on Election Day, daytrading individual state results.)
Another issue which might make “terrorism futures” ineffective is that there’s usually a time lag before market sorts out good information from bad; someone with the ability to run a believable disinformation campaign (like the people who ran the exit polls and leaked their results) will be able to manipulate the market and cloud its predictive ability.
Riffing off Jet’s comment, Osama or his henchmen could make lots of money by creating preparations for an attack, and leaking word of them to people with the appropriate connections, then bet on the market move which would occur once the word gets out.
It’s a damn poor theory that will get rejected merely on account of contrary evidence.
When an economist says the evidence is “mixed,” it means the theory says one thing and the data says the opposite.
Another nice post - always fun to read.
Let me simply point out something that two comments made in the post are in direct contradiction to each other:
1. “Obviously markets react to the news, but in these two instances they appear to have over-reacted.”
2. “In economic terms, we need to look at the implications for the efficient markets hypothesis, which comes in various levels of strength. The weak form, that you can’t predicted prices on the basis of their own past movements is well confirmed, and not of much interest here.”
If (1) is true, then whenever there is good news the price will rise by a lot (too much). However, if markets over-react, then we can all make money by selling the contract when the price rises by a lot, and then covering our positions by buying when the market price reverts to fundamentals as it becomes clear that the original news wasn’t that interesting (or holding the contract until expiry would be a winning strategy given that our positions are based on market mis-pricing). Thus, if (1) is true, then one can predict future price movements will work to offset past large price movements, and hence (2) is not true as markets are not weak-form efficient.
For my money, the jury is out as to when and where even the weak form of the EMH is true - sometimes it works, sometimes we see over-reaction, and sometimes we see under-reaction. (BTW, the consensus view among behavioral finance folk - largely reflecting the response to earnings announcements - seems to be that at a daily horizon we see under-reaction, possibly reflecting “anchoring” of views among traders.)
I should add that if the wise punters reading this blog truly believe that there is a consistent pattern of over-reaction in these markets, you can of course bet against the crowd, which would both make you a tidy profit, and push the markets closer to being efficient. And I wouldn’t be surprised to find stronger evidence favoring market efficiency as these markets mature.
Finally, I’m not sure I understand the source of the claim that “The (stylised) fact is that strong form efficiency doesn’t hold in political markets.” Falsifying strong-form efficiency requires one to test whether private information is reflected in market prices. The problem is that (by definition), the economist never sees this private information, and hence cannot falsify this hypothesis. (This may present its own scientific problems!) Of course we can all agree that the private information held by 8 million different Iraqis as to whether each voted was not perfectly aggregated by this market, which is (strictly speaking) a falsification of the strong-form of the EMH, albeit not a particularly interesting one. My interpretation of the Iraq market is not that it didn’t immediately aggregate the private information of a few who knew the truth, but instead that there really wasn’t much useful information out there to aggregate (except perhaps that held by a few top-ranking election officials).
Thanks for these points, Justin. I worried about the apparent contradiction before posting, but thought it wasn’t a problem. Supposing that markets over-react to news, there is excess volatility, but I don’t think this means that a strategy based purely on observations of prices can be profitable. I could be wrong on this, but I’d like to see a more convincing argument.
It does imply that a cool-headed trader can make money in circumstances where markets move dramatically in response to unimportant news, and this would have been the case here, at least in relation to the Kerry exit polls, I think. But this requires an observation of the news and a judgement that the market has over-reacted.
A follow-up. If markets over-react to some kinds of news and under-react to others, then semi-strong efficiency will fail, but there’s no problem with weak-form efficiency.
On your last para, I agree. I guess we could subdivide further, with a moderately strong version where some traders are known to have inside information and a super-strong version where the information is supposed to be sought out and revealed even when it is held by non-traders.
Simple sketch of a pseudo-proof:
1. News hits. The market over-reacts, and price rises a lot, to a level that is above “truth”. (Your conjecture)
2. Just prior to the market closing the truth becomes evident. Given that the contract will be redeemed in 10 minutes (when contracts are paid out) at a price equal to the actual outcome, or “truth”, the equilibrium market price = truth.
(1) and (2) imply that that subsequent to a positive over-reaction there must be a decline in the price before the market closes. Thus following price rises there must be price declines.
You are right that over-reaction implies excess volatility, but equally, it implies negative autocorrelation in price changes, and hence a falsification of weak-form efficiency.
Of course if there is over-reaction to non-news, and under-reaction to real news, then these two effects will cancel, and one may not see any autocorrelation in price changes.
And you are right that making money in such a setting requires a judgment that the news was important or spurious, but I think that this is precisely the point. The relevant question is whether the judgment inherent in the market prices is better or worse than alternative judgments. On US election day it was much better than my judgment: after the Kerry exit polls were released, I thought he was a sure thing; the markets were less sure, and still rated Bush a 30% chance.
John:
Your follow-up was quicker than mine, albeit contemporaneously written. (You are brutally quick! It took me four paragraphs to get the logic out!)
Bottom line is that I think that we agree on the weak efficiency v. semi-strong issue.
And on the rest, it will be fun to continue to watch the evidence roll in.
John, where are you getting the data that the bet was “roughly even money before voting began”?Justin’s article says that the market had priced the contract at $65 on election eve.
I’m also unconvinced that the price movements on the day of the election really count as any kind of revelation. We’ve always known that there’s excessive volatility on triple-witching days, or on expiration days for OJ contracts, etc. If you want to take that as evidence that there are other forecasting methods that are consistently better than collective judgments (whether in the form of markets or parimutuel pools or whatever), okay, but I need a little more evidence.
James, I stand corrected on the ‘even money’ (I was looking at the $50 figure for during the count, not at the $65 before the election), but I don’t think anything crucial hinges on this point.
The price changes observed on the day, in both cases, were more drastic than the kind of thing observed on normal expiration days. The implied odds on the early Iraq reports being wrong were around 20 to 1 on, which seems way too high to me.
At this point, I’m not really arguing that there are better forecasters than ‘collective judgements’, just that markets don’t seem to do much better than other kinds of collective judgements, such as a sample of the estimates published on blogs.
Roughly speaking, strong efficiency, which I’m rejecting, says markets are strictly better than any aggregate of public info (including public estimates) as long as there is any relevant private info.
This is the hypothesis relevant, for example, to terrorism futures.
It seems to me you set a ridiciously high standard (gathering all private info), and then declare markets to be useless if they can’t meet this standard. Why isn’t just being somewhat better than other collective judgements good enough to be useful?
“”On two occasions I have been asked [by members of Parliament!], ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.” (Charles Babbage)
I believe we are seeing a very large-scale demonstration of what could provoke such a question.
I should add that if the wise punters reading this blog truly believe that there is a consistent pattern of over-reaction in these markets, you can of course bet against the crowd, which would both make you a tidy profit, and push the markets closer to being efficient.
Just to point out (and this goes some way toward an answer to Robin’s objection) that if you follow the link to our US Presidential elections markets post, you’ll see that Brian Weatherson and myself made a number of specific, timestamped trading calls which if followed would have made serious money. For compliance reasons relating to my job I’m no longer allowed to have an account on Tradesports (they also do financial spread betting which would count as futures speculation), but paper-trading, I’m up bigtime.
“It seems to me you set a ridiciously high standard (gathering all private info)”
I’m always glad to see how ridiculous the strong-form efficient markets hypothesis appears (even to those who like its policy implications) when it is spelt out.
“Why isn’t just being somewhat better than other collective judgements good enough to be useful?”
The cases I cite suggest that market judgements are no better than others. Of course, there’s no way to do inferential statistics comparing these to instances to a sample of an undefined population of bloggers, but as DD says, a look at our posts and the associated links will be of interest.
I should say that, perhaps more than Justin, I think the IEM and Tradesports’ reactions to the exit-poll data count as mistakes and demonstrate that markets are a long way from perfect as aggregating mechanisms. I don’t think it’ll do to say that the market was reacting appropriately to new information — there was plenty of good information available (on Election Day in the U.S.) about the eventual outcome, and the market neglected that in favor of a single data point. That’s a bad performance. But it has to be weighed against a lengthy record of good performances, too.
On a more frivolous note, the Hollywood Stock Exchange runs an annual market in Oscar predictions, and the market was perfect this year, picking eight out of eight winners.
John suggests that market estimates are no better than other collective judgements, “such as a sample of estimates published on blogs”, but that there is no way to do “statistics comparing these instances to a sample of an undefined population of bloggers, but as DD says, a look at our posts and the associated links will be of interest.”
A sample of posts you happen to know of, chosen after the fact for their foresight, is not a useful collective judgment institution! If you ask me for a reliable estimate, before the fact, of if Michael Jackson will be found guilty of lewd acts, I point you to Tradesports.com, now estimating 70%. Anyone can go there 24/7 anytime over the next few months to easily find a clear estimate. What would you suggest someone to do instead - spend all day reading random blogs? If you can’t define your alternative well enough to do statistics, how could someone rely on your alternative in practice?
I think you’re missing the point here, Robin. It’s obvious that betting markets provide a reasonably good summary of aggregate judgements at any given time (subject to various problems like those noted in the post), and if you want information on what those judgements are then looking at quoted odds is a reasonable thing to do. As you say, in relation to the Michael Jackson trial, it’s a lot easier than assembling a panel of well-informed experts (whoever they might be) and polling them.
But that doesn’t imply that it’s a good idea to hand over judgements about terrorist attacks to such markets.
And, on the big questions of policy, it doesn’t imply (in Keynes’ phrase) that making investment decisions on the basis of a casino is a good idea.
To sustain these claims, you need strong or semi-strong EMH.
John, if “betting markets provide a reasonably good summary of aggregate judgements at any given time,” then why must they be semi-strong efficient to be useful for terrorist attacks or the big policy questions? Markets need only be as good or better than our other available institutions. Do you really think we have some other social institution that aggregates all public information on those topics? Where can I now go, for example, to get such a numerical summary estimate of the effect of an immediate US troop withdrawal from Iraq on average terrorist attack deaths in the G9 nations?
Robin, “reasonably good” does not mean “as good or better than any other available institutions”. The latter is broadly equivalent to “efficient” in the sense in which it is used here.
I don’t think the semi-strong efficiency claim (markets are as good at aggregating public info as any available alternative) has been established, either in relation to elections or for capital markets (excess volatility being the obvious problem) In addition, in many cases, and obviously in the case of terrorism, governments have access to information that is not public.
That said, if what you want is numerical summary information about aggregate opinion on factual hypotheses, betting markets provide a convenient source.
À Gauche
Jeremy Alder
Amaravati
Anggarrgoon
Audhumlan Conspiracy
H.E. Baber
Philip Blosser
Paul Broderick
Matt Brown
Diana Buccafurni
Brandon Butler
Keith Burgess-Jackson
Certain Doubts
David Chalmers
Noam Chomsky
The Conservative Philosopher
Desert Landscapes
Denis Dutton
David Efird
Karl Elliott
David Estlund
Experimental Philosophy
Fake Barn County
Kai von Fintel
Russell Arben Fox
Garden of Forking Paths
Roger Gathman
Michael Green
Scott Hagaman
Helen Habermann
David Hildebrand
John Holbo
Christopher Grau
Jonathan Ichikawa
Tom Irish
Michelle Jenkins
Adam Kotsko
Barry Lam
Language Hat
Language Log
Christian Lee
Brian Leiter
Stephen Lenhart
Clayton Littlejohn
Roderick T. Long
Joshua Macy
Mad Grad
Jonathan Martin
Matthew McGrattan
Marc Moffett
Geoffrey Nunberg
Orange Philosophy
Philosophy Carnival
Philosophy, et cetera
Philosophy of Art
Douglas Portmore
Philosophy from the 617 (moribund)
Jeremy Pierce
Punishment Theory
Geoff Pynn
Timothy Quigley (moribund?)
Conor Roddy
Sappho's Breathing
Anders Schoubye
Wolfgang Schwartz
Scribo
Michael Sevel
Tom Stoneham (moribund)
Adam Swenson
Peter Suber
Eddie Thomas
Joe Ulatowski
Bruce Umbaugh
What is the name ...
Matt Weiner
Will Wilkinson
Jessica Wilson
Young Hegelian
Richard Zach
Psychology
Donyell Coleman
Deborah Frisch
Milt Rosenberg
Tom Stafford
Law
Ann Althouse
Stephen Bainbridge
Jack Balkin
Douglass A. Berman
Francesca Bignami
BlunkettWatch
Jack Bogdanski
Paul L. Caron
Conglomerate
Jeff Cooper
Disability Law
Displacement of Concepts
Wayne Eastman
Eric Fink
Victor Fleischer (on hiatus)
Peter Friedman
Michael Froomkin
Bernard Hibbitts
Walter Hutchens
InstaPundit
Andis Kaulins
Lawmeme
Edward Lee
Karl-Friedrich Lenz
Larry Lessig
Mirror of Justice
Eric Muller
Nathan Oman
Opinio Juris
John Palfrey
Ken Parish
Punishment Theory
Larry Ribstein
The Right Coast
D. Gordon Smith
Lawrence Solum
Peter Tillers
Transatlantic Assembly
Lawrence Velvel
David Wagner
Kim Weatherall
Yale Constitution Society
Tun Yin
History
Blogenspiel
Timothy Burke
Rebunk
Naomi Chana
Chapati Mystery
Cliopatria
Juan Cole
Cranky Professor
Greg Daly
James Davila
Sherman Dorn
Michael Drout
Frog in a Well
Frogs and Ravens
Early Modern Notes
Evan Garcia
George Mason History bloggers
Ghost in the Machine
Rebecca Goetz
Invisible Adjunct (inactive)
Jason Kuznicki
Konrad Mitchell Lawson
Danny Loss
Liberty and Power
Danny Loss
Ether MacAllum Stewart
Pam Mack
Heather Mathews
James Meadway
Medieval Studies
H.D. Miller
Caleb McDaniel
Marc Mulholland
Received Ideas
Renaissance Weblog
Nathaniel Robinson
Jacob Remes (moribund?)
Christopher Sheil
Red Ted
Time Travelling Is Easy
Brian Ulrich
Shana Worthen
Computers/media/communication
Lauren Andreacchi (moribund)
Eric Behrens
Joseph Bosco
Danah Boyd
David Brake
Collin Brooke
Maximilian Dornseif (moribund)
Jeff Erickson
Ed Felten
Lance Fortnow
Louise Ferguson
Anne Galloway
Jason Gallo
Josh Greenberg
Alex Halavais
Sariel Har-Peled
Tracy Kennedy
Tim Lambert
Liz Lawley
Michael O'Foghlu
Jose Luis Orihuela (moribund)
Alex Pang
Sebastian Paquet
Fernando Pereira
Pink Bunny of Battle
Ranting Professors
Jay Rosen
Ken Rufo
Douglas Rushkoff
Vika Safrin
Rob Schaap (Blogorrhoea)
Frank Schaap
Robert A. Stewart
Suresh Venkatasubramanian
Ray Trygstad
Jill Walker
Phil Windley
Siva Vaidahyanathan
Anthropology
Kerim Friedman
Alex Golub
Martijn de Koning
Nicholas Packwood
Geography
Stentor Danielson
Benjamin Heumann
Scott Whitlock
Education
Edward Bilodeau
Jenny D.
Richard Kahn
Progressive Teachers
Kelvin Thompson (defunct?)
Mark Byron
Business administration
Michael Watkins (moribund)
Literature, language, culture
Mike Arnzen
Brandon Barr
Michael Berube
The Blogora
Colin Brayton
John Bruce
Miriam Burstein
Chris Cagle
Jean Chu
Hans Coppens
Tyler Curtain
Cultural Revolution
Terry Dean
Joseph Duemer
Flaschenpost
Kathleen Fitzpatrick
Jonathan Goodwin
Rachael Groner
Alison Hale
Household Opera
Dennis Jerz
Jason Jones
Miriam Jones
Matthew Kirschenbaum
Steven Krause
Lilliputian Lilith
Catherine Liu
John Lovas
Gerald Lucas
Making Contact
Barry Mauer
Erin O'Connor
Print Culture
Clancy Ratcliff
Matthias Rip
A.G. Rud
Amardeep Singh
Steve Shaviro
Thanks ... Zombie
Vera Tobin
Chuck Tryon
University Diaries
Classics
Michael Hendry
David Meadows
Religion
AKM Adam
Ryan Overbey
Telford Work (moribund)
Library Science
Norma Bruce
Music
Kyle Gann
ionarts
Tim Rutherford-Johnson
Greg Sandow
Scott Spiegelberg
Biology/Medicine
Pradeep Atluri
Bloviator
Anthony Cox
Susan Ferrari (moribund)
Amy Greenwood
La Di Da
John M. Lynch
Charles Murtaugh (moribund)
Paul Z. Myers
Respectful of Otters
Josh Rosenau
Universal Acid
Amity Wilczek (moribund)
Theodore Wong (moribund)
Physics/Applied Physics
Trish Amuntrud
Sean Carroll
Jacques Distler
Stephen Hsu
Irascible Professor
Andrew Jaffe
Michael Nielsen
Chad Orzel
String Coffee Table
Math/Statistics
Dead Parrots
Andrew Gelman
Christopher Genovese
Moment, Linger on
Jason Rosenhouse
Vlorbik
Peter Woit
Complex Systems
Petter Holme
Luis Rocha
Cosma Shalizi
Bill Tozier
Chemistry
"Keneth Miles"
Engineering
Zack Amjal
Chris Hall
University Administration
Frank Admissions (moribund?)
Architecture/Urban development
City Comforts (urban planning)
Unfolio
Panchromatica
Earth Sciences
Our Take
Who Knows?
Bitch Ph.D.
Just Tenured
Playing School
Professor Goose
This Academic Life
Other sources of information
Arts and Letters Daily
Boston Review
Imprints
Political Theory Daily Review
Science and Technology Daily Review