Comments on: Terrorism futures, again

By: robin green

robin green — Tue, 23 Dec 2003 10:51:03 +0000

John – I never claimed that. Please stop putting words in my mouth. I only said that sometimes experts might know better than a whole bunch of non-experts, and that experts don’t always tell non-experts what they know – not, I hope, a controversial starting point. And no, by “free market coreligionist” I meant someone who draws grand conclusions about the power of markets based on over-simplistic theories – and refuses to listen to reason.

By: James Surowiecki

James Surowiecki — Mon, 22 Dec 2003 10:34:44 +0000

No one here believes that markets never get things wrong, so bringing up examples of mispricing isn’t going to move the discussion along. The question isn’t whether sometimes a small group of “experts”—assuming, for argument’s sake, that you know who they are and are not— can make better predictions. It’s whether they can make better predictions more often than markets (or whatever mechanism for aggregating people’s judgments you want to use). The evidence, to me, suggests that they can’t. You cannot use perfection as your standard for whether or not markets are smart unless you’re going to use the same standard in evaluating your experts. As for SCO, do you honestly think the volatility of the stock price of a small company whose entire future cash flow depends on the outcome of an often arbitrary and capricious legal process is a good argument against the relative efficiency of markets?

By: John S

John S — Mon, 22 Dec 2003 09:37:01 +0000

robin green, “…realise that experts sometimes make better predictions because they sometimes have information (or intuition born from not easily distillable experience) that can’t or won’t be freely released to the market” Like the experts who dreamt up the Stability and Growth Pact for the euro? I think Hanson is interesting which makes me, I take it, a free market coreligionist; on the other hand, you must be an undemocratic elitist because you believe policy making is best left entirely to the professionals.

By: robin green

robin green — Mon, 22 Dec 2003 08:47:43 +0000

Oh, and my favourite contemporary counterexample to stock market fundamentalism: SCO’s stock price fluctuations as abysmal indication of its lawsuit prospects against IBM, and indeed its prospects for survival. This will I think show that even when expert analysis (that is, expert relative to know-nothing investors) is *”freely available”* to markets, they can still get it wrong if they don’t listen, and particularly if the media doesn’t listen.

By: robin green

robin green — Mon, 22 Dec 2003 08:35:45 +0000

The trouble with a generalised Ideas Futures market is that can involve ideas which are: (a) seemingly impossible to make rational predictions about – and you’re not going to get a rational collective prediction by mushing together a bunch of irrational individual predictions or (b) subject to significant information inequalities, which don’t tally with power inequalities among traders, or even straddle traders and non-traders (people who know something but don’t participate in the futures market). In other words, if a little player knows something really useful but doesn’t tell anyone else, he may make a prediction but be completely swamped by everyone else stampeding in the opposite direction. What’s the alternative? To (a): stop expecting predictions of specific dates for highly uncertain developments, for example. To (b): realise that experts sometimes make better predictions because they sometimes have information (or intuition born from not easily distillable experience) that can’t or won’t be freely released to the market. Solution (b) seems to be really obvious, and (a) even more so. I don’t understand why Robin Hanson and his free market coreligionists don’t get these things.

By: James Surowiecki

James Surowiecki — Sat, 20 Dec 2003 15:39:12 +0000

The overbetting of longshots (and mild underbetting of heavy favorites) is a well-established inefficiency, which I tend to think is a function of the fact that gamblers have a local preference for risk. And it seems to be the case that the effect is strongest in late races, when bettors are taking a flyer looking to go home a winner. Whether you can systematically make economic profits taking advantage of this is another question. In this case, I think a concern with academic definitions of perfect (even perfect weak-form) efficiency obscures more than it clarifies. What interests me about parimutuel betting is that, even with the slight bias against heavy favorites, win-pool shares generally provide an excellent prediction of the outcome of races—that is, favorites win most often, place horses second most often, etc.—and the subjective probabilities of victory assigned by the bettors are generally close to the objective results (in terms of how often 3-to-1 horses win, etc.). I don’t see the point of discounting that degree of performance because there is this one systematic inefficiency (which is only exploitable in a tiny fraction of races). As far as pro football goes, I’ve never found anything written by Ed Thorp on it, other than a chapter on using the Kelly Criterion to bet. But I don’t think there is a “home team” effect in NFL betting. A recent historical study of the results of a straight “always bet the home team” rule found that, although successful slightly more than 50% of the time, it “has not won often enough to compensate gamblers for the required vigorish.” (Obviously, betting the road team would, therefore, not work either.) When inefficiencies are found, they’re usually quite limited in scope (they also tend to be present late in the season, suggesting that something similar to the longshot effect may be at work). One recent paper that seems rigorous found that betting the home underdog in weeks 15-17 would be a very profitable strategy. (Of course, this strategy was only documented retrospectively.) But even this author argues that “in aggregate, market prices are very close to true values.” Again, all I’m really interested in is whether markets offer a forecast that is, on average and over time, the best one available. In pro football, it seems to be that at least three-quarters of the time.

By: dsquared

dsquared — Sat, 20 Dec 2003 13:33:46 +0000

James, a couple of points: 1. There very definitely are systematic ways of making money on pari-mutuel horse racing and football betting. They’re called the “home team effect” and the “long shot effect” and they’re statistically very well established. Check out Burton Fabricand’s book on “The Science of Winning” for horse racing; I think Ed Thorp’s done one or two things on football. 2. I question the assumption of even weak-form efficiency in these markets; a casual look at the IEM market in the Democratic nominations, compared with http://www.tradesports.com’s market in the same race reveals a number of arbitrages.

By: James Surowiecki

James Surowiecki — Sat, 20 Dec 2003 10:07:26 +0000

John S., although I obviously completely agree with you about the value of markets as information-aggregation mechanisms, it seems pretty clear, as evidenced most recently by the performance of the U.S. stock market from 1999-2001, that the market’s judgment can be wrong for a sustained period of time, even though its wrongness creates, in theory, a profitable opportunity for investors to exploit. Saying markets work well at forecasting the future, and that on the whole they will offer better forecasts than, in this case, polls, doesn’t mean that there are not times when the polls will be more accurate, as seems to have been the case in Australia in the wake of the Tampa incident and 9/11.

By: James Surowiecki

James Surowiecki — Sat, 20 Dec 2003 09:57:52 +0000

John Q., judging from the Wolfers/Leigh 9/30/01 article, the Australian betting market does seem in September to have been relatively slow to incorporate new information relating to the overall popular vote. But to me, the most important piece of data in the original Wolfers and Leigh paper is this fact: the betting market called 43 of 47 individual races, including what W and L call “most marginal seats,” rights. In most of these races, they say, polls were not available (though of course the national polls would be relevant)—and where they were, the betting market outperformed the polls. And the betting market called got the “over/under” line on candidates’ vote shares right in 12 of 12 elections, again in the absence of local polls. That seems to suggest some measure of information aggregation outside of public sources may have been going on. I do think that the way Centrebet seems to have set up the betting is probably not the best approach, and may have something to do with why the market reacted slowly in September. Ideally, you’d like to aggregate all of bettors’ opinions into one number (like the point spread in an NFL game, or the price of a stock), or run a parimutuel system, rather than maintaining two distinct betting pools. In theory, simple arbitrage should make these approaches two systems the same, but the second system is more likely to yield flawed results, because the bettors in each pool (that is, those betting on the Coalition or on the ALP) are likely to be less diverse than a healthy market needs them to be. In the same vein, the IEM would probably be more accurate if it allowed shorting, which I don’t believe it does.

By: John S

John S — Sat, 20 Dec 2003 09:50:26 +0000

“the betting markets did worse than either the polls or expert opinion in absorbing new information” … which means you’re suggesting that market participants can be systematically wrong. I find that hard to believe because such a situation immediately opens up a profitable opportunity for someone to exploit. And what is the expert opinion you’re comparing betting markets with? Here you’re suggesting that there are individuals who can do a better job than markets, the classic argument for planning an economy rather than leaving markets to do their work. And what about a situation where there are no polls?

By: John Q

John Q — Sat, 20 Dec 2003 08:03:32 +0000

James, you have a point, but I think the Australian evidence is relevant here, because the odds changed radically over the year leading up to the campaign, making a comparison easier than in the case when both estimates are very accurate. As the link I put up shows, the betting markets did worse than either the polls or expert opinion in absorbing new information.

By: James Surowiecki

James Surowiecki — Sat, 20 Dec 2003 03:50:29 +0000

John, I think you are holding the IEM to an effectively impossible standard. If the polls are relatively accurate, how could the IEM add (I would say “aggregate”) lots of information that’s not in the polls? In other words, even if there is all this relevant private information or analysis that the IEM is, in fact, aggregating, its effects may be going unnoticed because the polls are, relatively speaking, pretty good. In U.S. presidential elections, the IEM’s forecast has been off, on average, by 1.37% in absolute terms (I’m citing from memory here, so I may be slightly off), which is yes, better than the polls. Certainly the market could do better. But I don’t think it would be mathematically possible for it to do better enough to satisfy what you seem to be looking for. What this means is that your second point (prices would contain little information if there were no public sources) does not follow from your first (your evaluation of the efficiency, or lack thereof, of the IEM). In fact, it’s possible that the less accurate the public sources were, the more valuable something like the IEM would be, because there would be more room for forecasting improvement. In any case, what we know is that the IEM is better than any public source we have (although I’d be happy to see how a weighted-average of polls would do). It seems curious to conclude from this that markets are therefore useless as information-gathering mechanisms. We also know that sports pari-mutuel markets and odds markets, which operate on the same basic principle as the IEM, are relatively efficient. At least in sports like horse-racing and professional football, it is very difficult, if not impossible, to find a systematic means of making better predictions than the market, and in horse-racing, the final odds on horses are relatively reliable predictors of final outcomes. By the way, I don’t really believe that the point of a market is to synthesize expert opinion, at least not in the traditional sense. If you look at the makeup of the IEM traders, especially in the early days, it’s not clear that there were that many experts. I think the reason to have markets like the IEM is that we don’t know, in advance, who has information that might be relevant. The market provides an incentive for people who might have that information to reveal it.

By: Decnavda

Decnavda — Sat, 20 Dec 2003 01:00:21 +0000

john q.- 1. No, the markets do not ADD information, they synthasize expert opinions. 2. Without polls, markets would still provide the BEST predictions, although, yes, they might still not be very good. 3. Weightiing the averages of past polls is a job for experts, who will no doubt disagree as to how to weight them and how far back to go. As a non-expert, I do not know how to judge between them of pull out enivitable baises. Again, if you are arguing that these markets do not provide much useful information for experts, I would probably agree. But I think they are the best way for a non-expert decision maker to see the most honest synthasis of expert opinion.

By: John Q

John Q — Sat, 20 Dec 2003 00:40:23 +0000

To try and clarify my main point, I think the evidence is that prices in betting and futures markets do not add a lot of information to what’s in the polls. If you grant that, then in a situation where there are no public sources of information comparable to polls, prices will not contain much information. Coming back to the Iowa Markets, the analog of the weak efficient markets hypothesis is that the market should do better than any estimate that can be derived from polling data (including past elections). The test mentioned above, outperforming a single poll on the day it is published, is much weaker than this, since it ignores all the evidence from previous polls. Especially where samples are small, you would a weighted average of recent polls will typically do better than the most recent poll.

By: Decnavda

Decnavda — Fri, 19 Dec 2003 21:43:35 +0000

jeremy- There are a ton of Science & Technology claims traded at Forsight Exchange. It’s play money, but yes, people have thought about it.