As far as I can tell, the tracking polls are telling more or less the same story at present. As far as I can see, the punditosphere seems to have got rather ahead of the data here; there’s a potential test of whether they have any actual predictive ability.

(the really interesting thing is that the single most probable IEM outcome is still Bush to win with less than 52% of the popular vote. The big move of the bids has been from Bush>52 to Kerry >52!)

To be sure, the IEM tracks overall votes and Tradesports electoral votes, so these leanings could be consistent. And if Kerry wins the popular vote and loses outright they will be. But that looks rather unlikely. Kerry’s national vote has trailed his battleground states vote in just about every poll that’s looked at this split. This is not particularly surprising since the Bush campaign and its surrogates have massively overspent the Kerry campaign (and its surrogates) on national advertising with Kerry focusing almost exclusively on battleground state advertising.

The IEM numbers are fairly close, but if they hold I suspect one or other (or quite likely both) markets will end up on the wrong side of this election. On the other hand, if Kerry does repeat Al Gore’s efforts and win the popular vote without taking over the White House, I might have to revise my faith in the success of these markets. (Of course if that happens I’ll have much more to worry about than being wrong on a technical question like this one.)

By the way, there might be a small prize for the first CT reader to find an online use of the “see, Kerry agrees with Bin Laden” talking point that is no doubt being lined up on the Mighty Wurlitzer …

The central argument of the Manski paper is that, for reasonable-looking assumptions, the market price is (skipping over some tough-looking calculations, which is what you have to do when Acrobat refuses to render math fonts) the average belief of the trading community, weighted by the amount of money each trader is prepared to commit to the market. In Manski’s model, this weighted average is not a meaningful measure of the central tendency of the distribution of beliefs, other than that the true average belief has to fall somewhere within fairly wide bounds of the market price, these bounds being determined by the distribution of the size of trading accounts.

This is a pretty attractive result; it gives some substance to James Surowiecki’s gut feel that preference aggregation mechanisms other than markets might be able to outperform markets, because they don’t have this confounding factor of the size of trading accounts. It also weakens the “arbitrage-based” arguments in favour of markets over other forms of economic organisation, because the distortions introduced by this factor can’t be arbitraged away unless you have information about the distribution of account size which one wouldn’t normally have.

However, I think that the actual state of the world is somewhat better for the markets crowd than Manski’s paper suggests. As I see it, the engine of the paper is that it has two big assumptions, both of which I regard as unrealistic:

1) traders in the market are assumed to be price-takers
2) the distribution of account sizes is assumed to be independent of the distribution of beliefs.

I think that the first assumption is unreasonable as a characterisation of markets in general; it’s a sort of Arrow-Debreu world in which prices are arrived at by a process of tatonnement or sealed envelope auction, and then set “all at once” at a price that equates total supply to total demand. I personally think that this is sociologically an unrealistic characterisation of all market processes everywhere[1], a particularly bad way of describing securities markets in general[2] and, importantly, factually wrong as a characterisation of the IEM.

The Iowa Electronic Markets operate as an open limit order book; people place orders into the market indicating the price and size that they’re prepared to buy or sell. Tradesports is more or less the same. In both of these markets, if you see the current price quoted at 0.58-0.60, you simply cannot place a Buy order at 0.60 that is bigger than the current Ask size[3] at that price. If you want to put more money on, you have to enter a Bid of your own at 0.60 and hope that the 0.58 bid gets taken out. For anyone not familiar with these markets, to whom the preceding will have meant precisely nothing, just take away the view that as you enter Buy orders, you eat up the population of sellers at a price, so if you are putting on a big position like the program trade I dumped on Kerry as a practical joke[4], you end up getting the quoted price only for your first few units and paying worse prices for the rest of your trade.

And when you think about it, this stylised fact means that the size of your endowment[5] or trading account can’t be independent of your beliefs. Assumption 2) was always unrealistic (it’s a feature of Manski’s model that there are no such things as weakly or strongly held beliefs; you just decide on your own fair value and buy units below that price), but in a world of limited liquidity, it’s simply unsustainable.

Consider, for example, a trader with a very large endowment[6] of $500 in his trading account, facing the following order book

Price : Size
50 : 100
51: 50
52: 200
53: 100
54: 50

I’ve only given one half of the order book, because I’m assuming our hypothetical trader (call him “Fatty”) is a buyer. But how strong a buyer is he?

If Fatty thinks that the fair value of the contract is 55 or higher, he will spend his entire $500 endowment in Manski’s model. But if he only thinks the contract is worth 52, then the fact that his account is worth $500 is irrelevant; he’s only going to invest $100 @50 and $50 @51, so his account might as well only have been a $150 one. So I don’t think it makes sense to make the strong assumption that the size of trading accounts is independent of beliefs about fair price; people with extreme beliefs are going to be able to deploy big accounts but people with beliefs near the market aren’t. I’m pretty sure looking at the model that the wide error bands Manski derives are the result of big accounts having a big effect on the demand for contracts, and since the phenomenon I’ve identified would tend to attenuate that phenomenon, I’m guessing that things are probably a little bit better than his paper suggests. But the main point; that one cannot read off IEM prices as straightforward probabilities, in the way that the IEM website appears to be claiming you can[7], is well made and well taken.

Footnote:
[1]Historically, it was this assumption which marked the big split between the Austrian economists and what was to become the neoclassical school. For an Austrian, abstracting from the actual things people do in a market to start talking about a frictionless abstraction of a market, is about the dumbest thing you can do. This is related to the point of Hayek scholarship I keep making; that markets are social institutions that (in my opinion, others differ) can’t be conjured out of thin air.
[2]This view is now mainstream in economics; the field of “market microstructure” deals with the way in which the price discovery process works itself out in securities markets.
[3]Or possibly Bid. I have a kind of dyslexia with respect to these two, and repeated checking doesn’t seem to help. Give us a shout in comments if I got it wrong.
[4]No I didn’t, you conspiratorial bastards.
[5]Sniggering when professors of economics use the word “endowment” is considered bad form.
[6]I said shut up!
[7]The IEM trading accounts size limit of $500 probably helps them a bit too.

Swift Boats blah, forged memos blah, Kerry campaign blah blah. The latest thing I’m hearing is that some polls are biased against Kerry (I’ve linked to a comparatively sane version of the thesis here; the other kind is not exactly hard to find). Forget about all that, give me something I can work with. One of the most important lessons that experiences teaches you when you start hustling economic data for a living is the importance of picking an indicator and sticking to it. If you choose a number that you regard as representative and look at its evolution over time, you’ll end up getting a feel for the reality that it represents. If, on the other hand, you start chopping and changing the way you look at things, you will, with probability 1, confuse yourself.

The CT fave indicator is the Approval-Disapproval Spread Point Chart. Henry linked to Nasi Lemak’s version to illustrate a point about optical econometrics. I used something similar as the basis of my abortive trading system on the Iowa Electronic Markets. Paul Krugman linked to it once but I can’t be bothered chasing up the link. It’s a good indicator. It was Lee Atwater’s number of choice, and it captures something important about the public perceptions which underly voting behaviour. Sure, it isn’t a direct question about voting intentions, and it doesn’t capture some of the horse race flavour of the headline polls, but I think it would be a mistake to dump it now.

(There will now be a pause while everyone clicks over to Dr. Pollkatz to look at the chart)

Have a look. The really ropey period for Kerry in the polls has coincided with a flattening out and perhaps even a small increase in the Bush A/D spread. It almost looks like this line is having one of its periodic bounces – one might label this one “Republican Convention”, the other two being “Iraq War” and “Saddam Captured”

Anything to do with the Swift Boats would not affect people’s perceptions of George Bush. Similarly, biases in pollsters’ treatment of registered voters for the two parties ought not to show up in the A/D spread figures where the same adjustments aren’t made. Therefore, I would advance the argument that these two phenomena are explanations which cannot explain the whole dataset, and should therefore be discarded in favour of theories which can.

In fact, looking at the data, I am attracted by the following four bald assertions:

1) The recent Bush rally is both within the normal variability of the data and not out of line with past rallies. It just feels so much more important because it’s closer to the election date.

2) The rally is, however, genuine. It’s not likely to be an artifact of polling, and it’s too spread out in time to be linked to any particular blogospheric news story of interest.

3) It is notable that the Bush A/D spread has flattened out more or less exactly when it reached the zero line. This is unlikely to be a coincidence.

4) The temptation is very strong, as noted above, to label the most recent spike upward the “Republican Convention” effect.

I can’t prove any of these, so to some extent I am talking out my arse, and am contributing to the quadrennial pile of election-related bullshit. However, all four of them are at least consistent with the data, which I hope puts them in the top quintile of that pile.

fn1. As in “free to make your own mistakes.”

Here’s another bit of bad news; even at these prices, he’s still overvalued.

Note to readers. There is quite a lot of financial jargon in this post, because I’m dealing with quite a few issues that are only of interest to finance bods (and only marginally to them). The interesting stuff is toward the end.

I mean “Overvalued” in a technical sense here; given the prices of the other contracts trading on IEM, the “DEM04” contract on the winner-takes-all market should be priced significantly lower than the 0.454 which was its price as of the last trade when I was writing this.

I found out this little anomaly earlier this year, when I was playing around with get rich quick schemes[1] and idly wondering whether I could put together a program-trading system that would dump large numbers of trades onto the IEM all at once and make a few of my enemies shit their pants. I assumed it was due to teething troubles as the market for the ‘04 Presidential race settled down, and that things would regularise soon enough.

The issue was, that I was working on some Visual Basic code to try and calculate the “implied volatility”[2] of the vote-share market from the WTA market in order to compare it against the realised volatility. I’ve worked a bit with the Black-Scholes model for pricing binary options in the past, and I know that the sensitivity to volatility of this kind of option is a pretty badly-behaved function, so it didn’t surprise me that the implied volatility formula I’d ripped off from Paul Wilmott’s book didn’t seem to work. What did surprise me is that when I’d changed the code around, put in a more robust optimisation routine, corrected a few of my arithmetic errors and debugged, it still didn’t work. I was buggering around with this, off and on, for about two months, and nothing I could do could get it to work.

So I did what I ought to have done in the first place and drew a few pictures. Below, I’ve plotted a chart of volatility against price. The purple dots represent the Black-Scholes value of a binary call with 0.152 years to maturity (ie today’s date until the election) at an interest rate of 1.75% (CD rate), with a strike price of 0.50 (see footnote 3[3]), with the underlying at 0.491 (the current quote for KERR in the vote-share market), for different assumptions about volatility. The blue dots are simply a horizontal line at 0.454, which is the current quote for DEM04 on the WTA market. “BS valuation” refers to “Black-Scholes“, you cheeky kids.

As you can see, the model value gets close to, but doesn’t reach the market value. In other words, there is no value for volatility which can be plugged into the Black-Scholes model and deliver the market price. Or to put it another way, the option is overvalued.

This isn’t a transient phenomenon, either. I put together a horrendous Excel spreadsheet which takes minutes to calculate, but which produces a line like the one above for every day of data in the life of the DEM04 contract. Below, I’ve plotted them as deviations from the market price of the option on the same day. As you can see, these curves typically don’t cross the x-axis; it’s the rule rather than the exception that the DEM04 contract is structurally overpriced.

So what’s going on here? Seasoned finance pros will already be champing at the bit, ready to tell me that the Black-Scholes model is a completely incorrect model to use in this context. Well, chaps, allow me to differ. Fair enough, if I was acting as a market maker in size for DEM04, I would probably want to use a more realistic model. But I simply don’t believe that B-S would give such wildly inaccurate prices; one of the good properties of the model is that it’s surprisingly robust. And further investigation of the data suggests that whatever’s going on here, it’s unlikely that we can explain this all away as “holes in Black-Scholes”.

For one thing, this is a phenomenon which is almost entirely one of the DEM04 contract. The REP04 contract prices perfectly well in a Black-Scholes framework, for the most part. This allows us to get another handle on the extent of the overpricing of DEM04; we can solve the model for the implied volatility of the BUSH|KERR vote-share contract (which must be the same as the volatility of the KERR contract) and plug this number into the valuation formula for DEM04. Below, I’ve charted this (careful; avert your eyes if you’re a Kerry supporter).

As you can see, the yellow line (reflecting the “fair value” of DEM04; I’ve put this on the right hand scale for some reason) is way below the blue line (its market value). Don’t pay too much attention to the frightening drop-off in fair value in the last few days; the REP04 winner-takes-all contract has risen much faster in value than the BUSH vote-share contract, a phenomenon which could only be justified by a sharp drop in implied volatility[4], which drastically reduces the fair value of DEM04.

In any case, don’t pay much attention to that chart at all because it’s very badly drawn and poorly laid out it’s clear that any attempt to work on the basis of consistency between the DEM04 and REP04 prices is doomed to failure. The two markets are simply not consistent.

Why so sure? Well, they don’t obey a basic and obvious parity relation. Think about it this way. If you were to buy one DEM04 contract plus one REP04 contract, then you would have a portfolio which would pay out $1 for certain in November. How much would you be prepared to pay for this portfolio? Well, basically one dollar, minus the interest that you could have earned by not buying the portfolio and leaving the money in the bank. So in other words, the difference between the sum of the value of (DEM04 + REP04), and 1.00, is the implied rate of interest which one earns in the financial universe of the Iowa Electronic Market.

I’ve plotted this implied money rate below:

I hope it’s clear. In general, over the last three months, the Iowa Presidential Election Winner-Takes-All market has been pricing on the basis of a negative nominal interest rate. This is not consistent with most concepts of market efficiency.

(as an aside, really astute financial econometricians will right now be screaming about “bid-ask bounce”. The idea here is that the chart above has been plotted using closing prices. However, if the last trade of the day in DEM04 was a buy and the last trade in REP04 was a sell, you shouldn’t really just add them together without making an adjustment for the bid-ask spread in the IEM. This is a fair enough point, but I checked that it doesn’t make a qualitative difference. As I sit here typing, the WTA market has bids of 0.454 for DEM04 and 0.556 for REP04. That means you could sell one of each and get an interest-free loan of a buck.)

So what does this all mean?

Well first, on a practical note, it seems to me that if you want to back the Democratic candidate in the 2004 Presidential elections, you should do so by selling REP04 rather than buying DEM04, and vice versa; this way you take advantage of the fact that option premium is overvalued in DEM04 relative to REP04.

Second, on a more theoretical basis, to my mind this is, unless I have made a howler somewhere, more or less conclusive proof that the IEM is not “efficient” in the financial economics sense of the term. A market in which basic parity relations are not observed is not efficient. One can make all sorts of excuses about the amounts of money at stake, the restricted universe of traders, the fact that short-sellers on IEM do not have full use of funds, etc, but the plain fact of the matter is that the prices on IEM are not consistent with the time value of money.

Third, we can pick up some clues as to what’s going on here from the prices themselves. Although the parity relation does not hold, casually going through the numbers suggests to me that, most of the time, the mid-price of DEM04 plus the mid-price of REP04 adds up to 1, near enough. So there’s a sort of parity rule being enforced by the market participants. That suggests to me that pricing here is being driven by a social convention that the prices ought to reflect people’s estimates of the probability of Kerry or Bush winning. This isn’t how stock options are priced. Financial options are priced on the basis of Black-Scholes and similar models, via an arbitrage pricing argument. Donald Mackenzie studied the sociological process by which financial markets moved from a “probability-pricing” norm to an “arbitrage pricing” norm, and this doesn’t seem to have happened in the IEM (yet; I’ve not entirely given up on my program-trading operation!)

This suggests that James Surowiecki is right and Robin Hanson is wrong on the way in which “information markets” of this kind work. (Update: In comments below, Robin Hanson argues pretty convincingly that this is a pretty egregious caricature of his views. Sorry Robin.)In comments to my review of James’ book, the two of them outlined the difference in their views. James fundamentally thinks of markets as a way for people to “vote” and aggregate their views, with any predictive power they have coming from the fact that they’re a kind of crowd. In particular, for James Surowiecki, markets are just a handy way of organising information aggregation; voting and other ways of summarising crowd opinion might work just as well.

Robin Hanson, on the other hand, appeared to be giving a much more particular role to markets as opposed to any other kind of social organisation. He commented that the reason why markets generated information was ” simple idea that anyone who notices a mistake in the price of a speculative market can make money by fixing that mistake.” To me, this seems like he’s committed to a view that efficiency in the sense of obeying parity conditions is the mechanism by which markets gather information.

This seems like a trival distinction, but it has big practical implications. In particular, if you believe in something like Robin’s view, then you would say that the maximally informative market prices are the most recent ones (because any difference between the prices and the state of the world is most likely to have been arbitraged away). If you believe in something like James’ view, then you might say that a more informative view of the opinion of the crowd might be a moving average of, say, the last five days’ trading. Personally, I’m still something of a sceptic about whether ‘toy’ markets of this kind really deliver the goods at all, and I must say that this exercise hasn’t exactly made me a believer[6]. But it seems to me that if markets like this work at all, they have to work under the Wisdom of Crowds model rather than the Theory of Sharks. And since all the big financial markets operate on the basis of “sharks” and arbitrage pricing, we need to separate them in our minds from toy markets like this; markets which don’t have a Hayekian reason to exist shouldn’t draw credibility from markets which do.

Footnotes:
[1]I promise that this system was in profit when I stopped the experiment; I moved jobs and left all the files behind. My guess is that it would be in loss at the moment, since it seemed to be pretty structurally long Kerry.
[2]Basically, the WTA contracts can be considered to be options (specifically “cash-or-nothing” binary options) written on the vote-share contracts with a strike price of 0.50. Option prices depend on the anticipated volatility of the underlying, so the “implied volatility” is the volatility number that you plug into the formula to make the model price consistent with the market price.
[3]As discussed here, this is consistent with the contract specification. The Iowa vote-share market is for a two-horse race, and the “winner” for the purposes of the WTA contract is the winner of the Iowa VS market. The IEM actually paid out on Gore in 2000 for this reason.
[4]As in, loosely speaking, the Bush contract is currently “in the money”, so volatility in the underlying hurts the binary option more than it helps.[5]
[5]Readers may think that this divergence between the vote-share and WTA contracts is unlikely to reflect a genuine decrease in uncertainty about the vote-share and may represent an investment opportunity. Since CT isn’t in the business of giving advice, readers who want to pursue that line of thought are on their own.
[6]A lot of the problem is that it simply isn’t profitable enough to arbitrage away the breaches of parity, unless you’re doing it just for the fun of proving a point. But I don’t think this is the only problem. And furthermore, even if it was the only problem, it appears to me to be a more or less insoluble one, unless we really believe that in the future we will live in a world where material proportions of peoples’ savings will be diverted away from productive enterprise and into these zero-sum games.
[7]I hereby claim the CT title for “Most Footnotes”
[8]And “Most Gratuitous Self-Citations”.
[9]In response to comments, here’s a quick plot of the sensitivity of the option to the volatility assumption (vega). This is plotting the first derivative of the value of the option with respect to volatility. If this doesn’t mean much to you, just laugh at the grotty Excel.

In 1762, Georges-Louis LeClerc, the Compte de Buffon, threw a hundred baguettes out of his laboratory window and watched where they landed on the courtyard below. He hadn’t gone insane and he wasn’t making a statement about the French baking industry. He was carrying out a whole new experiment and revolutionising the way that we look at mathematics, information and breadsticks.

This was what Buffon did; he measured the baguettes before throwing them out of the window. When they had finished bouncing around, he counted how many of them were lying across cracks in the paving stones. He then divided the length of a baguette by the number that were lying on the cracks and multiplied by 200. The number came too … 3.1415927! Yes, while his contemporaries at the Encylopedie were hard at work using conic sections and convergent sequences to establish the value of pi, Buffon had it worked out more or less accurately using yesterday’s lunch leftovers. Although no individual loaf of bread had any particular information about the value of pi, when you aggregated the little bits of information contained in the length and falling of each baguette, something amazing happens. And even more amazingly, you can repeat the same experiment with needles, cucumbers, rolling pins or even everyday American lumps of wood, and get the same result! This phenomenon is what I call “The Wisdom of Sticks”.

Well that’s somewhat unfair (although truth is stranger than fiction; you can actually estimate the value of pi by throwing sticks onto a pavement, and James has about a hundred historical anecdotes like this one). But it’s a somewhat pointed parody, because there is quite a lot of material in TWOC which, while entertaining, is pretty tangential to the real underlying point. I’d group this material into two main categories.

Party tricks: As my example of the Buffon problem suggests, statistics can throw up some funny old things. For example, you and I are playing a simple coin tossing game, best of twenty, and I win the first toss. I offer you a side-bet that you will not take the lead at any time in the next 19 tosses; if I can get odds of 3-1 or better out of you I’m a winner. There are a number of startling demonstrations (people guessing the number of jellybeans in a jar[2], for example) which look to me like carefully designed tricks with sampling theory rather than informational magic. Looking at the working paper describing the much-vaunted Hewlett Packard forecasting market, I’d be very tempted to place it into this category as well; the result that “the market did better than the official forecast” (Result 3 in the paper) is based on comparing a probability distribution of market prices with a point estimate for the official forecast and really doesn’t sum up the actual findings; that the official and market forecasts were very close to one another indeed.

Semi-attached anecdotes: Stories which are interesting, but which aren’t relevant to the main thesis of the book and have been uncomfortably shoehorned into the argument. Examples; the review of the invention of the motor car (an example of the wisdom of crowds – sure, Henry Ford invented mass production, but it was crowds of people who bought his products), the material on the Linux operating system (an informal hierarchy of people who use email and call each other ‘dude’ is a hierarchy, not a crowd). There are quite a few of these, and as you read the book, it’s worth thinking from time to time something along the lines of “but what the heck has this got to do with the wisdom of crowds?”

So when you strip away these ephemera, is there anything left? Well yes. In particular, monetary policy is better made by committees than individuals, in studies that can’t be explained away as party tricks. I’m also prepared to give a certain amount of credibility to the idea that pari-mutuel betting markets forecast horse race winners very well, although only a certain amount, because there’s equal evidence that they don’t forecast places and shows anything like as well. But the important thing to note is that TWOC does, at its core, describe a genuine phenomenon. In some circumstances, groups can outperform individuals at some kinds of decision problem. For interesting values of N, N heads are better than one. Where does the book go with this concept?

As I see it, James’ book goes in two directions as regards actual policy suggestions. As regular CT readers will be expecting, I don’t really agree with either of them, but one is very much more interesting than the other. I’ll call the first idea “market epiphenomenalism” and the second, more interesting one the “silver bullet theory”, for reasons that I hope will become clear.

Market epiphenomenalism

“Market epiphenomenalism” is my attempt at a name for what Robin Hanson thinks about markets; in my view it’s based on a misreading of Hayek’s original work on markets as information processing entities. To recap my argument from the post linked above, Hayek certainly did believe that markets were information-aggregating entities, but this view can only be understood in the context of his whole philosophy. He certainly didn’t think that you bring your information to the market, I bring my bit of information and that the market exchange process grinds them all up and turns them into one wonderful summary statistic called the “Market price”. Markets don’t “aggregate information” in this sense; the sense in which they aggregate information is that they co-ordinate actions without the existence of any centralised information. To make this more concrete, think of the Google IPO. People were bidding in that auction, but they aren’t setting the price (the price was being set, and suddenly reduced, by the company). What they’re doing is bringing along their little bits of capital and agreeing a set of terms on which they’re prepared to part with it. The market is aggregating these little lumps of capital into one big lump that can finance something interesting, and that’s what the market is there fore. The fact that it does so at a particular price, and that this price needs to, over the long term, bear some relation to reality if the market is to survive and keep performing its function, is an epiphenomenon.

This important point from Hayek has two implications. First, that there’s much more to a market than the prices, and it’s dangerous to kid yourself that the prices, hauled out of the context of the rest of the market, can stand as information on their own at all, much less as information of the kind that you need to solve your own decision problem. For one thing, the participants in a market are likely to have loss functions for their forecasting problem which may not be the same as your own; for example, if you’re forecasting sales, you might want to make forecasts which are systematically too high (in order to set stretch targets for your salesforce) or too low (because overruns are more expensive than underruns for you). It’s highly unlikely, for example, that traders in the Policy Analysis Market would have the same attitude toward different kinds of forecasting errors as the policymakers themselves.

The second and more important point is that if Hayek is right, you can’t just will markets into existence by saying “hey I’ve designed a few state-contingent securities, let’s gamble”. The reason why markets work is that they are performing a function, and that they are bringing people together whose little bits of capital add up to the whole investment capital of the economy, or whose total short positions add up to all the grain that has been grown. Opinions are like assholes in that everyone has one[3], but unlike assholes in that it’s very easy to pluck a bunch of them out of thin air on a whim. Markets like PAM which have no real underlying need to exist, aren’t information processing entities in Hayek’s sense because they’re not co-ordinating actions, and thus attempts to use their record of closing prices as if it were information are doubly problematic. I note in passing that this isn’t a trivial point of Hayek scholarship and I don’t think any counterargument on Hayekian grounds is possible, because the point is fundamental to the Austrian critique of planning. After all, if you can construct a Policy Analysis Market out of thin air and gain all the Hayekian advantages of information processing, why couldn’t the Central Planning Committee just set up a bunch of these “shadow markets” and run communism on the basis of the output? If the Austrians were right about the planned economy, one of the corollaries has to be that toy “markets” are closer to children playing at shops than to the real thing.

But it has to be said that this is more of a hobby horse of Hanson than of James Surowiecki. At some points in the book (and more clearly, in some comments he’s made while arguing on the internet), he backs away from straightforward market idolatry and toward something much more interesting; a general theory about preference aggregation[4]. And that’s what I’d call the “silver bullet theory”.

The Silver Bullet?

The phrase “silver bullet”, in this context, comes from Fred Brooks’ book, The Mythical Man Month. In one of the articles which makes up that book, Brooks made the prediction that, as of the 1980s when most of the merely accidental problems of computer programming (bulky interfaces, lack of high-level languages, sheer novelty, etc) had been got rid of, there was now no further development at all which could promise even a single order-of-magnitude improvement in the productivity of software systems products[5]. For the last twenty years, the silver bullet prediction has looked pretty good (for about five minutes around the time of the VA Software IPO, some people thought that the Linux development model was a kind of silver bullet, but over time, I think it’s grudgingly been accepted that ten years to build a decent UNIX clone is pretty good for free, but it’s not an order of magnitude improvement.

There are a number of reasons why Brooks made his prediction but the core thesis of his book with respect to software development is that it is an “unpartitionable problem”; that software development tasks cannot be divided up among people very well and that for this reason there are fundamental cognitive constraints on the speed of software development. Under the classic model, most decision-making roles are also unpartitionable; the buck has to stop in some specific place, rather than having a nickel stop in each of twenty different places.

The thesis of TWOC, on the other hand, is that decision-making is a much more partitionable task than it is commonly supposed to be, and that it should be partitioned much more. As you can see, this comes close to suggesting that there is a “silver bullet” for a raft of organisational problems; that if we were to structure our businesses and governments so as to maximise our usage of the wisdom of crowds, we would get much better decisions for less or the same effort. Although James doesn’t actually say as much in the book, there are already some breathless futurologists out there who see a world in which nobody does normal white collar jobs any more, and instead we all sit around trading futures on our company’s internal markets. Sort of “The New Industrial State meets Trading Places.

Hmm. There is a lot of sense to the underlying idea; that people making decisions ought to discuss things with one another, and that effective communication can help groups outperform even their best members; the example of the monetary policy committees is a good one, and the book provides a pathological example of the lack of use of group discussion which helped lead to the last Space Shuttle crash. This is something which I agree with, and which is terribly important.

But I don’t like the actual “silver bullet” that James proposes. If I read TWOC correctly, it’s arguing for something a bit more radical than people talking to one another. Like anyone who’s ever said “why don’t we have a rule that you can only talk if you’re holding this bunch of keys” to try to forestall a nasty argument, the author of TWOC seems to believe that problems in human communication can be solved by simple institutional means. In the case of TWOC, the silver bullet is something called “preference aggregation”. This doesn’t have to be a market price; voting procedures are also preference aggregation, as (presumably) opinion polls could be; I understand from internet comments that James believes the best preference aggregation mechanism of all to be the hybrid market-like system used by the Tote and by pari-mutuel horse betting. Anything that takes the opinions of a group of people and distils them into a single number, with that number representing concentrated Wisdom of Crowds would suffice, though.

What’s wrong with preference aggregation?

I don’t like this idea. Specifically, it seems like too much of a free lunch. You take a bunch of views which contain some information, compress them down to a single number, and somehow come out with more information (or more usable information, which amounts to the same thing) than you started with. Granted, you can throw a basketful of bread off a roof and come up with an estimate of pi, but genuine free lunches like that are few and far between. Aside from this, I think that there are a two very serious problems with preference aggregation as a solution to organisational problems:

First, they’re black boxes. However partitionable decisions can be made, they’re not perfectly partitionable. In general, the bigger and more important the decision, the less partitionable it is. Which means that, in most important or interesting situations, aggregated preferences are going to be a decision support tool for a particular decision maker. And decision support tools shouldn’t be black boxes. I’ve spent quite a percentage of my professional life over the last ten years arguing this point and by now I think it’s won. Any improvement in decision quality that you get from a black box is more than outweighed by the fact that you can’t explain the output of the tool. You can ask questions of a committee, but you can’t ask a market why it thinks what it thinks. For most practical purposes, this is likely to mean that there would have to be an absolutely huge gain in information for it to be a good idea to move to aggregated preferences as a decision support tool.

Second, I think that the preference aggregation solution encourages pathological behaviour. Even assuming, in my view per impossibile, that careful design of the aggregation mechanism can get people to honestly estimate their information rather than “gaming the system” by misrepresenting their own views, there is a much more fundamental problem. A lot of TWOC, and for my money some of the most valuable insights in the book, is taken up with describing the conditions under which crowds perform really badly – market panics and groupthink. In order to avoid these pathological outcomes, the book is very clear that it is vitally important that you have a group of genuinely independent individuals, aggregating views which they have arrived at by thinking for themselves.

I happen to agree massively with James on the subject of it being vitally important that people think for themselves. But surely it sends entirely the wrong message to people if you’re on the one hand telling them to think for themselves, and on the other hand presenting a magical silver bullet/black box aggregation mechanism to which the eventual decision is going to be delegated? There are real-world examples of this pathological outcome at work; in particular, the Federal Reserve did not take any action to pop (or even to publicly criticise after 1996) the stock market bubble because it was not prepared to set its own judgement against that of the equity crowd. In general, I would argue that monetary authorities pay far too much attention to interest rate futures. There is a strong incentive created by all of these aggregation mechanisms for people to free-ride on the mental effort of others.

So what to make of it?

As I say, TWOC is a good book, but one that I think should be read critically rather than credulously. Crowds can give you very good answers if you know what questions to ask and if you know the correct way to interpret the answers (so could Buffon’s loaves of bread, mind). The wisdom of crowds should not be ignored, and it’s certainly a good idea to listen to what people are saying, but there is no magic solution here, and you shouldn’t delegate your own thinking to it. Even Humphrey Neill, author of “The Art of Contrary Thinking (the book in which the magazine cover indicator was first trailed) used to believe that the crowd was right most of the time. Specifically (and to be fair, in the context of stock market investment, this is close to being an analytical truth), the crowd was right during trends and wrong at turning points. I think that this captures an important part of the truth behind the “Wisdom of Crowds”; crowds are very good at answering questions, but not so good at deciding what questions to ask. Or another way to put it is that, whatever the I Ching might claim, you can’t tell the future by throwing sticks.

Footnotes:
[1] Don’t know if it’s still there, but as of this morning, Kerry was being quoted at 0.51 in the vote share market and 0.49 in the winner-takes-all market. As I’ve noted before, this isn’t anything to do with the electoral college; despite the name the Iowa WTA market is simply a binary option written on the vote share contract with a strike of 0.5.
[2] By the way, I’ve been in a lecture hall where the jellybean experiment was tried and it failed rather embarrassingly. As the theory predicts it will, every now and then.
[3] And that in general, nobody’s all that interested in anyone else’s
[4] The main reason why James seems so keen on markets is that he believes that they aggregate the preferences of all participants rather than just the marginal buyers and sellers. I think that this view is both mathematically and economically incoherent, but I’ve snipped the discussion of “Surowiecki on market microstructure” for reasons of space and time. I think my main objection can be summed up in a joke I made some while ago in CT comments that if people who neither bought nor sold a security were contributing to the eventual price by their abstention, then it seems to follow that there are Tibetan monks who have never heard the name of Jesus Christ but who were nevertheless part of the price discovery process in the Google IPO.
[5]”Software systems products” is a very specific definition in the context of Brooks’ book, btw; whatever your counterexample, he’s probably thought of it.

Some would say that this is fighting talk, and that if we really thought this, we ought to be trying to make some money out of it. So here goes …

Big thanks to Nasi Lemak for sharing a dataset of historical poll data with me. I have used that data to construct and backtest a trading system for the IEM Kerry vote-share contract (KERR) which uses only published poll data and generates favourable backtesting over the last four months. The equity curve for this system so far is below the fold; I plan to use it to trade the IEM vote-share market over the rest of the campaign.

I’m not giving away any details of the system just yet; I’ll reveal them all at the end of the experiment. When the winner-takes-all market is launched, it is quite likely that I will bring another system into play based on trading the WTA contracts as if they were derivatives on the VS market, however that will not affect the current system which is entirely based on publicly available poll data. If I refine the system (currently it only makes use of Bush’s “Approval” ratings, whereas going forward I’d like to drive it off the approval/disapproval spread. I’d also like it to be able to double up and reduce bet size; currently it’s a pretty crude on/off switch), I’ll keep a record of what I did and when. But the proof of the pudding will be in my final P&L; I think that the IEM follows polls rather than leading them, and this is my way of finding out if I’m right. I’ll update as and when the system suggests I trade.

As of today, as the chart above shows, the system is long KERR, which I bought this morning at $0.495. I’ve entered into this trend fairly late, so I’m more or less expecting a few drawdowns early in trading, but if the system works, it ought to be well in profit by November. Wish me luck …

1. In the 1996 Presidential vote-share market, after the candidates have been nominated and adopted, what should the sum of the values of the CL|DOLE plus V.DOLE (Dole vote share) contracts be?

2. What percentage return would you have made in the 2000 winner-takes-all market by buying the BUSH contract at the point when DEM was at its peak and holding to maturity?

3. You hold a porfolio in the current 2004 Presidential vote-share market long BU|KERR but short BU|CLINT. If George Bush were to announce tomorrow that he had decided to withdraw from the race, what would be your profit or loss?

Answers below the fold. Historical price and prospectus data available on the IEM website.

These are all trick questions, in case you hadn’t guessed …

1. In the 1996 Presidential vote-share market, what should the sum of the values of the VS_CLINT (Clinton vote share) plus VS_DOLE (Dole vote share) contracts be?

If you answered 1.00, 100% or some similar, you’re wrong and if you’ve got a finance qualification you should give yourself a slap on the wrist. That’s one answer that has to be wrong.

If you answered “100% minus Perot’s share of the vote”, then good try, you’re thinking, but wrong. The Iowa vote share markets are the share between the Republican and Democratic candidate; shares of third-party candidates don’t count. This is sometimes very relevant indeed; see below.

The correct answer is 1.00 minus the time value of money. Because you have to pay your money upfront to receive the return after the election, a portfolio of 1 Dole plus 1 Clinton should sell at a discount to face value reflecting that fact.

2. What percentage return would you have made in the 2000 winner-takes-all market by buying the BUSH contract at the point when GORE was at its peak and holding to maturity?

You would have lost all your money. The 2000 winner-takes-all contract paid out on Gore, not Bush. There are two factors at work here.

a) Although it is called “winner takes all”, the WTA market is not actually a betting market on the winner. It is a market in binary options which pay 1.00 if the underlying vote-share contract is above a strike price of 0.50.
b) Gore won the popular vote by a small margin. If you strike out Nader’s votes (as you have to for the VS market calculation [Update: Oh do you really, laughing boy? See below]), this margin became signficant.

3. You would neither win nor lose anything. BU is simply an identifier meaning “the Republican candidate”.

All of which is by way of suggesting that, whenever you’re doing research on any kind of financial time series, you need to be very careful indeed about the specification of the contracts involved. Lots of academic studies comparing mutual funds to the S&P500 index do not observe this basic requirement, by the way.

UPDATE Actually, my answer to 1 was only correct for the 1996 and market. In 1992, there was a Perot vote share market, and in 2000 a Buchanan VS market and both times the vote shares were calculated three ways. Consider me slapped.

Can we no longer hear about the “predictive power” of the Iowa Electronic Markets, please? They were bamboozled to exactly the same degree as the rest of us.

[UPDATE]: A couple of people in comments have pointed out that this market is for the nomination, not the Iowa Caucus itself. Fair point, but sadly, no. Either the Iowa Caucus is an important determinant of who gets the nomination, or it isn’t. If it isn’t, there shouldn’t have been anything like this sharp movement on the 19th. If it is important, then trading in these contracts ought to have reflected relative chances of winning in Iowa. Either way, big spikes like this on news days are not consistent with semistrong market efficiency. I’d also note that the Iowa Electronic Markets are strongly linked with Iowa University’s business school, so the Iowa caucus is their best chance of having local tacit knowledge. While we’re noting things, I’d make a few points on the alternative prediction methods. The Irish Independent’s online poll seems to have done at least as well as IEM if not a little better (fair enough, I don’t have a time series for this one), and BBC Newsnight ran a big feature on Kerry last week; they’d clearly picked up the buzz.

Bonus Arbitrage Update: Mark Kleiman is posting the IEM and Tradesports numbers every day on his weblog, and I haven’t seen a single day so far when there hasn’t been at least one arbitrage between them. Today’s is Buy Clinton@1.8 on IEM and Sell Clinton@2.5 on Tradesports. (NB this isn’t investment advice). I’d also point out that the arbs aren’t always this small and aren’t always in the small long-odds contracts; on Tuesday you could have bought Kerry@33.8 on Iowa and sold him @ 36.1 on Tradesports. I haven’t got an Iowa account and can’t be bothered getting approved to have one, but anyone who cares can make a few bucks a day trading the two against each other.

One might think that, but one would be wrong. In actual fact, it is possible to trade futures on the US Homeland Security indicator at Tradesports.com. So, since the threat level was raised from “Yellow” to “Orange” over the weekend we can go to the tape and see whether the traders there had any advance steer on this movement. Did they?

Did they, to put not too fine a point on it, my arse. Here’s the contracts page; a little playing around with Tradesports’ Javascript navigation reveals that, overnight, the “Yellow” contract gapped down from 86 to 12 while the “Orange” contract gapped up from 11 to 88. In the days immediately before the raise, Orange had been gradually declinign and Yellow had been gradually appreciating. I can only conclude that traders on the Tradesports contract had absolutely no additional information relevant to forecasting the Homeland Security threat level. I’d also point out that if any CT readers fancy having a punt that the threat will stay Orange for the remaining nine months of the year, they certainly should not buy Orange at 88; selling Red at 5 and Yellow at 10 gives you a better profit if Orange persists, plus a free bet on Blue and Green. It doesn’t say wonders about the efficiency or liquidty of the market that it has glaring anomalies in it like that (it’s also possible to bet on the Lord of the Rings film taking more than $116m over the weekend, which it did, and you can also have a look at a US Economic numbers book in which the probability of the Personal Incomes number being above 0.4% is quoted at 55-65, while the probability of it being above 0.7% is quoted 61-70. (Update: Or at least it was before I took out some sucker’s bid; if you want to fact-check my ass, email me and I’ll send you the screenshot).

The problem here is that the Tradesports financial and political books aren’t really all that deep or liquid, compared to their sports books. There just aren’t enough sharks like me preying on silly or late bids and offers to keep the market respecting basic pricing identities, let alone the millions of savvy traders “aggregating information” that the market-philes dream of. If you’re interested in the Democratic Nomination race, for example, it’s possible to buy Joseph Lieberman at 1.3% on the Iowa Electronic Market and then sell him short at 2.3% on Tradesports; you can also buy Gephardt in Iowa for 7.6% and sell him on Tradesports for 9%, among other intermarket arbitrages, but I for one could not be fagged registering for both sites in order to do so. I’ve argued in the past that an adaptation of Hayek’s critique of planning suggests that these playing-at-shops markets don’t work, but I now think that in practice, the real problem is simply going to be lack of interest. Every year, futures contracts are launched on actual things where there are genuine buyers and sellers on both sides, but fail to take off through lack of open interest. It’s much less likely that a purely speculative contract is going to attract enough liquidity for price discovery to be meaningful, and the Homeland Security performance suggests that the traders don’t have any useful information to aggregate.

Three endnotes: First, all price-related discussion above was clearly based on prices on my screen as I looked at them; it is likely that they will have changed by the time you read this. Second, Crooked Timber is not in the business of provding investment advice for betting on Tradesports, and any spread beeting CT readers carry out should only be done after full investigation of the risks involved. And third, I at least will no longer be entering into discussion on the subject of whether or not trading exchanges might “add something valuable to the process”, because the empirical evidence is that they don’t. Anyone wanting to convince me otherwise is going to need to cough up actual data; in particular, I reserve the right to gently mock anyone who claims that some hypothetical i>other idea futures market would be materially more liquid than Tradesports without giving a good reason why (endorsement by the government does not constitute a good reason). If I get time over the holidays, I will muse on why it was that so many people believed so passionately in this dead duck of an idea.

Finally, I am positive that Tradesports used to offer a “WMD found in Iraq” contract but I can’t find it for the life of me. I’ve sent them an email about it and will report back.

As it happens, I’d already looked at this and come fairly rapidly to the conclusion that the betting markets weren’t much good, so I was struck by the money quote from author Justin Wolfers

The data suggests the Australian betting market is extraordinarily efficient. And why not? There’s a huge incentive for participants to do their homework, collect reliable information and make sure the price is right.”

Looking at the report and also the Iowa studies, the evidence in support of this claim still seems very weak to me. In 2001, for example,

The night before the election, Howard [the incumbent Liberal PM] was ahead in two of three major polls ….[on Centrebet] Howard was the favorite with odds of $1.55, suggesting a 64 percent probability of his winning the election,”

To be fair, Wolfers doesn’t put much weight on the election-night market. He says

data from Centrebet, Australia’s largest bookmaker, demonstrated the impact of current events on the betting odds throughout the nine months leading up to the election, reflecting immediately the electorate’s seesawing response to such events as leaders’ televised debates and the Sept. 11 attacks in America.

But enough of this ad hoc dispute. What test should we be applying here? It’s not appropriate, as nearly everyone in this field has done, to treat polls and betting markets as separate predictions. Punters in the betting markets have access to the polls. So they should do at least as well, on average, as any mechanical rule based on poll data. The test “have the markets done better than the polls” implicitly compares the actual betting strategies to the rule “at even money bet on whichever candidate is ahead in the polls”. Even compared to this simple-minded rule, the improvement shown by the markets is marginal at best.

The real issue we should consider, before rushing to embrace terrorism futures and the like, is how betting markets would perform in the absence of information from polls. You’d have to go back before World War II for this, but it’s my impression that predictions of election outcomes in this period were often way off the mark