Via the Marginal Revolution lads, here’s a working paper by Charles Manski, an economist at Northwestern who’s interested in a question that we’ve often returned to at CT; what are the prices in markets like the Iowa Electronic Markets (***MARKET UPDATE*** Kerry still “dying on arse”) actually measuring? Can we really take a market price of 0.70 and unproblematically read off it that “the market thinks there is a 70% chance”?
The central argument of the Manski paper is that, for reasonable-looking assumptions, the market price is (skipping over some tough-looking calculations, which is what you have to do when Acrobat refuses to render math fonts) the average belief of the trading community, weighted by the amount of money each trader is prepared to commit to the market. In Manski’s model, this weighted average is not a meaningful measure of the central tendency of the distribution of beliefs, other than that the true average belief has to fall somewhere within fairly wide bounds of the market price, these bounds being determined by the distribution of the size of trading accounts.
This is a pretty attractive result; it gives some substance to James Surowiecki’s gut feel that preference aggregation mechanisms other than markets might be able to outperform markets, because they don’t have this confounding factor of the size of trading accounts. It also weakens the “arbitrage-based” arguments in favour of markets over other forms of economic organisation, because the distortions introduced by this factor can’t be arbitraged away unless you have information about the distribution of account size which one wouldn’t normally have.
However, I think that the actual state of the world is somewhat better for the markets crowd than Manski’s paper suggests. As I see it, the engine of the paper is that it has two big assumptions, both of which I regard as unrealistic:
1) traders in the market are assumed to be price-takers
2) the distribution of account sizes is assumed to be independent of the distribution of beliefs.
I think that the first assumption is unreasonable as a characterisation of markets in general; it’s a sort of Arrow-Debreu world in which prices are arrived at by a process of tatonnement or sealed envelope auction, and then set “all at once” at a price that equates total supply to total demand. I personally think that this is sociologically an unrealistic characterisation of all market processes everywhere[1], a particularly bad way of describing securities markets in general[2] and, importantly, factually wrong as a characterisation of the IEM.
The Iowa Electronic Markets operate as an open limit order book; people place orders into the market indicating the price and size that they’re prepared to buy or sell. Tradesports is more or less the same. In both of these markets, if you see the current price quoted at 0.58-0.60, you simply cannot place a Buy order at 0.60 that is bigger than the current Ask size[3] at that price. If you want to put more money on, you have to enter a Bid of your own at 0.60 and hope that the 0.58 bid gets taken out. For anyone not familiar with these markets, to whom the preceding will have meant precisely nothing, just take away the view that as you enter Buy orders, you eat up the population of sellers at a price, so if you are putting on a big position like the program trade I dumped on Kerry as a practical joke[4], you end up getting the quoted price only for your first few units and paying worse prices for the rest of your trade.
And when you think about it, this stylised fact means that the size of your endowment[5] or trading account can’t be independent of your beliefs. Assumption 2) was always unrealistic (it’s a feature of Manski’s model that there are no such things as weakly or strongly held beliefs; you just decide on your own fair value and buy units below that price), but in a world of limited liquidity, it’s simply unsustainable.
Consider, for example, a trader with a very large endowment[6] of $500 in his trading account, facing the following order book
Price : Size
50 : 100
51: 50
52: 200
53: 100
54: 50
I’ve only given one half of the order book, because I’m assuming our hypothetical trader (call him “Fatty”) is a buyer. But how strong a buyer is he?
If Fatty thinks that the fair value of the contract is 55 or higher, he will spend his entire $500 endowment in Manski’s model. But if he only thinks the contract is worth 52, then the fact that his account is worth $500 is irrelevant; he’s only going to invest $100 @50 and $50 @51, so his account might as well only have been a $150 one. So I don’t think it makes sense to make the strong assumption that the size of trading accounts is independent of beliefs about fair price; people with extreme beliefs are going to be able to deploy big accounts but people with beliefs near the market aren’t. I’m pretty sure looking at the model that the wide error bands Manski derives are the result of big accounts having a big effect on the demand for contracts, and since the phenomenon I’ve identified would tend to attenuate that phenomenon, I’m guessing that things are probably a little bit better than his paper suggests. But the main point; that one cannot read off IEM prices as straightforward probabilities, in the way that the IEM website appears to be claiming you can[7], is well made and well taken.
Footnote:
[1]Historically, it was this assumption which marked the big split between the Austrian economists and what was to become the neoclassical school. For an Austrian, abstracting from the actual things people do in a market to start talking about a frictionless abstraction of a market, is about the dumbest thing you can do. This is related to the point of Hayek scholarship I keep making; that markets are social institutions that (in my opinion, others differ) can’t be conjured out of thin air.
[2]This view is now mainstream in economics; the field of “market microstructure” deals with the way in which the price discovery process works itself out in securities markets.
[3]Or possibly Bid. I have a kind of dyslexia with respect to these two, and repeated checking doesn’t seem to help. Give us a shout in comments if I got it wrong.
[4]No I didn’t, you conspiratorial bastards.
[5]Sniggering when professors of economics use the word “endowment” is considered bad form.
[6]I said shut up!
[7]The IEM trading accounts size limit of $500 probably helps them a bit too.
{ 25 comments }
Cranky Observer 09.24.04 at 1:17 am
Is there any verifiable evidence anywhere that these so-called Iowa Electonic Markets have any relevance outside the world of the economics classroom? There is after all no real daily market in Presidential elections that compares to the NYSE; particularly given that Las Vegas prohibits betting on that election.
I see an increasing number of references to IEM on blogs, but are the in any way empirically validated? That is, do they have any meaning?
Cranky
James Surowiecki 09.24.04 at 1:27 am
Cranky, what do you mean by relevance? The IEM vote-share market (which doesn’t raise the probability problem the way the Winner-Takes-All market) does performed well — at least by my standards, though Daniel may disagree — against polls in the 1988-2000 presidential elections, and its election-eve forecasts in those elections was off by 1.37% (in absolute terms), which was better than any poll. Of course, as John and Daniel have pointed out, the IEM traders can use the polls as information, but I think the evidence we have suggests that there is additional information aggregation going on. In any case, there are papers here — http://www.biz.uiowa.edu/iem/ — that have more of the details.
James Surowiecki 09.24.04 at 1:36 am
I think Daniel’s second point is right: it’s unlikely that the size of trading accounts is independent of beliefs about fair price. Of course, I see this as the reason why markets are pretty good at setting prices independent of arbitrage, etc.: because I think the market price in a state-contingent security market is closer to the average belief of the trading community than Manski says it is, and therefore closer to the result you’d get via other mechanisms — like the one Manski suggests, which is just asking people what their subjective probabilities are. (Though I think that’d probably work very well, too.)
By the way, Daniel, I thought I was the only one who had ask/bid dyslexia. Good to know I’m not alone.
Jeremy Osner 09.24.04 at 1:54 am
When I have troubles like that, I invent lame mnemonics. So: you Bid to Buy, you aSk to Sell. And when I say lame I mean unworthy of a five-year-old; but this very mnemonic does help me remember what traders are talking about when they specify programs they want me to write.
Jeremy Osner 09.24.04 at 1:57 am
(Further complicating the matter is that many traders say “offer” instead of “ask”, and I have to remember that “offer” means “ask” instead of “bid”, with which it is more naturally synonymous.)
nnyhav 09.24.04 at 4:46 am
Yeah, what I was trying to get at in my bumbling fashion about distro price-dependence last round. The whole digital-option analogy breaks because of the boundedness. Oh, well.
You really want to get turned around on bid/ask? Foreign exchange is for you — market maxim: “Buy high, sell low”.
MQ 09.24.04 at 6:11 am
Daniel and James are always very polite to each other despite the fact that Daniel has uncorked a whole series of intellectually devastating critiques of James’ book. This kind of niceness is rare in the blogsphere, and should be commended.
So James, if what we can get from a market is “close to” the result we get from just asking people their subjective probabilities, then why bother with prediction markets at all? We already have many nice ways to ask people their subjective probabilities of an event without using a market mechanism.
Andrew Boucher 09.24.04 at 6:34 am
“It also weakens the “arbitrage-based†arguments in favour of markets over other forms of economic organisation, because the distortions introduced by this factor can’t be arbitraged away unless you have information about the distribution of account size which one wouldn’t normally have.”
I’m not sure I understand this. Arbitrage as I understand it only needs price information – distortion between prices of two correlated products. The reason why arbitrage doesn’t work at the IEM is that there isn’t enough money to attract arbitrageurs vs. other markets (or vs. other IEM products) and the difficulty in placing orders.
I also didn’t understand the argument about Fatty. If Fatty-55 is willing to spend all of his money, so would Fatty-52. The difference is that Fatty-52 can’t immediate achieve his wish and has to leave a bid in at 52.
Manski’s idea is IMHO close to accurate.
James Surowiecki 09.24.04 at 8:28 am
It is strange to hear someone say that a critique of your work has been “intellectually devastating” when you continue to believe that your work is fundamentally correct. But then it’s differences of opinions that make horse races.
MQ, as you’ll see from Daniel’s post, I’ve never been wedded to market mechanisms. (Daniel writes that Manski’s work supports my idea that “preference aggregation mechanisms other than markets” might work as well as, or even better than, markets.) One of the central ideas of “The Wisdom of Crowds” is that the intelligence you can see in markets is a specific example of a more general phenomenon. I do think markets have certain operational advantages — they provide financial incentives, they allow people to update their opinions quickly and easily, they’re relatively simple to run, they’re anonymous, etc. But when I’ve talked to organizations about using collective decision-making methods, I’ve suggested that they experiment with various methods of eliciting forecasts.
dsquared 09.24.04 at 1:45 pm
they’re not all that devastating to be honest.
btw, “why markets?” for sure, but also “why not markets?”. There is no particularly strong ideological or practical reason to be against markets just because they’re markets.
Alex 09.24.04 at 2:06 pm
Do the probabilities implied by IEM/Tradesports, lacking the risk management value or commercial utility of IR/forex/commodity options &swaps, contain any valuable knowledge to anyone other than punters and pundits? Whether they are efficient or not (and in the sense of no meaningful arbs I think they are) is anyone going to base any worldy decision on their results? arguments toward the IEM’s marginally superior predictive value seem to lack sufficient data, and long haul averages don’t account for some spectacular last minute flameouts (Dean shares spring to mind.) FWIW I have recently found IEM to lag the bigger and deeper Tradesports market.
Speaking as a punter, .075/.080 on Kerry >=52% of the pop. vote does seem awfully cheap, as do Kerry’s odds overall. (Disclaimer: I lost half my account value “catching the falling knife” of Ross Perot shares in ’96.) I strongly some wishful thinking by certain well funded Repubs, or perhaps something more coordinated?
James Surowiecki 09.24.04 at 3:19 pm
“There is no particularly strong ideological or practical reason to be against markets just because they’re markets.”
My sentiments, exactly.
Like Alex, I think the Kerry prices seem awfully cheap, though this may just be wishful thinking. What I remain intrigued by is the fact that although IEM’s WTA market is pricing Bush as a prohibitive favorite (Manski’s strictures aside for the moment), the Vote-Share market is predicting he’ll only get 51.6% of the two-party vote. I realize there’s no obvious way to read off from one market to the other, but that gap still seems pretty wide to me.
Alex R 09.24.04 at 3:30 pm
The other Alex makes a nice point about the lack of commercial utility of IEM and similar markets.
Ordinary derivatives markets have two major types of participants: speculators, who believe they have some superior knowledge that will enable their trades to “beat the market”, and hedgers, who have some extra-market risk that they would like to use the market to lay off. The speculators are, of course, essential for liquidity and to ensure that contracts are correctly priced, but the hedgers are not only a source of stability, but are also a big reason the markets exist at all.
The IEM/Tradesports markets, due to their small size and perhaps other restrictions, will be almost useless for hedging.
For example, suppose that I manage a small defense contractor that expects to make 10 or 20 million dollars a year selling components for experimental missile defense systems to the Pentagon or its contractors. If I think Kerry is likely to kill or cut back the missile defense program if elected president, I might be well advised to try to purchase a derivatives contract paying some millions of dollars if Kerry is elected. But the IEM/Tradesports markets just aren’t big enough for this.
I don’t know what the research says about how the proportion of “speculators” versus “hedgers” affects how derivatives markets work, but I would expect that the lack of true hedgers would make IEM and similar markets quite different from, say, commodites futures markets.
Neel Krishnaswami 09.24.04 at 3:37 pm
This is a naive question, but why is it considered a bad thing that traders in an information market have access to polls and forecasts? One of the classical examples is that orange juice price futures do better than the National Weather Service in predicting cold snaps in Florida, and the reason is precisely that the traders can combine forecast information with other information (which, following Hayek, we might call tacit knowledge).
Alex 09.24.04 at 6:02 pm
nnyhav,
modeling-wise i suspect that your point concerning the shape of the underlying distribution is quite right. I’m not a quant but i recall skew being distinctly important to modeling binaries properly* (see Taleb) more even than volatility. I suspect there exists a model somewhere (maybe Garman Kohlhagen or a variant???) that is capable of rendering premia on probabilities with greater consistency than Black or Black-Scholes — perhaps you know of one? options on ratios having surely been pitched by some bank or insurance company at some point in recorded history.
*in my book, “consistent with other binaries on the same underlier.”
Victor 09.24.04 at 6:16 pm
I disagree strongly with many of your interpretations of Manski’s paper. A few examples:
In Manski’s model, this weighted average is not a meaningful measure of the central tendency of the distribution of beliefs, other than that the true average belief has to fall somewhere within fairly wide bounds of the market price, these bounds being determined by the distribution of the size of trading accounts.
In Manski’s paper, the bounds are determined by the price itself. He makes no statement about the underlying distribution of the size of the trading accounts, nor about the underlying distribution of beliefs.
The easiest way to see this mechanically is in eq. 2 … pi-m is the only variable (pi-m is the market price).
Intuitively, Manski’s point is that the market price only tells you that the seller’s belief was between 0 and pi-m while the buyer’s belief was between pi-m and 1. The market therefore is an imperfect information collector; as a result, information aggregates are likewise imperfect.
Assumption 2) was always unrealistic
Indeed. Assumption 2 forms the best-case argument for the efficient market hypothesis’ conclusion that price is a sufficient statistic for all private information. Relax assumption two, and the case gets worse. See his first paragraph on page 4, or the last paragraph on page 9. Notice that in either event — with a relaxes assumption two or without — the broad bounding problem remains.
This is because the fundamental argument he is making deals with the imperfect collection of information rather than the size and distribution of wealth.
My prior post on this subject (also see the comment section): http://www.deadparrots.net/archives/economics/0409candid_admission_time_what_does_tradesports_tell_us.html
dsquared 09.24.04 at 6:23 pm
One of the classical examples is that orange juice price futures do better than the National Weather Service in predicting cold snaps in Florida
Neel, this is an urban myth, they don’t. I looked into this one a while ago, and so far, the best that the market-ideologues could do (in critiquing Richard Roll’s original work on the FCOJ futures market, which showed that the contracts were far too volatile to be providing any realistic forecasts of supply) was to show that FCOJ futures made a reasonable job of responding to subzero frosts in Florida as they happened, not predicting them.
In fact, as the film Trading Places featuring Eddie Murphy and Dan Ackroyd reminds us, 40% of the entire volatility of the September FCOJ contract takes place on the day when the NWS unveils its Florida growers’ forecast, which would not be consistent with the market having any forecasting advantage at all.
Alex, Nnyhav; there is probably an exact model for options on ratios in Wilmott (my copy is a little bit too far to be bothered getting it), but I would be flat out amazed if B-S gave completely wrong results in the case of the IEM. Given the standard deviations of vote shares, the issue of “boundedness” of the distribution would only become a factor about 5 sd from the exercise price, and I just don’t believe that there is enough state-price out in that part of the distribution to make much of a difference at all.
dsquared 09.24.04 at 6:53 pm
Victor might have a point here: I’ll try and pick up the math fonts and confirm.
Alex 09.24.04 at 7:12 pm
dsquared an example might be instructive here.
an ordinary at-the-money binary call on some generic lognormally distributed future, swap or asset will in the absence of skew always be worth less than the same put. On the order of 55%/45% — try it out, you’ll see what I mean. this as I understand it is a function of the lognormality of the underlier. I’m sure you’d agree such a disparity should NOT exist in a binary market on vote shares. That’s why its especially appropriate here.
Alex 09.24.04 at 7:41 pm
restated by Taleb on pg. 284 of ‘dynamic hedging'(sorry for no link:)
“[an asset’s distribution] will exhibit a bulging right side that would illustrate the lognormality effect… [T]his will result in an increase in the risk-neutral frequency of observations to the left to maintain the fair dice condition in the environment. Such increase in observations is needed to compensate for the difference between the payoff to the right and the payoff to the left. The value of the bet will therefore drop.” A bit confusing and counterintuitive I realize, but obviously accurate when you look at modeled values.
James B. Shearer 09.24.04 at 7:56 pm
Manski’s model is unrealistic in that it assumes people will invest (bet) the same amount regardless of how different the market price is from their estimate of the correct price (true odds). If you assume instead the amount bet is proportional to this difference, then the market clearing price will be the weighted (by amount available to invest) average of the investor’s estimates of the correct price. And the volume will provide an indication of the spread in the estimates of the correct price.
Neel Krishnaswami 09.24.04 at 9:26 pm
Neel, this [orange juice price futures predict cold snaps] is an urban myth, they don’t.
I appreciate the correction. I recall reading about it as an en passant comment in one of the articles in Dennis Mueller’s anthology Public Choice II, and as a result I considered it reasonably credible. So I take it this is a myth in the style of Steven Cheung’s “Fable of the Bees”?
jaytee20 09.26.04 at 5:59 am
I reccomend http://dir.salon.com/tech/feature/2000/11/16/election_prediction/index.html?pn=2.
My condensation: IEM performance was sneer-worthy in 2000. And as long as such a market is relatively small, it is subject to substantial distortion from the source of funds which drive it.
The “psy ops” possibility deserves much more consideration than the preliminary efforts at analysing these markets have contained. The Bush gang likes to project a sense of inevitability, of fait accompli. Given the size of these markets, could the Bush price be propped up, for strategic reasons?
If you study these markets, right now, without allowing for this latter possibility, I say your work is ipso facto useless.
jaytee20 09.26.04 at 6:04 am
I reccomend http://dir.salon.com/tech/feature/2000/11/16/election_prediction/index.html?pn=2.
My condensation: IEM performance was sneer-worthy in 2000. And as long as such a market is relatively small, it is subject to substantial distortion from the source of funds which drive it.
The “psy ops” possibility deserves much more consideration than the preliminary efforts at analysing these markets have contained. The Bush gang likes to project a sense of inevitability, of fait accompli. Given the size of these markets, could the Bush price be propped up, for strategic reasons?
If you study these markets, right now, without allowing for this latter possibility, I say your work is ipso facto useless.
jaytee20 09.26.04 at 6:04 am
I reccomend http://dir.salon.com/tech/feature/2000/11/16/election_prediction/index.html?pn=2.
My condensation: IEM performance was sneer-worthy in 2000. And as long as such a market is relatively small, it is subject to substantial distortion from the source of funds which drive it.
The “psy ops” possibility deserves much more consideration than the preliminary efforts at analysing these markets have contained. The Bush gang likes to project a sense of inevitability, of fait accompli. Given the size of these markets, could the Bush price be propped up, for strategic reasons?
If you study these markets, right now, without allowing for this latter possibility, I say your work is ipso facto useless.
Comments on this entry are closed.