Back in December I wrote:
”[…] the proposed “Policy Analysis Market” (which claims on its website that it’s going to launch in March; sadly there is no currently existing futures market which allows me to bet that it won’t)
Historical note; it didn’t.
BTW, for those who care about that sort of thing, while we’ve expressed plenty of scepticism over the marginal value of election betting numbers in the past, they are probably no worse than polling numbers and available with greater frequency. Bush is currently more or less holding steady on IEM, but weakening on Tradesports. Note that these two figures are not directly comparable, as the IEM contract is for vote share while the Tradesports one is “winner take all”.
Since the funding was pulled in June of last year, its no wonder you won your bet.
But to the point of the PAM, it’s a shame that you won your bet. Futures markets, even with play money, have shown incredible insight into trends. Examples (with a nod to Eli Lehrer):
Is there anywhere online I can bet actual cash money on Osama bin Laden being produced, dead or alive, in the month of October?
(Of course, I also need persons willing to stake their dosh that he won’t, but there seems to be little that is beyond the Merkin right.)
Des, Which of the following points best describes your comment: “there seems to be little that is beyond the Merkin right”:
Justin: We’ve been round the houses on CT on the subject of betting markets. I have to say that I regard the claims in the Hollywood Stock Exchange marketing material (including the marketing material in the form of academic papers) to be heavily overstated. I also dispute a number of claims about the IEM; there were also a number of occasions on which it forecast a Gore landslide. I’ve not got time to get into them right now, but if you click on my “past posts” list in the left hand menu bar, you’ll see a number of exchanges between myself and James Surowiecki.
Btw, they were claiming in December that they had new funding (and hence would launch in March). I don’t recall them ever having published information on the reliability of their methods against expert opinion.
Des: Tradesports.com has a number of Osama bin Laden contracts. I’ve put a speculative bet on the Dec ‘04 contract myself, because traffic on a conspiracy theory mailing list got me thinking that maybe the Americans captured him ages ago and have got him stored up for a big pre-election news event.
Dsquared: Cheers; now all I need is a browser that works with it…
At what point to the market-based solutions begin to make fairly solid predictions about events? Is there a time period when they really begin to have predictive value?
Well, they’ve got some predictive value right now. About the same as polls right now. Markets aren’t actively evil ways of finding out about the world, they’re just not magic bullets.
Who needs smelly old predictions when you can have a cash-weighted averaging Bayesian likelihood estimator?
I know I don’t!
Justin is wrong about the Foresight Exchange being dead. And still a place where you can bet that Osami Bin Laden will be elected President,
if you happen to have really, really
hot secret information. (Or if you don’t mind wasting play money on a joke.)
IEM and its friends don’t predict the future. But they do tell you a lot about the present: roughly speaking, if sufficiently actively traded, an IEM market reduces conventional wisdom to a number. To the extent that conventional wisdom is likely to be right about an outcome, an IEM market is likely to “predict” the correct outcome.
Which means they’re more useful than polls, which merely feed conventional wisdom.
Dsquared,
“Well, they’ve got some predictive value right now. About the same as polls right now.”
The former is about bets on the question “Who do you think will win?” and the latter is about answers to the question “Who will you vote for?”. These are two different kinds of question (and it interests me that they give about the same results).
What about if we polled people and asked “Who do you think will win?” rather than have them bet on it? Would you say that the market or this poll would do better, and in what ways?
Weighting by cash is a non-trivial improvement on standard polling, since it allows persons to qualify the strength of their opinion.
Doesn’t stop the results being massively skewed by utter stupidity of course - you only have to look at English bookmakers odds for an English sports team to win something, which are always so short as to be worth shorting as a result of patriotic sentiment about Our Boys.
There has been some new work done on the performance of the Hollywood Stock Exchange by professors at Harvard and Penn, although it’s still unpublished. Their conclusion is that the HSX’s price the night, while not fantastically accurate (median error is 16%, I thik), is the single best forecast of a movie’s opening-weekend performance available. That’s roughly the claim — that they’re better than any other available method — that I think can be made for most collective forecasts, whether they’re made via markets, sophisticated voting mechanisms, or a pari-mutuel system.
There has been some new work done on the performance of the Hollywood Stock Exchange by professors at Harvard and Penn, although it’s still unpublished. Their conclusion is that the HSX’s price the night, while not fantastically accurate (median error is 16%, I think), is the single best forecast of a movie’s opening-weekend performance available. That’s roughly the claim — that they’re better than any other available method — that I think can be made for most collective forecasts, whether they’re made via markets, sophisticated voting mechanisms, or a pari-mutuel system.
There has been some new work done on the performance of the Hollywood Stock Exchange by professors at Harvard and Penn, although it’s still unpublished. Their conclusion is that the HSX’s price the night, while not fantastically accurate (median error is 16%, I think), is the single best forecast of a movie’s opening-weekend performance available. That’s roughly the claim — that they’re better than any other available method — that I think can be made for most collective forecasts, whether they’re made via markets, sophisticated voting mechanisms, or a pari-mutuel system.
What about if we polled people and asked “Who do you think will win?” rather than have them bet on it?
The UK used to do this, IIRC and it didn’t make much difference.
James: If I’ve got you right, I already don’t like this work before seeing it. Median error is a silly loss function for measuring the performance of solutions to this kind of forecasting problem. Big errors are much more costly than small ones, so the tails matter much moer than a median error criterion would pick up. Unless that’s a mistype for “mean-squared error”, I would suspect that someone had cherry-picked a criterion by which HSX did well.
The comparison of these markets to polls is an interesting one — finacial markets have no polls to compete against. What if there were a tradition in place of polling randomly selected groups of people on a particular company’s likelihood of making a profit? Would such a poll have about the same odds of making accurate predictions, as the stock market has?
Jam’s point about encapsulating conventional wisdom vs. feeding conventional wisdom falls short a bit when you consider that electoral markets results can be used the same way poll results are, to feed cw.
Here’s a link to PAM’s old website and the overview they provide:
À Gauche
Jeremy Alder
Amaravati
Anggarrgoon
Audhumlan Conspiracy
H.E. Baber
Philip Blosser
Paul Broderick
Matt Brown
Diana Buccafurni
Brandon Butler
Keith Burgess-Jackson
Certain Doubts
David Chalmers
Noam Chomsky
The Conservative Philosopher
Desert Landscapes
Denis Dutton
David Efird
Karl Elliott
David Estlund
Experimental Philosophy
Fake Barn County
Kai von Fintel
Russell Arben Fox
Garden of Forking Paths
Roger Gathman
Michael Green
Scott Hagaman
Helen Habermann
David Hildebrand
John Holbo
Christopher Grau
Jonathan Ichikawa
Tom Irish
Michelle Jenkins
Adam Kotsko
Barry Lam
Language Hat
Language Log
Christian Lee
Brian Leiter
Stephen Lenhart
Clayton Littlejohn
Roderick T. Long
Joshua Macy
Mad Grad
Jonathan Martin
Matthew McGrattan
Marc Moffett
Geoffrey Nunberg
Orange Philosophy
Philosophy Carnival
Philosophy, et cetera
Philosophy of Art
Douglas Portmore
Philosophy from the 617 (moribund)
Jeremy Pierce
Punishment Theory
Geoff Pynn
Timothy Quigley (moribund?)
Conor Roddy
Sappho's Breathing
Anders Schoubye
Wolfgang Schwartz
Scribo
Michael Sevel
Tom Stoneham (moribund)
Adam Swenson
Peter Suber
Eddie Thomas
Joe Ulatowski
Bruce Umbaugh
What is the name ...
Matt Weiner
Will Wilkinson
Jessica Wilson
Young Hegelian
Richard Zach
Psychology
Donyell Coleman
Deborah Frisch
Milt Rosenberg
Tom Stafford
Law
Ann Althouse
Stephen Bainbridge
Jack Balkin
Douglass A. Berman
Francesca Bignami
BlunkettWatch
Jack Bogdanski
Paul L. Caron
Conglomerate
Jeff Cooper
Disability Law
Displacement of Concepts
Wayne Eastman
Eric Fink
Victor Fleischer (on hiatus)
Peter Friedman
Michael Froomkin
Bernard Hibbitts
Walter Hutchens
InstaPundit
Andis Kaulins
Lawmeme
Edward Lee
Karl-Friedrich Lenz
Larry Lessig
Mirror of Justice
Eric Muller
Nathan Oman
Opinio Juris
John Palfrey
Ken Parish
Punishment Theory
Larry Ribstein
The Right Coast
D. Gordon Smith
Lawrence Solum
Peter Tillers
Transatlantic Assembly
Lawrence Velvel
David Wagner
Kim Weatherall
Yale Constitution Society
Tun Yin
History
Blogenspiel
Timothy Burke
Rebunk
Naomi Chana
Chapati Mystery
Cliopatria
Juan Cole
Cranky Professor
Greg Daly
James Davila
Sherman Dorn
Michael Drout
Frog in a Well
Frogs and Ravens
Early Modern Notes
Evan Garcia
George Mason History bloggers
Ghost in the Machine
Rebecca Goetz
Invisible Adjunct (inactive)
Jason Kuznicki
Konrad Mitchell Lawson
Danny Loss
Liberty and Power
Danny Loss
Ether MacAllum Stewart
Pam Mack
Heather Mathews
James Meadway
Medieval Studies
H.D. Miller
Caleb McDaniel
Marc Mulholland
Received Ideas
Renaissance Weblog
Nathaniel Robinson
Jacob Remes (moribund?)
Christopher Sheil
Red Ted
Time Travelling Is Easy
Brian Ulrich
Shana Worthen
Computers/media/communication
Lauren Andreacchi (moribund)
Eric Behrens
Joseph Bosco
Danah Boyd
David Brake
Collin Brooke
Maximilian Dornseif (moribund)
Jeff Erickson
Ed Felten
Lance Fortnow
Louise Ferguson
Anne Galloway
Jason Gallo
Josh Greenberg
Alex Halavais
Sariel Har-Peled
Tracy Kennedy
Tim Lambert
Liz Lawley
Michael O'Foghlu
Jose Luis Orihuela (moribund)
Alex Pang
Sebastian Paquet
Fernando Pereira
Pink Bunny of Battle
Ranting Professors
Jay Rosen
Ken Rufo
Douglas Rushkoff
Vika Safrin
Rob Schaap (Blogorrhoea)
Frank Schaap
Robert A. Stewart
Suresh Venkatasubramanian
Ray Trygstad
Jill Walker
Phil Windley
Siva Vaidahyanathan
Anthropology
Kerim Friedman
Alex Golub
Martijn de Koning
Nicholas Packwood
Geography
Stentor Danielson
Benjamin Heumann
Scott Whitlock
Education
Edward Bilodeau
Jenny D.
Richard Kahn
Progressive Teachers
Kelvin Thompson (defunct?)
Mark Byron
Business administration
Michael Watkins (moribund)
Literature, language, culture
Mike Arnzen
Brandon Barr
Michael Berube
The Blogora
Colin Brayton
John Bruce
Miriam Burstein
Chris Cagle
Jean Chu
Hans Coppens
Tyler Curtain
Cultural Revolution
Terry Dean
Joseph Duemer
Flaschenpost
Kathleen Fitzpatrick
Jonathan Goodwin
Rachael Groner
Alison Hale
Household Opera
Dennis Jerz
Jason Jones
Miriam Jones
Matthew Kirschenbaum
Steven Krause
Lilliputian Lilith
Catherine Liu
John Lovas
Gerald Lucas
Making Contact
Barry Mauer
Erin O'Connor
Print Culture
Clancy Ratcliff
Matthias Rip
A.G. Rud
Amardeep Singh
Steve Shaviro
Thanks ... Zombie
Vera Tobin
Chuck Tryon
University Diaries
Classics
Michael Hendry
David Meadows
Religion
AKM Adam
Ryan Overbey
Telford Work (moribund)
Library Science
Norma Bruce
Music
Kyle Gann
ionarts
Tim Rutherford-Johnson
Greg Sandow
Scott Spiegelberg
Biology/Medicine
Pradeep Atluri
Bloviator
Anthony Cox
Susan Ferrari (moribund)
Amy Greenwood
La Di Da
John M. Lynch
Charles Murtaugh (moribund)
Paul Z. Myers
Respectful of Otters
Josh Rosenau
Universal Acid
Amity Wilczek (moribund)
Theodore Wong (moribund)
Physics/Applied Physics
Trish Amuntrud
Sean Carroll
Jacques Distler
Stephen Hsu
Irascible Professor
Andrew Jaffe
Michael Nielsen
Chad Orzel
String Coffee Table
Math/Statistics
Dead Parrots
Andrew Gelman
Christopher Genovese
Moment, Linger on
Jason Rosenhouse
Vlorbik
Peter Woit
Complex Systems
Petter Holme
Luis Rocha
Cosma Shalizi
Bill Tozier
Chemistry
"Keneth Miles"
Engineering
Zack Amjal
Chris Hall
University Administration
Frank Admissions (moribund?)
Architecture/Urban development
City Comforts (urban planning)
Unfolio
Panchromatica
Earth Sciences
Our Take
Who Knows?
Bitch Ph.D.
Just Tenured
Playing School
Professor Goose
This Academic Life
Other sources of information
Arts and Letters Daily
Boston Review
Imprints
Political Theory Daily Review
Science and Technology Daily Review