The open data movement is a hammer which has gathered the support of many nails. There are the curious taxpayers, who feel their annual checks mean they deserve a peek at the interesting facts the government has collected. There are the ambitious business owners, who see an opportunity to privatize profits from work with socialized costs. And there are the self-styled activists, who believe that if we reveal the data on what the government is really doing, we will arrest corruption by exposing it to sunlight.
The coalition is a confusing mix of these very different motivations (as Tom Slee observes), and the benefits of such a tactical alliance has come with the cost of some confusion. So let’s be clear about what open data can and cannot do.
If the St. Louis Fed publishes reams of economic data, it can certainly make it easier for Mr. Yglesias to make his fantastic charts. If the MTA makes real-time subway information public, it can certainly let Mr. Ernst improve his fantastic app. And, as the talented Mr. Lee pointed out to me, his careful collection of data about members of Congress and the bills they’e passing can be an invaluable resource for professional activists.
So, if I got to choose whether the government should share the data it’s collected, I’d happily vote yes. In fact, I spent several years of my life using the FOIA laws to force it to do just that. I can’t claim my work had any particular impact, but as a curious taxpayer, it was a weirdly-enjoyable hobby.
But the open data movement often claims to be much more than that. They insist open data will not just help a few people with their jobs or a few kids with their hobbies but, as the Sunlight Foundation puts it, "make government transparent and accountable." And that I just don’t see.
I’ve outlined my theory why elsewhere, but the short version is pretty simple: people hide their crimes. Imagine you learn lots of bribes are exchanged at top of the Capitol Reflecting Pool. So you lobby Congress hard to set up bright lights and a camera to catch the perpretrators. The video would be livestreamed to the Internet so dedicated watchdogs can name and shame the bribetaking politicians. Your lobbying succeeds and, on January 1st, the lights go up and the cameras switch on.
But as an engaged citizenry tunes in, there’s is nothing but disappointment. Nobody seems to be taking bribes; just a couple pieces of litter blowing by the pool.
Was Congress really squeaky-clean after all? Of course not—the bribes just moved to the other end at the pool, out of the spotlights.
When you have time to prepare, it’s pretty easy to disguise the data. And this is exactly the pattern we’ve seen. It’s always been investigative journalism, not data mining, that’s revealed the big scandals about politicians. I, more than anyone, would love to believe that the next great Watergate is just lying in plain sight to be uncovered by a swashbuckling econometrician, but the sad fact is, it simply isn’t so.
But it’s also worth pausing to ask: what was any of this supposed to achieve? Imagine, for some strange reason, members of Congress didn’t bother avoiding the spotlight. Every day, we saw them, in full HD video, taking money from prominent businessmen. Do we really think even this (far-fetched) instance of transparency would change much? After all, most Americans already think Congress is corrupt. Most Americans think money actually buys politicians’ votes. Seeing it happen in video might be striking, and maybe make for some good segments on the evening news (or, these days, some viral YouTube videos), but would it really change anything?
After a couple weeks of chatter, and perhaps a few grandstanding legislative proposals, I suspect it’d just fade into the background. More dramatic examples are not exactly what’s most missing from the reform debate—Lessig’s recent book has enough to last us a couple decades. Structural reforms have failed because of the incompetence of reformers, not because there’s a lack of evidence that there’s a problem. (Free tip to structural reformers: get state legislators to sign on to your constitutional amendment. They’re very susceptible to public pressure, there’s a lot of them (so you’ll have a constant narrative of progress), and they’re the ones you’ll ultimately need to actually pass the amendment.)
But maybe open data was supposed to improve politics in other ways. Structural reform is an ambitious goal—maybe the open data proponents wanted something much more modest. But all the more modest stories suffer from a similar excess of naïveté. Whenever geeks turn their eyes to politics, they always have the same reaction: There’s so much inefficiency! And they naturally propose the obvious ideas for reducing it—for example: If only it was easier for citizens to read bills, citizens with relevant expertise could assist Congress by sharing their hard-earned wisdom!
The fact is, Congress isn’t interested in availing itself of your wisdom any more than the sausagemaker needs your help tidying the floor. Lawmaking is The Wire, not Schoolhouse Rock. It’s about blood and war and power, not evidence and argument and policy. (I have one friend who was startled to learn that when members of Congress debate an issue on C-SPAN, they’re speaking not to each other but to cameras in a largely-empty room.)
I don’t want this to sound overly harsh. The truth is, it’s really hard to do effective philanthropy. With a little work, you could mount a similar critique of the vast majority of our bumbling efforts to do good. Most ideas for helping people that seem reasonable in the abstract, turn out to fall apart upon close confrontation with reality. The real question is what happens then. There’s no shame in admitting your mistakes, learning from them, and trying again. Indeed, as my old professor Carol Dweck has shown, that’s the only real route to success. But most of us are too vain or too proud to take that route. We insist that the purity of our intentions reduces the need for careful scrutiny of our effects. Or we try to make ourselves feel better by grasping at any factoid that suggests we had an impact.
I have no particular interest in correcting people’s pride or vanity. This movement is populated by my friends and I respect them enormously and wish them well. Throwing darts at their day jobs has only made my life worse. But this stuff matters—funders and volunteers face tough choices about which causes to pursue. It’s important that they know the case for opening up data to hold government accountable simply isn’t there. (And that they should invest in metaresarch, inxluding open scientific data, instead.) It’s nothing personal—just trying to help everyone do their best. I dearly hope that if anyone ever has a similar critique of the causes I pursue, they will be even more blunt in pointing out my folly.