Open Data: Better Politics, Winning Politics… But Still Politics

by tom_lee on July 6, 2012

My alma mater had a celebrity professor of political science who was principally known for two things. First, for accidentally leaving his wireless mic on during mid-lecture restroom breaks. And second, for the slogan “Politics is a good thing!” which he relentlessly promoted via mediums as diverse as lectures, TV appearances and TA’s t-shirts.

Well, we all make mistakes. But only political science professors seem to make that second kind of mistake. This glib celebration of a maximally vague conception of politics always rankled, conflating, it seemed to me, everything from a heartfelt PTA meeting speech to Caesar bleeding to death on the Senate floor. I never liked that class, and “politics” still often seems unmanageably broad to me. Pondering what open data has to do with “good” or “better” politics, I find that adding a qualifier only leaves me more confused.

Still, I suspect that my professor and I would have been able to agree on at least a few concrete things. A free press; universal suffrage; a public education system; the secret ballot: these are unobjectionable, broadly agreeable foundations of democracy. Open data should perhaps be the newest addition to that list. In one sense, open data is pre-political.

Whatever parts of our political system you happen to value, unencumbered government data almost certainly plays a role in their support. Knocking on doors to get out the vote? That’s made possible by “Census TIGER/Line map data”: and voter roll information. Wielding facts and figures in the poli/econ blogosphere? The open data policies of “BLS”:, the “Fed”:, “CBO”: and other institutions power these debates. Even the tedious daily point-scoring of cable news is enabled, in part, by “video”:, audio and “text material”: provided by various publications and outlets of the House and Senate. Whether or not you consider these mechanisms constitutive of “good politics”–in the sense of representing a productive and positive kind of deliberation–the fact that they are possible seems like an undeniably good thing.

* * *

These examples embrace a broad conception of what “open data” means. More specific definitions exist: the “Open Knowledge Foundation has a good one”: ; at Sunlight we tend to gesture toward “a set of more prescriptive and explicitly government-focused principles”: But across definitions, the broad outlines are the same: open data is digital information that is unencumbered by fees, credentialing, licenses and other unnecessary limitations on its use and distribution.

The rationale behind offering such resources is straightforward. Copies of a given piece of digital information have no marginal cost; supply is infinite. By embracing that limitlessness we can lower the costs facing potential users of the information, spurring more use. Who knows what benefits might result? More original research; more “technology startups”:; “more and better public interest advocacy”:–these are all plausible benefits to open data. In addition to this pragmatic calculus, one can argue that citizens should be extended a right to free access to information by and about their government (this line of thought is probably best embodied by the work of “Carl Malamud”: Both justifications are plausible, inspiringly egalitarian, and perhaps a bit hand-wavingly utopian. In my experience, those are all useful attributes: people find them appealing. So in this sense, too–the sense of being a winning and uncontroversial issue–open data is “good politics.” Certainly this has been our experience at the Sunlight Foundation, where we have successfully attracted support for open data policies from legislators and citizens representing a wide variety of ideological perspectives.

* * *

Of course, someone is going to have to pay for the collection, organization and distribution of all this data. Government might already be doing some of those things in order to fulfill its other responsibilities, but there are typically additional costs that have to be born somehow. At present our public sector data is paid for through a mix of general funds and user fees. The above account–that unlimited supply allows for potentially vast surpluses–argues for socializing those costs, and “in the past Sunlight has found itself arguing to that effect”:

And here, I suppose, is where open data becomes a bit more controversial than apple pie and miniature American flags. Tom Slee started this conversation on a skeptical note, proposing that open data is little more than a catchy brand name that’s being used to justify new IT expenditures. Many open data advocates, myself included, rejected this idea: open data, we said, is the rare partnership where corporate interests and the public good are well-aligned. This produced some understandable eye-rolling from Slee and others. And certainly articles with titles like “”How To Cash In On Government As A Platform”:” have done little to quell concerns that so-called civic hackers’ talk of public service is just so much first-date patter as they greedily eye the public’s assets.

In my experience, this concern is unjustified: most open data advocates I know–including the author of that Techcrunch article–have a genuine interest in doing work for the public good, even if it means a pay cut. To the extent that these individuals embrace the rapacious language of Silicon Valley startup culture, it’s usually to make the cause more palatable and interesting to their fellow coders. But take my anecdotal experience for what it’s worth. In a world where “the Department of Defense casually announces they have a couple of spare Hubble Telescopes lying around”:, I think it’s difficult to make the case that the our government’s modest open data initiatives are much of a threat to the Treasury, much less the best example of the threat posed by public/private partnerships. This is doubly true thanks to our movement’s insistence on nonproprietary formats and open source code, which allow more flexibility and competition in subsequent procurements.

* * *

But none of this means that complacency about open data is justified. The current generation of open data advocates is commendably enthusiastic, but we deserve at least some criticism for our callowness. Our community has a strong incentive to insist that the idea of open data is so new that its limits can’t–and shouldn’t!–yet be pondered. In truth, the U.S. has had something like an open data policy since at least its first census; and that agency has been distributing its information electronically “since the seventies”: Our tendency to ignore this history in favor of an emphasis on novelty and exciting promises is not only making us look foolish, but is beginning to produce a sense of disappointed malaise both within the movement and among its allies.

This disappointment can be seen in some of the other contributions to this roundtable. Both Clay Shirky’s and Aaron Swartz’s offerings lament the open data movement’s failure to rack up concrete victories against corruption. That our community fostered such unrealistic expectations is a strike against us. “There is good empirical evidence that open records laws produce lower-corruption equilibria”: But the dream of writing a cron job that moves misbehaving lawmakers smoothly from office to prison was never likely to come to pass (though public disclosure systems have inarguably played important roles in the downfall of figures like Bob Ney, Jack Abramoff, John Ensign and Duke Cunningham). Open data’s effect on corruption will more commonly involve altering malefactors’ cost:benefit calculations, consigning corrupt acts to a counterfactual that we’re unlikely to ever be able to precisely measure. This is just how enforcement works: putting more cops on a beat doesn’t reduce crime simply because they arrest more law-breakers. Applying digital technology to sunshine law disclosures can clearly produce more and better oversight, but it is unlikely to transform that well-established practice into a solved problem.

I fear that our community is presently setting itself up for similar disappointment in the promises we are making about open data’s commercial potential. For all of the excitement about “Brightscope”:, “DarkSky”: and a handful of others, the supply of stories about open data startups seems clearly unable to keep up with demand. One way of explaining this is to point out that although open data may be useful, it’s also easily accessible to rival businesses; perhaps open data is destined to be as simultaneously useful to and taken-for-granted by businesses as city streets and water pipes. Less palatably, one can acknowledge that the government data resellers of decades past have grown up into the Elseviers and LexisNexises of today.

Unfortunately, there doesn’t yet seem to be much appetite for considering these possibilities. At the moment it’s more common to hear that we’re still in this sector’s early days; that a wave of civic startups is just over the horizon. Perhaps that’s right. Certainly it would be useful if it were. But at the moment, I think there’s reason for doubt.

* * *

It seems to me that both of these misjudgments stem from the same underlying error. And it is this problem of philosophy, more than anything else, that threatens to make the final judgment of the open data movement a negative one.

Like most open data advocates, I came to this field by way of software engineering. Writing code is a valuable skill, and an intellectual exercise that I would encourage anyone to explore. But like any discipline, programming colors how one views the world. Computer code is about boolean logic–algorithmic procedures that map input to output in a way that is mathematically perfect and so powerful that it can engender a kind of giddy terror. Many of us who are trained to write code know some linear algebra, but our knowledge of statistics is often surprisingly meager. This makes it all too easy to give in to the hopeful assumption that reality is a comprehensibly deterministic machine; that data are necessarily objective. Once you begin scanning early adopter-types for this expectation you can see it everywhere, from the thirst for “a perfect weather app”: to “the quantified self movement’s breezy confidence that the human body is an experimental apparatus that works even when n=1”:

The open data movement is no different. It is not uncommon to hear open data advocates promise that newly-released information will allow government to make better decisions. It’s a dream embodied by sites like “”: and the EPA’s “Apps for the Environment contest”: And in one sense, it’s a perfectly coherent vision: information *can* lead to better decisions; so can opening deliberative processes to include more qualified participants. If you have any faith at all in democracy and rational deliberation, these ideas are inescapable.

But these ideas can also be easily overextended into the assumption that governance has computable solutions–that politics lingers not because, even after decades of thoughtful analysis, groups have competing claims that must be resolved; but rather because post-partisan technocracy is only now becoming able to offer definitive answers. This is the same wishful thinking that motivates efforts like “We The People”:, “MADISON”:, “Americans Elect”:, and “optimism about the net bloc’s ability to translate its successful activism against SOPA/PIPA to other issues”:

This tendency to deny of the inescapability of politics is a relatively quiet current in the open data movement, but it is a real one. And while I doubt that open data as a cause will live or die by the success of its commercial ambitions, the implicit promise that open data can rescue policy from politics seems destined to end in disappointment. We can smooth the flow of information through our institutions, but this alone will rarely be enough to redeem them, much less render them obsolete.



bianca steele 07.06.12 at 2:13 pm

Re. “Politics is a good thing!” being silly: I thought that way too, until the first time a person who self-identified as affiliating with technocracy told me that because I disagreed with her about what was reasonable or scientifically proven, I was engaging in politics and she wasn’t, and that seemed perfectly reasonable to other people. That’s not what I think of when I hear the word “politics,” but it made me wonder how many people who think “politics is bad” really think “people who don’t agree with me should shut up.”

I’m not sure what this has to do with open data, and don’t want to derail the thread (when I read Tom Lee’s posts, I agree with more than 50% of them, usually), except if people think it’s going to be easier than it will. I’m not as worried about programmers being in charge and thinking they can program anything (I’m a software engineer by career and training myself). I’m more worried about thinking there’s a way to get things done from a distance–whether it’s from an ivory tower or a high-tech beltway doesn’t seem to make much difference.


Alex 07.06.12 at 2:28 pm

I presume this is the impact of Bernard Crick’s In Defence of Politics.


bianca steele 07.06.12 at 2:30 pm

And there’s some good scholarship on how government has used new technologies, in the form of statistics (there’s a reason it begins with “stat[e]”), for ends people might not like–Theodore Porter, for one. But there’s lots of room for more.


Scott Martens 07.06.12 at 2:50 pm

“Politics is a good thing!” – that’s part of the point of Farrell & Shalizi’s “cognitive democracy”, and it’s a good point. Open data surely has a role to play in that, but the focus on government data strikes me as missing something.

First, I do think the government has some legitimate secrets and the debate over what those are is genuinely difficult. The Wikileaks affair is very revealing about government, however, there is a legitimate role for government officials to acknowledge in secret that, for example, Israel and Saudi Arabia are pretty awful countries with awful policies, while carefully never saying any such thing in public, because it is seriously detrimental to many legitimate public interests. In the same way, there is a legitimate interest in keeping operational information about military activities closed, at least in some contexts. In both cases, there is very real abuse, but I’m not at all sure that a sunshine policy across the board for government is a good idea.

And, on the flip side of the same argument, I have a legitimate interest in many private activities that depend on my support to operate and that I depend on to live. Banks, for example, have certainly screwed up millions of lives lately in ways that being forced to operate openly might have prevented. The LIBOR scandal should make that case plain enough.

As for the software engineer’s outlook on open data… I came into software from linguistics, where the fantasy that “reality is a comprehensibly deterministic machine” and “data are necessarily objective” didn’t survive the passing of MITI’s “Fifth Generation Computer”. Really existing data usually looks like’s collections of scanned books. At best, it looks like the tables at It almost never looks like fantasies of ontologically clean OWL data or easy-to-handle spreadsheets. “Formats that easily lend themselves to machine processing” is a pretty empty demand in all but a few special cases, and the code that best copes with real data is driven by statistics in almost every case.

Asking for open data often means asking for free access to data that someone carefully and purposefully collected and interpreted because they thought it would be useful for something. It’s not necessarily stuff that’s already there, being held up by self-preserving politicians and bureaucrats. I don’t think the Open Data movement, as currently constituted, has done much asking who will organize data into useful forms, what priorities ought to motivate them, and out of whose budget it ought to come.


straightwood 07.06.12 at 6:10 pm

There is a problem of focus here. Open Data is such a diffuse notion that it resists practical definition and offers no straightforward path of implementation. What people are really after here is open institutional governance. The accessibility of non-sensitive data is moot, since it will be cheaper to publish it than to hide it. The data on who did what to and for whom will be much, much more difficult to obtain and preserve. So the Open Data strategy seems to be to put out a vast driftnet without naming the big fish of malfeasance that are the targets, perhaps in the hope that there will be less political resistance. Common sense suggests a pursuit of data openness prioritized by political significance, not logical categorization.

Confusion aids no one here. Those who stand to lose from the exposure of mischief are shrewd enough to recognize threats to their power, and those who stand to gain from access to information supportive of reform should not be distracted by amorphous initiatives that end up retarding meaningful political change.


rea 07.07.12 at 2:30 am

Well, open data is not always a good thing, as your professor demonstrated.


Peter Hovde 07.07.12 at 6:32 pm

“Knowledge is Good.”


gordon 07.08.12 at 1:10 am

Scott Martens (at 4): “…there is a legitimate role for government officials to acknowledge in secret that, for example, Israel and Saudi Arabia are pretty awful countries with awful policies, while carefully never saying any such thing in public, because it is seriously detrimental to many legitimate public interests.”

Really? What does that mean? I don’t think it means that the US Govt. is just worried about disclosure of its beliefs about awful policies. I think it means that the US Govt. is worried about disclosure that it actually supports awful policies, even when knowing their awfulness. There is a difference. So Mr Martens is saying that if there is a public interest for the US Govt. in somebody else’s awfulness it’s OK for the US Govt. to support that awfulness, only we shouldn’t talk about it.

Where does that end? Anywhere? How awful does a second party’s awfulness have to be before the US Govt. stops supporting it even if there is some public interest for it to continue? How do you make that calculation?

Perhaps the fear of disclosure of this “awfulness-tolerance” has something to do with public opinion. Would US public opinion draw the line (if there is a line) somewhere other than where the US Govt. would draw it? Would that be a bad thing?


grooft 07.08.12 at 6:02 pm

2012 International Open Government Data Conference
July 10-12, 2012 World Bank Headquarters

Co-sponsored by the U.S. General Services Administration,, the World Bank Open Data Initiative and the Open Development Technology Alliance

Registration is Closed – But web streaming is available.

Join open data and open government leaders from around the world for three days of speakers, panels, and collaboration on international open government data. The conference will be organized around two tracks: policy and technology.
Participate Online

Follow the live blog and webstream from the conference on World Bank Live.
Also, make sure to follow us on Twitter via #IOGDC and

Comments on this entry are closed.