Open Data – The Democratic Imperative

Open Data are the basis for government innovation. This isn’t because open data make government more transparent or accountable. Like Tom Slee, I have serious doubts about whether it does either of those things. In any event, shining a light on the misdeeds of ineffective institutions isn’t as imperative as redesigning how they work.  Instead, open data can provide the raw material to convene informed conversations inside and outside institutions about what’s broken and the empirical foundation for developing solutions together.

The ability of third parties to participate is what makes open data truly transformative. The organization that collects and maintains information is not always in the exclusive position to use it well.  For example, US regulators have compiled hospital infection rates for a long time.  Accessible only to government professionals, they had limited resources to make adequate use of the information.  When HHS made the data publicly available by publishing the data online in a computable format, then Microsoft and Google were able to mash up that information with mapping data to create search engines that allow anyone – from the investigative journalist to the parent of the sick child — to decide which hospital to choose (or whether it is safer to stay home). When data are open — namely legally and technically accessible and capable of being machine processed – those with technical know how can create sophisticated and useful tools, visualizations, models and analysis as well as spot mistakes or mix and mash across datasets to yield insights. As Matt Parker, put it: “By making data open, you enable others to bring fresh perspectives, insights, and additional resources to your data, and that’s when it can become really valuable.”

Complex Democracy

Solving complex challenges requires many people with diverse skills and talents working together. In modern society, we weave our collective expertise together, enabling us to make complex products such as cars and computers that we cannot make alone. The more complex and diverse the products, the more successful – measured both in terms of wealth and well-being – the society over time.

Educating our young or curing cancer are the cars and computers of governance. They are complex social problems that require us to bring our diverse talents to bear. But our centralized institutions of government do not adequately leverage our collective knowledge to improve governance and solve problems. We can’t foster complexity if we limit public participation to voting in annual elections or commenting on already written rules. There’s no excuse for failing to take advantage of people’s talents, abilities and desire to play a role in governing ourselves and our own communities.

Hackathons as a Model for Engagement

Open data create obvious new ways for geeky citizens to play a role in governance. All over the world, local transportation authorities are making schedules available for free and then inviting tech savvy citizens – civic coders — to create iPhone apps that tell commuters when their bus or train is coming. There’s obvious value to the public as well as to institutions from having better data to inform planning, policymaking and the expenditure of resources. But what’s exciting about mashathons, hackathons, data dives and datapaloozas (a Todd Park favorite term) is that these are intelligible models for taking action.

Wikipedia works because we know what tasks are required of us to write an encyclopedia entry. Only the high priesthood of government professionals knows how to write a law, craft a policy, draft procurement RFPs, or appropriate funds. Hackathons aren’t the only model for participatory governance but they are one way for us to get involved that showcases how it might be possible to move away from centralized to distributed action.

Making government more participatory wouldn’t have worked as well if we had only focused on releasing data-as-in-FOIA about the workings of government — politicians’ tax returns, who-met-with-whom, and even spending data. By defining High Value Data to include information: “to increase agency accountability and responsiveness; improve public knowledge of the agency and its operations; further the core mission of the agency; create economic opportunity; or respond to need and demand as identified through public consultation,” the hope was to speak to more people’s interests, talents and abilities.  We took a lot of flak at the time from those with passion for specific kinds of data. I have written previously that the “open” in open gov was never meant to suggest data-as-in-FOIA but, rather, meant open as in open innovation and therefore always had to go beyond “civil liberties data” to include all the information that government collects as well as information that citizens might crowdsource and provide to make government smarter.

The Hard Work of Opening Data

Moving toward open innovation as a default way of working in government is not easy. It takes a religious fervor (hence the sense of movement) for those who want to open up data.

It requires doing the hard and costly work of persuading data owners to shift from paper to digital and machine-readable formats and then to release that data despite political and technical challenges. But to foster engagement also requires curating the guest list for the hackathons to get subject matter experts, stakeholders, data geeks, activists, designers, computer scientists, data junkies and entrepreneurs together.

The host of a good dinner party doesn’t just leave the guests to fend for themselves. He introduces people, points out what they might have in common and seeds the conversation. Transit camps have been so successful because the conversation starts itself. Everyone wants to know when their bus is coming. But give people a data set about freight routes for transshipping goods or Form 990 tax returns and some explanation might be required.

Creating a participatory innovation ecosystem is about a lot more than just publishing data sets. It requires doing the hosting, convening, persuading, and demonstrating involved in inviting diverse people to participate. The institutional players have to be prepared to collaborate with the innovators; those outside government have to know how to collaborate; civil society activists have to ensure that innovators know the problems that need solving; and research is needed to figure out what works.

Using Data to Re-Regulate

The curatorial function is about coming up with strategies for using data to develop innovative solutions to protect consumers and serve the public interest. If we merely throw data over the transom, entrepreneurs, especially large ones, are likely to be the only entities with the wherewithal to do anything with the raw information.

But when we focus on data as a means to the end of bringing people with diverse skills together to solve problems then open data can improve upon the blunt instrument of regulation enforced by litigation.

With open data (also called Smart Disclosure), the US government is experimenting with using light touch regulation combined with technical innovation (and a firm belief in behavioral economics) to create consumer decision tools. For example, the Department of Transportation enacted a rule to require airlines to make all their fees and charges transparent. Because the data is open, innovators can create new visualizations to help consumers understand the costs and make informed decisions. No Child Left Behind requires states to gather and report school performance data, which is now being used by GreatSchools.org (in cooperation with the Department of Education) to help parents choose between public schools. The tool is in use 40-50% percent of all K-12 households. The White House Open Educational Data Initiative is spurring university Presidents to provide data voluntarily to help students and parents compare college costs and college aid “so they can make more informed decisions about where to enroll.”

But until we stop talking about data and start talking about complex and collaborative governance, we will fail to appreciate how open data can both protect consumers, lessen the burdens on entrepreneurs and catalyze more effective institutions.