Advocates inside and outside government are working hard to embrace open data practices. International bodies, governments and even rock stars have moved past the question of why we need open, shared systems of government data to questions about how to build and maintain those systems.
As we form new alliances and secure new policies to enable efficient sharing of data on financial flows, international aid, public services and related sectors, organizations and coalitions must work to link our activities as effectively as we hope to link our data sets.
Transparency and accountability groups like our own are investing staff time, research, and money to join this open data revolution. This moment of innovation may resemble earlier milestones for technology and advocacy, but we need to keep the particular challenges of open data in mind, and to clearly define our responsibilities in the creation of a vibrant, reliable open data ecosystem.
The digital revolutions that gave organizations a human voice (e.g., blogs) and their supporters a bigger role (e.g., Facebook) were, from a certain point of view, selfish. One group with one mission could speak louder and reach further, even if they were the lone voice speaking.
Big data is different: The more that groups collaborate, the more meaning their data has. Like a wiki entry or a potluck meal, it fulfills its purpose as more people add pieces.
So while it’s vital for transparency groups and governments to continue opening and disseminating data, we must also create strong channels of ongoing collaboration—not among systems but among people.
This alignment of efforts is already underway at the policy level. In recent months, the G8 countries, the European Union and the Canadian government have each made new commitments to open data practices, following the historic actions of the U.S. Congress and regulators opening up oil and mining payments in the 2010 Dodd-Frank reforms. The Open Government Partnership says open data is central to many of the 300 commitments made by participating countries. In the words of Martin Tisné, former T/AI director and now policy director for T/AI funder the Omidyar Network, “Transparency is now seen as a key pillar of development – a necessary (but not sufficient) condition to enable growth, accountability and social change.”
Meanwhile, the tools for linking data sets are gaining adoption beyond the technology community. In the past 18 months, both the United States and the United Kingdom moved their public data portals onto CKAN, a community repository for data sharing created by the Open Knowledge Foundation. The World Bank announced several new initiatives in June that seek to collect and open up data, including a collaboration with OpenCorporates.com on private sector data. The World Bank Institute also continues to help define good practices and data-driven tools that support open contracting by governments. And, as another of many examples, the African Development Bank launched a new Open Data Platform in July to help “scale up the collection, management, analysis, and sharing of quality statistics relating to the continent’s development.”
What remains to be seen is how well we as advocates for increased data-sharing and data-driven collaboration can align our own sharing and collaboration. Can we work together regularly, toward measurable ends, and create a functional data commons that uses linked data sets to reveal more than any single group’s information can show?
Besides our own initiative, here are three efforts currently underway that seek to join advocates, practitioners and data for productive teamwork:
Launched in 2012, the Open Data Institute works across the private and public sectors to create “untapped markets and business opportunities” for open government data. The institute was founded by web inventor Sir Tim Berners-Lee and the recently-knighted Sir Nigel Shadbolt, a pioneer in the UK and global open data movements. Among ODI’s projects is the creation of a data schema designed to standardize the licensing and usage rules that accompany data sets published online.
During summer 2013, our colleagues at the Sunlight Foundation joined with fellow TABridge members Fundar and Open Knowledge Foundation, along with several other groups, to establish a Global Open Data Initiative, which seeks to increase awareness of open data issues and help governments define open data approaches. As part of this effort, Sunlight recently updated their open data policy guidelines, “a living document created to help define the landscape of what open data policies can and should do.”
In April of this year, a diverse group of technology and transparency advocates working on aid (Publish What You Fund), natural resources revenues (EITI, Publish What You Pay, Natural Resource Charter), corporate ownership (Open Corporates), government contracts (World Bank Institute, Open Oil) and government spending (International Budget Partnership) sat down in a first attempt to assess how their individual sets of government and analytical data could become interoperable. Initially conceived at T/AI’s 2012 Bridging Session, this first effort to “Follow the Data” was convened by Revenue Watch Institute and Open Knowledge Foundation with support and participation by the Omidyar Network and the UK Department for International Development (DfID).
Since the first “follow the data” meeting, DfID has organized a series of hack days in London, Lagos and Sydney, to get local civil society and developers working together to make sense of data on oil, gas and mining revenues. These experiments have helped to highlight the power of interoperable data to track flows of money from mining royalties or international donations, through to government budgets and local spending. To understand more about the process of linking data, the practical challenges, and the high value of oil and mining data in particular, read these three posts by Tim Davies, Adam McGreggor and Jonathan Gray, all members of the growing Follow the Data community.
If the promise of a rich open data ecosystem is collective wisdom greater than any individual data set can provide, and the tools and rules for reliable linkages are well underway, what does this opportunity demand of us—as campaigners, NGOs, techies and funding groups? What must we do to help bring this crowdsourced picture into clearer focus? And what are the essential elements to making sure that data-driven collaboration will flourish among all groups?
First, we need to create an expectation of cooperation among transparency groups, donors and technologists. We will inevitably compete for funding and have days or weeks on end where the demands of daily work overtake innovation projects and coalition conference calls. But we must establish a norm of shared efforts with agreed goals, real deadlines and cycles of assessment and iteration, where the same groups that collaborated in preparation for Conference A are still thinking together six months later in Research Paper X.
Second, we need to seek and cultivate the will to open up, not only by finding more champions within government, but by building the demand for open data projects and cross-sector teamwork among our own colleagues and managers. It isn’t only databases that need to open up, it’s doors and calendars and minds.
Third, we need to ensure there are steady resources available. Donors need to provide financial incentives not only for technology projects, but for capacity development, community outreach and continuing partnerships among the divergent groups whose combined efforts will be needed if we hope to follow the data to actionable knowledge. And donors are not the only group that need to allocate resources. Organizational leaders need to understand that data projects take real time and don’t happen “on the side.” And staffers inside NGOs and governments need to accept that these new activities are as important as the work to which they’re more accustomed.
With steady cooperation, demonstrable will from participants and leaders, and sufficient resources in time and funding, the open data experiments now cropping up among several groups can develop in a climate that supports more regular, deliberate collaboration.
The more consciously we link our activities and findings, the more seamlessly we will be able to link our data, and the more shared knowledge will result. In the campaign for open data, as in all transparency and accountability work, there’s more than safety in numbers, there’s power.
