How Open Data Can Strengthen National Statistical Offices
In the last five years, many national governments have announced open data initiatives, and states and cities have joined in. Releasing data openly should make governments more credible. Citizens perceive greater transparency and accountability when they can measure results. They see the impact of open data on improved service delivery. Entrepreneurs create value-added applications from open datasets. Data quality improves when data are well documented and open to public review.
But many national statistical offices, particularly in developing countries, continue to watch the open data movement from a distance, either afraid to act or uninterested. Asked about their concerns, statisticians cite worries about data quality, cost, privacy, and information technology issues. The pinch is that open data is not a fad. The increasing momentum of the open data movement shows that it is here to stay. And the progress already made to open data created in other government ministries and agencies is impressive. So why are many national statistical offices still on the sidelines? They have a lot to offer. After all, they have been in the information business longer than any other government department. If national statistical offices can overcome their worries and address the challenges of open data, the potential opportunities and benefits for everyone are enormous.
What are some of the opportunities for the national statistical office? Most important national statistical offices can and should respond to the growing demand for data: at home from citizens and civil society organizations, from businesses and news media, and from the global initiatives such as Rio+20 and the post-2015 sustainable development agenda and other data-intensive global programs such as the Global Alliance for Vaccines and Immunization (GAVI), the Open Budget Initiative, and the International Aid Transparency Initiative (IATI), among many others.
Another benefit for the national statistical office is that open data initiatives support improvements on the supply side, including modernization of statistical systems, upgraded IT infrastructure, and responsive user services. By better responding to demand and improving the efficiency of the supply side, national statistical offices can reach a larger audience and raise their profile and influence, which can, in turn, help with resource mobilization. Countries and agencies that have embraced these opportunities have modernized their statistical systems, improved data coverage, and reached a larger audience. Good examples are Britain, Canada, New Zealand, Kenya, and Ghana and international organizations such as the World Bank.
Measuring the benefits of open data
What do we know about the impact and value of open data? One of the reasons for interest in Open Data is its potential economic value. A recent report by the McKinsey Global Institute estimated the economic value of open data at $3 trillion per year, which reflects the commercial value of opening up public data.
Another view of the value of open data comes from the open government community. A survey of country experts who provided assessments for the Open Data Barometer assessed the impact of open government data. (Figure 1) They found the highest impact to be on transparency and accountability of governments. It is interesting to note that impact of open data on economic growth and environmental sustainability is less widely recognized.
Figure 1: The Impact of Open Data
Source: Web Foundation, “Open Data Barometer: 2013 Global Report”
What data are open?
There are many definitions of open data, but they all agree broadly on these five points: open data are free, available to all, accessible, licensed for use and reuse, and well documented. Figure 2 shows an example of how the open data definition has been applied to assessing the openness of government datasets. The Open Data Index uses 9 criteria and applies them to 10 categories of data in 70 countries. Such indexes are popular because they summarize a lot of information – even if they oversimplify. In this example, the 9 characteristics are applied to datasets providing public transport time tables. The goal is to see how far countries have gone in opening their datasets.
Figure 2: Rating Openness
Source: Open Knowledge Foundation
https://index.okfn.org/country/dataset/timetables
Much of the focus on the open data movement has been on government data that are not produced by the national statistical system, such as timetables, postcodes, maps, election results, legislative records, and the budget. Of the ten datasets included in the Open Data Index, perhaps three are the customary responsibility of the national statistical office. Of the fourteen datasets included in the Open Data Barometer six or seven are outputs of the national statistical system. Notably missing from both indexes are labor force statistics, money and banking statistics, price statistics, and measures of poverty and social welfare. (See “Monitoring Data Quality and Openness” for further discussion of the two indexes.) In many cases, national statistical offices already have dissemination programs that make many of these data public, although they may not satisfy all the requirements for open data. Our goal should be that the major outputs of the national statistical system such as the census, the national accounts and balance of payments, labor force, trade, finance, and household and business surveys, are evaluated by such assessment systems.
Whose data are open?
So far the attention of the open government and open data movements has been largely on high-income and middle-income countries. Most low-income countries have been left out. The Open Government Partnership launched in 2011 has been a prominent advocate for open data. There are now 64 countries that are members of the partnership and have subscribed to its principles. Of these 27 were classified by the World Bank as high-income (GNI per capita greater than $12,616 in 2012) and 20 as upper-middle-income (GNI per capita between $4,085 and $12,615). There are 12 lower-middle-income members and only 5 low-income (GNI per capita of $1,035 or less). The regional distribution of members is highly skewed toward Europe and the Americas. Only 7 members come from Sub-Saharan Africa. The Open Data Index and Open Data Barometer have a similar bias. Both omit more than half the countries classified as low- or middle-income.
Including developing countries is a major challenge for the open government and open data movements. National statistical offices should be at the center of this effort. Even where high-level political commitment is lacking, national statistical offices can provide a natural entry point. Under the United Nations Fundamental Principles of Official Statistics statistical offices have a mandate to publish their data under terms that closely parallel standards for open data. National statisticians already participate in international networks through the UN Statistical Commission, regional economic commissions, and non-governmental bodies such as the International Statistical Institute. But for many poor countries, there is a significant gap between the willingness of the statistical office to provide data and its capacity to do so.
Support for Open Data
The good news is that support for open data is growing. The World Bank’s Open Data Readiness Assessment tool identifies the steps necessary to create an open statistical system and complements national strategies for the development of statistics. There are new partnerships around open data, which bring with them access to valuable technical support and knowledge. The recently formed Partnership for Open Data has resources to promote the benefits of open data in developing countries. So far financial support from donors has been meager, but some are beginning to show interest in open data reforms. Some trust fund and grant donors seem to be interested in folding open data projects into larger statistical systems improvement projects. More needs to be done to mainstream open data national statistical development plans, and national statistical offices need to increase their efforts to tap into this growing source of knowledge and technical and financial support.
What Can NSOs Do to Open Data?
National statistical offices can play a key role in an open data initiative. Our advice, based on the experience of those that have gone ahead, is for national statistical offices to align their work with the national open data initiative, where possible. And if there is none, go ahead anyway. We recommend updating the national statistical plans and dissemination policies. We also encourage statistical offices to take advantage of partnerships with others who have done it and to reach out to current and future data users and stakeholders.
Open Data is a change management program. Like any change management program there is some pain for the gain. NSOs have to organize, standardize, and prioritize. Once they are open, many data sites have seen huge increases in the number of users. For example, the World Bank saw its “hits” increase by 20 times after it opened its databases. NSOs will also gain better quality data and metadata and lower costs of dissemination through the investments in the user interface and bulk download facilities.
Partners in the Data Revolution
Open data can indeed be the key to unlocking the data revolution, helping national statistical offices to improve their statistical systems and better meet their users’ needs. To mobilize and coordinate a global data revolution including open data as one of its goals, we need to build a broad coalition in support of open data and we need donors to provide support for open data that focuses on building national capacity (and not just extracting data). As partners we should encourage innovation and sharing of knowledge and tools. And as the High Level Panel reminded us, we must always be willing to measure impacts and account for our results.
This article is based on a talk given by Shaida Badiee to the Seminar on Emerging Issues “Managing the Data Revolution” on 29 February 2014 in conjunction with the 45th session of the UN Statistical Commission. A copy of the PowerPoint slides from the talk is available on the ODW website.
