The past two decades have seen efforts on multiple fronts to improve the quality and availability of what we will call development data: the statistical information needed for planning, monitoring, and assessing the social and economic development of a country. Although our focus is on poor developing countries, the same data are needed by rich countries to set policies, monitor programs, and report on results to their citizens. But while the rich, industrialized countries of Europe, North America, and Asia have well established national and subnational statistical systems, the statistical systems of poor countries, especially those in Africa and parts of Asia, are underfinanced with undertrained staff lacking the skills and equipment to produce reliable statistics across the broad range of issues needed by their government, business sectors, and citizens. The same definitions, standards, and methods that are applied to the production of statistics in rich countries should apply to statistical systems in developing countries. The challenge for all those who work with development data is to make it possible for developing countries to produce high quality statistics at a reasonable cost that serve the needs of their country.
Data gaps and new demands
Much of the recent effort to increase the reliability, frequency, and coverage of development data has focused on the indicators needed to monitor progress toward the Millennium Development Goals (MDGs). The 2004 Marrakech Action Plan for Statistics, which set the agenda for international support for statistics in developing countries, included improvements in MDG indicators among its six action items. The availability of the MDGs indicators has improved, but many of these improvements were the result of modeling, special surveys, or other data collection programs funded and carried out by bilateral donors and international agencies. In poor countries many gaps remain in fundamental statistical series: demographics, economic well-being, educational attainment, health status, and the economic and physical condition of the built and natural environment. The recent report by the Data2X project of the UN Foundation documented extensive gaps in the availability of gender-disaggregated statistics needed to document the status of women and girls.
The challenges facing statistical offices are growing larger. The indicators proposed for the Sustainable Development Goals for 2030 are more numerous and complicated than the MDGs. The 2009 Sen-Stiglitz-Fitoussi report on the Measurement of Economic Performance and Social Progress has laid out an ambitious program of fundamental changes in the measurement of economic, social, and environmental indicators. But six years after the publication of the 2008 revision to the System of National Accounts, many developing countries are still trying to bring their national accounts into conformity with the 1993 revision.
Quality assurance frameworks – work in progress
In 1996 the International Monetary Fund proposed a system of voluntary standards for certain categories of economic and financial data. The Special Data Dissemination Standard (SDDS) was intended to apply to high- and middle-income countries participating in international financial markets. In 1997 the IMF launched a less stringent program, the General Data Dissemination System (GDDS), which encouraged countries with less developed statistical systems to report on existing statistical practices and to set priorities for improving them. Both systems emphasized the importance of disseminating reliable information to final users, although neither specially mandated open data standards. In 2001 the IMF proposed to standardize the framework through which it documented the quality-related features of the governance of statistical systems, statistical processes, and statistical products. This became known as the Data Quality Assessment Framework (DQAF). The IMF has produced DQAFs for important economic and financial datasets. In cooperation with the World Bank it has also produced a DQAF for poverty statistics and population statistics. UNESCO working with the World Bank has produced a DQAF for education statistics.
While the GDDS and DQAF have provided a useful organizing framework for documenting statistical practices and can and should provide input the construction of a NSDS, but they are not complete in many developing countries; they do not cover all important statistical sectors; and many of those available are out of date. Other international, regional, and national statistical agencies have developed quality assurance frameworks. Some are focused on specific sectors, others take a system-wide approach. But there is no universally recognized standard for such frameworks.
Building statistical capacity: more is needed
The creation of PARIS21 in 1999 established a center for advocacy on behalf of national statistical systems and a meeting place for national statisticians, policy makers, and international donors. The PARIS21 Guidelines for national strategies for the development of statistics (NSDS) have encouraged a more rigorous approach to capacity building and the development of donor partnerships to support statistics. Besides drawing attention to the need for MDG indicators, the Marrakech Action Plan for Statistics mobilized more than $100 million for partnerships that supported the completion of the 2010 round of censuses, implemented the World Bank’s Accelerated Data Program to harvest indicators from existing surveys, and established the International Household Survey Network. Donor support for statistics increased by 60 percent, reaching $1.6 billion in the period 2008-2010. In 2011 the five-point Busan Action Plan for Statistics was adopted at the 4th High Level Forum on Aid Effectiveness. It called for fully integrating statistics in decision making; promoting open access to statistics; and increasing resources for statistical systems. According to the most recent PRESS report by PARIS21, donors provided $394 million for statistics in 2013, an increase over 2012 but 23 percent less than in 2011.
Despite these efforts, international support for statistics in developing countries remain fragmented and lacking a consensus on priorities, methods, and funding. In our experience, poor countries are unable or unwilling to borrow substantial sums of money to finance statistical projects and make do with limited grants. Donors too have been reluctant to finance comprehensive statistical capacity building programs. To make progress national statistical offices will need to demonstrate better uses of the existing funds and convince donors, other stakeholders, and their own governments to bring additional money to the table. As far as possible, funding should be used to support system-wide reform and capacity building, rather than buying “products” such as surveys or the delivery of indicators. Perhaps donors would be more willing to support performance-based programs linked to progress towards accuracy, timeliness, and openness of core statistics if they had a clearer picture of how to measure success.
A global approach to data quality
A recent report by the Oxford Martin Commission for Future Generations – Now for the Long Term – suggests the creation of a new international agency to “undertake quality control of global statistics, assess domestic practices, regulate misuse, and improve data collection” (pages 59-60), in short, a global quality assurance program (for the purposes of this paper, GQAP). Working as an independent body within the existing international system governed by the UN Statistical Commission and in cooperation with the statistical offices of the regional and specialized agencies of the United Nation, could perform three complementary roles:
1. It could act on behalf of policy makers, researchers, and other data users as an independent watchdog organization with a high-level of credibility among countries, international organizations, and statisticians to identify deficiencies in data quality and coverage, propose methodological improvements, and call out failures to adhere to the UN Fundamental Principals of Official Statistics or the misuse or misrepresentation of statistics.
2. It could make the case and build partnerships for global or regional actions to address high priority gaps in coverage and quality and recommend implementation plans. It should do so based on rigorous and transparent assessment of the costs and benefits to the global community and the financial burden on developing countries.
3. In some cases it might help to fast track initiatives that have been delayed or play a more active role in implementation of statistical programs of its own design. In other cases it should defer to agencies that have taken responsibility for specific sectors while encouraging them to extend the scope and impact of their work. GQAP will achieve greater influence from its ideas and the knowledge it generates than it can by operating programs.
As a new entrant to a complex environment, GQAP should begin with a scoping exercise that both surveys the field and identifies likely points of action. It would be particularly helpful to look for synergies and redundancies and opportunities for increased efficiency in the global statistical system. The scoping exercise should be accompanied by realistic costing. The five-point action plan outlined in the Busan Action Plan and the complementary actions to improve gender statistics endorsed as part of the Busan Joint Action Plan for Gender Equality and Development would make a good starting point. The data quality assessment frameworks discussed above provide a starting model for the systematic collection of country-level, quality-related metadata.
