A Comment on Chandy and Zhang, “Stuffing data gaps with dollars: What will it cost to close the data deficit in poor countries?”
by Eric Swanson
Brookings Institution researchers, Laurence Chandy and Christine Zhang1,Anchor: #1  have recently published estimates of the cost of filling the gaps in existing census and survey programs needed to produce indicators for monitoring the Sustainable Development Goals (SDGs). Their article entitled “Stuffing data gaps with dollars: What will it cost to close the data deficit in poor countries?“ appears to demonstrate that other cost assessments, such as the Data for Development assessment published by the Sustainable Development Systems Network (SNSN)2 Anchor: #2 , are far too high and the cost of the additional surveys needed to provide adequate coverage of core SDG indicators is small.
The Chandy-Zhang method is clear and concise and they adopt the same unit costs used in the SDSN study, but they overlook the substantial spending needed just to maintain the current level of statistical production. In the end the principal difference is in presentation not in substance. But presentation matters. Just as Morten Jerven3 Anchor: #3 scared people with an outlandish figure for the cost of producing SDG indicators in his Benefits and Costs of the Data for Development Targets, Chandy-Zhang run the risk of creating complacency over the need for serious, continuing investments in the statistical capacity of developing countries. Neither overstatement nor understatement will encourage wise decision making.
Adopting a strictly marginalist approach, Chandy-Zhang look only at the difference between the frequencies with which censuses, health surveys (such as the DHS and MICS), and living standards surveys (such as the World Bank’s LSMS) have been conducted in the recent past by developing countries and a recommended frequency. For censuses, they accept the common standard of one per decade. For living standard surveys, they take two per decade as a minimum and consider five per decade as an alternative. Then they adopt the SDSN unit cost estimates to calculate the cost of the additional surveys needed in each deficient country to reach the standard. For LSMS-type surveys, they conclude that an additional 100 living standard surveys, or 10 a year, at a cost of “a paltry $17 million,” would bring all countries to a minimum frequency of two per decade. At the much higher rate of five per decade, an additional $35 million a year would be needed. For health surveys, such DHS and MICS, the shortfall is $6 million year, and for censuses $108 million a year.
In contrast, the SDSN study finds a need for spending $134 to $173 million a year on household survey programs and $320 million on censuses, in addition to annual expenditures of $220 million to upgrade civil registration and vital statistics, $90 million on educational management information systems, $23 million on economic statistics, $80 million on geospatial systems, and $34 million on other environmental monitoring. The grand total is around $900 to $940 million a year for the next 15 years, about half of which would need to be met by donor contributions. The increase over current levels of aid for statistics is about $200 million. Although still small numbers compared to the grand scale of official development assistance, they are not “paltry.”4Anchor: #4 
The largest difference between the two methods is that Chandy-Zhang look only at incremental requirements and assume that all existing programs will continue unchanged under whatever funding arrangements exist. But this is ingenuous. In the long run – the fifteen years of the SDGs – everything is incremental. The SDSN looked at the full cost of the programs and then backed out a donor share based on a review of recent levels of donor support for statistics. Whether donors or countries will sustain and then increase their spending on statistics remains an open question, but at least we have a notion of the full cost.
In the interest of comparing like with like, we estimate the cost of the missing living standard surveys the countries included in the SDSN study. The SDSN cost assessment was limited to 77 low- and lower-middle income countries that are currently IDA eligible or so-called blend countries, representing the poorest and most aid dependent countries in the world. (Chandy-Zhang define their sample of developing countries somewhat differently.) Based on World Development Indicators, over the ten years from 2002 to 2011, 40 of these countries conducted fewer than two surveys and 71 conducted fewer than five.5Anchor: #5  To make up the shortfall from two per decade, an additional 64 surveys would have been needed, and to make up the shortfall from five per decade, an additional 268 surveys. Pricing these at $1.7 million each, the incremental costs are $10.9 million and $45.6 million a year. The difference from Chandy-Zhang is entirely due to differences in the countries included.
How do these costs compare with the SDSN estimates? By construction they are identical. The estimated cost of conducting three living standard surveys for 77 countries over 15 years – $393 million out of a total of $2.0 to $2.6 billion for all household surveys — was included in the SDSN estimates. The difference with Chandy-Zhang is that their approach looks at the historical shortfall while the SDSN method looks at the full cost of the program going forward. And while Chandy-Zhang do not make an explicit assignment of total costs between donors and developing countries, they suggest that the incremental costs could be borne entirely by the World Bank, UNICEF, and USAID. Perhaps they could and should. But the full cost of enabling the statistical systems of 77 of the poorest countries in the world to monitor their own development programs will be substantially higher and will require significant commitments from many donors and from each and every country.
Chandy-Zhang conclude with three recommendations that deserve comment.
In the first, they ask the IMF to review the “standard” for the frequency of household surveys and censuses in the GDDS (General Data Dissemination System). It should be noted that the GDDS does not set standards and the IMF does not take responsibility for social statistics. The frequencies cited by Chandy-Zhang are recommendations, which were included in data quality assessment frameworks for population, poverty, and health statistics prepared by the World Bank. Whether these frequencies should be revised is an issue that should be taken up by the various international organizations that sponsor surveys and censuses, the countries themselves, and the expert groups that advise them.
Next, they ask the World Bank to commit to completing a living standard survey by 2020 in all countries that fall short of the GDDS recommendation. Even if the World Bank were to do this, it would not address the larger problem of establishing a sustainable statistical program. A more holistic approach of the type described in the SDSN cost assessment is needed.
The same could be said of their recommendation that UNICEF and USAID support countries that have not conducted a health survey within the last five years. UNICEF and USAID are already providing significant support for health surveys, which has been essential to creating and maintaining the existing record of health statistics. More will be needed in the future, but that is not all.
Building a sustainable data ecosystem in developing countries will require investments both from external donors and by countries themselves with further support coming from civil society organizations and the private sector. It will also require engagement of both data producers and data users. And a commitment to open data. This is what is meant by a data revolution.
References
1. Chandy, Laurence and Christine Zhang. “Stuffing data gaps with dollars: What will it cost to close the data deficit in poor countries?“(31 August 2015). The Brookings Institution. http://www.brookings.edu/research/opinions/2015/08/31-data-deficit-poor-countries-chandy-zhang
2. Espey, J., et al., (2015) Data for Development: A Needs Assessment for SDG Monitoring and Statistical Capacity Development. SDSN, Open Data Watch, Paris21, UNIDO, UNICEF, ODI, CIESIN, World Bank Group and Simon Frazer University. SDSN: New York. Available at: http://unsdsn.org/resources/publications/a-needs-assessment-for-sdg-monitoring-and-statistical-capacity-development/.
3. Morton Jerven (2015), Benefits and Costs of the Data for Development Targets for the Post-2015 Development Agenda, http://www.copenhagenconsensus.com/sites/default/files/data_assessment_-_jerven.pdf
4. Official Development Assistance recorded by the OECD for 2014 was $135.2 billion. See http://www.oecd.org/dac/stats/development-aid-stable-in-2014-but-flows-to-poorest-countries-still-falling.htm
5.Based on recorded values for the poverty headcount ratio at the international poverty line in the World Development Indicators database: http://databank.worldbank.org/data/reports.aspx?source=world-development-indicators. Downloaded 6 September 2015.