By Eric Swanson and Lorenz Noe, Open Data Watch
The world is short of data. At a moment when everyone talks about the growing volume of big data, that may seem a contrarian statement. But a look at the websites of national statistical offices or the database of the Sustainable Development Goals (SDG) indicators shows many gaps. The median development data coverage score of the 178 countries included in the Open Data Inventory is only 44 percent, and in the recent Bridging the Gap report, Open Data Watch documented large coverage gaps in 104 indicators needed to measure the well-being of women and children in 15 African countries (ODW 2019). In this blog, we move from the general to the specific, looking at the gaps in data needed to monitor the first of the SDG goals: eradicating poverty.
The results are not reassuring. After a decade of progress through 2011, the number of countries with two or more poverty estimates in a ten-year period has fell from 122 to 107 in the decade ending in 2017. Looking ahead, we find that 75 low- and middle-income countries are in danger of not having enough surveys to report reliable trends through 2020. Due to lags in reporting, we find ourselves in late 2019 with little information on surveys conducted in the past three years. As the world embarks on the “Decade of Delivery and Action” for sustainable development, missing data will severely hamstring efforts to fight poverty and deliver solutions in places where help is most needed.
Umar Serajuddin and colleagues at the World Bank drew attention to the sparsity of poverty data with their paper, Data Deprivation: Another Deprivation to End (Serajuddin et. al. 2015). They used the record of poverty estimates published in the World Development Indicators (WDI) database (World Bank, n.d.a) to document the frequency of and intervals between poverty estimates. Out of 155 countries in their sample, they found 29 countries lacked any published observation on poverty over the period 2002 to 2011, the most recent period available to them. Another 28 had only one observation, and 20 more had two, widely separated observations. They identified these 77 countries as data deprived and called for greater attention to statistical capacity building by the international community. As they point out, the lack of regular observations on social and human development is an obstacle to implementing policies to achieve the SDGs. It is in that sense that they term the lack of data a “deprivation,” akin to the deprivations of hunger or lack of health and education.
New estimates of data deprivation
Although there are many causes of the lack of poverty data, Serajuddin et. al. identify “the main bottleneck for poverty data [as a] lack of household surveys.” In this blog we revisit their 2015 paper in several ways. First, we use the record of household income and expenditure surveys maintained by the World Bank’s PovcalNet database (World Bank, n.d.b), rather than the national and international poverty estimates published in the WDI. While it is arguable that publication of the indicator estimates is most relevant for informing policies, we are principally concerned with the existence of surveys needed to construct poverty indicators and their frequency. Second, we extend the time interval over which data are available through 2017 and calculate the deficit in surveys over the period 2011 to 2020.
Like Serajuddin et. al., we find the number of countries with surveys has been increasing. The PovcalNet archive now includes surveys from high-income as well as low- and middle- income countries. It also includes surveys that were administered only in urban areas of some countries and records separately surveys that provide an income-based or expenditure-based poverty estimate. For this analysis we included only surveys that are national in scope and, to avoid double counting, we count only once a survey that may have generated both income- and expenditure-based estimates. For a few countries, PovcalNet records separately surveys conducted in urban and rural areas in the same year. These too are counted as a single survey in our analysis.
Figure 1 shows the number of surveys available in each year by income group. Our sample differs from the 155 countries included in the earlier study: we include all countries that have at least one survey in PovcalNet over the period 1981 to 2017. There are 164 countries in our sample, of which 27 are currently classified by the World Bank as low-income, 97 as middle-income, and 40 as high-income. Together they produced 1563 surveys between 1981 and 2017. Not included are 53 countries or territories for which population data are available from the WDI database but for which there is no record of any survey in PovcalNet.
Figure 1 Surveys recorded in PovcalNet
The peak year of survey activity was 2010 when 89 countries conducted household income or expenditure surveys. The sharp rise in survey activity from 2002 onwards may reflect the demand for indicators to monitor progress toward the first Millennium Development Goal (MDG): Eradicate extreme hunger and poverty. Some of the increase is also due to the inclusion of more surveys from high-income countries in PovcalNet.
Like the MDGs, the SDGs target poverty eradication, so the decline in poverty surveys in 2016 and 2017 most likely reflects lags in reporting and curating surveys included in PovcalNet and not a substantial reduction in effort. But as much as countries and international organizations have tried to improve the measurement of poverty, large gaps in the statistical record remain.
Survey frequency
To monitor changes in poverty rates, at least two observations are needed, but there does not appear to be a consensus on the optimal frequency of poverty surveys. The cost of conducting surveys and the capacity of statistical offices to manage and analyze data limits how often surveys can be conducted, particularly in low-income countries. Serajuddin et. al. suggest an interval of three to five years between surveys. The Data for Development report (SDSN 2015) and subsequent estimates of the cost of producing statistics for the SDGs assumed household income or expenditure surveys would be conducted every four years. For our purposes here, we follow Serajuddin et. al. and adopt five years as the maximum acceptable interval between surveys.
In Data Deprivation, gaps in poverty estimates are classified into five categories:
- Extreme deprivation – No data in a ten-year reference period
- Moderate deprivation – One observation in a ten-year period
- Vulnerable to data deprivation — two data points in a ten-year period, but at an interval of more than five years apart
- Minimum requirement – two data points in a ten-year period within five years or less
- Satisfactory – three or more data points in a ten-year period
Figure 2 shows the availability of surveys for the 164 countries in our sample in a ten-year rolling window starting in 1990. There was a decline in the number of countries with fewer than two surveys through 2011, but there were an additional 31 countries with two observations more than 5 years apart, leaving only 91 that meet or exceed the minimum requirement proposed by Serajuddin et. al. With 44.5 percent of our sample at or below the minimum threshold, these estimates are somewhat better than the results reported by Serajuddin et. al. who found half their sample of 155 countries to be vulnerable or data deprived for the same period. Lags in incorporating poverty observations in the WDI database toward the end of their observation period may explain some of this difference, just as we expect more surveys for 2016 and 2017 to appear in PovcalNet over the next several years. Of greater concern is the continuing rise in the number of countries with no surveys or only one survey in the 10-year window, which by 2017 was above the level last reached in 2007.
Figure 2 Evidence of data deprivation
Countries at risk
As figure 2 shows, there has been no reduction in data deprivation since 2010 and many countries remain vulnerable. In table 1, we list 75 low- and middle-income countries that as of 2017, the last year for which data are available, had conducted only one or no surveys since 2011 (Six high-income countries also reported one or fewer surveys over the period.) Unless unreported surveys have already been conducted or new surveys will be conducted in the remaining years of this decade, these countries will arrive in 2020 in a state of moderate or extreme data deprivation. This means that policy makers, citizens, and other stakeholders including bilateral and multilateral donors will lack sufficient information to assess progress toward the SDG goal of eradicating extreme poverty everywhere
.
Table 1
Low- and middle-income countries in danger of data deprivation by 2020
| Country | No. of surveys in last 7 years | Year of last survey | Country | No. of surveys in 7 years | Year of last survey | |
| Guyana | 0 | 1998 | India | 1 | 2012 | |
| Turkmenistan | 0 | 1998 | Iraq | 1 | 2012 | |
| Belize | 0 | 1999 | Lao PDR | 1 | 2012 | |
| Suriname | 0 | 1999 | Lebanon | 1 | 2012 | |
| Uzbekistan | 0 | 2003 | Madagascar | 1 | 2012 | |
| Jamaica | 0 | 2004 | Mauritius | 1 | 2012 | |
| Syrian Arab Republic | 0 | 2004 | Tanzania | 1 | 2012 | |
| Azerbaijan | 0 | 2005 | Fiji | 1 | 2013 | |
| Kiribati | 0 | 2006 | Micronesia, Fed. Sts. | 1 | 2013 | |
| Venezuela | 0 | 2006 | Samoa | 1 | 2013 | |
| Cabo Verde | 0 | 2007 | Solomon Islands | 1 | 2013 | |
| Central African Republic | 0 | 2008 | Burkina Faso | 1 | 2014 | |
| Angola | 0 | 2009 | Burundi | 1 | 2014 | |
| Eswatini | 0 | 2009 | Cameroon | 1 | 2014 | |
| South Sudan | 0 | 2009 | Comoros | 1 | 2014 | |
| Sudan | 0 | 2009 | Guatemala | 1 | 2014 | |
| Guinea-Bissau | 0 | 2010 | Mauritania | 1 | 2014 | |
| Jordan | 0 | 2010 | Morocco | 1 | 2014 | |
| Lesotho | 0 | 2010 | Mozambique | 1 | 2014 | |
| Maldives | 0 | 2010 | Nicaragua | 1 | 2014 | |
| Mali | 0 | 2010 | Timor-Leste | 1 | 2014 | |
| Nepal | 0 | 2010 | Yemen | 1 | 2014 | |
| Nigeria | 0 | 2010 | Cote d’Ivoire | 1 | 2015 | |
| Papua New Guinea | 0 | 2010 | Gambia | 1 | 2015 | |
| Sao Tome and Principe | 0 | 2010 | Myanmar | 1 | 2015 | |
| Tuvalu | 0 | 2010 | Namibia | 1 | 2015 | |
| Vanuatu | 0 | 2010 | South Africa | 1 | 2015 | |
| Algeria | 1 | 2011 | Tajikistan | 1 | 2015 | |
| Bosnia and Herzegovina | 1 | 2011 | Tonga | 1 | 2015 | |
| Chad | 1 | 2011 | Tunisia | 1 | 2015 | |
| Congo, Republic of | 1 | 2011 | Zambia | 1 | 2015 | |
| Senegal | 1 | 2011 | Bangladesh | 1 | 2016 | |
| Sierra Leone | 1 | 2011 | Botswana | 1 | 2016 | |
| Zimbabwe | 1 | 2011 | Kenya | 1 | 2016 | |
| Albania | 1 | 2012 | Malawi | 1 | 2016 | |
| Congo, Dem. Rep. | 1 | 2012 | St. Lucia | 1 | 2016 | |
| Guinea | 1 | 2012 | Gabon | 1 | 2017 | |
| Haiti | 1 | 2012 |
Conclusion and further work
Serajuddin et. al. and our recent work have shown that many countries lack adequate data on poverty. Without sustained, regular measurements using comparable instruments, trends in poverty and the associated characteristics of people, households, and their communities cannot be monitored or understood. Poverty surveys gather more than data on income or consumption; they record education levels, health status, employment, and living conditions of individuals and households, all of which are needed to inform public policies and private investments. But they are not the sole source of development data, nor can they supply all the data for monitoring the SDGs. A combination of censuses, surveys, and administrative data are needed to provide a complete picture of the social, economic, and environmental conditions that affect the welfare of people.
There are many possible ways to measure data deprivation. Gaps in poverty surveys or the indicators based on those surveys are only one measure. The Open Data Inventory for example, measures the coverage of 21 categories of statistics, including poverty and income indicators, taking into account the frequency of observations and the availability of subnational data. Further work is needed to provide an adequate accounting of the availability and usability of the microdata underpinnings of widely used aggregate indicators.
Our initial investigations into the microdata foundations of the SDG indicators suggest that in many countries, particularly low- and lower-middle-income countries, survey programs are conducted haphazardly, with their content and frequency determined more by the interest of bilateral and multilateral sponsors than by national needs. International guidelines for survey design and frequency would help to inform planning and implementation of survey programs. Financial, technical, and political support for the collection, dissemination, and use of census and survey data is also needed.
We have also encountered another gap in preparing this review. The lack of forward planning for data collection leaves unanswered – or unanswerable — the question raised by table 1: will countries that lack recent surveys be able to close those gaps and provide timely data on important development indicators? Information about surveys planned or underway would be of value to everyone who must make decisions now or in the future on the basis of empirical evidence.
References
Open Data Watch (ODW). 2015. Bridging the Gap: Mapping Gender Data Availability in Africa. https://opendatawatch.com/publications/brochure-bridging-gender-data-gaps-in-africa/.
_____. n.d. Open Data Inventory. https://odin.opendatawatch.com/.
Serajuddin, Umar; Hiroki Uematsu, Christina Wieser, Nobuo Yoshida, and Andrew Dabalen. 2015. Data Deprivation: Another Deprivation to End. World Bank: Policy Research Working Paper 7252. http://documents.worldbank.org/curated/en/700611468172787967/Data-deprivation-another-deprivation-to-end
Sustainable Development Solutions Network (SDSN). 2015. Data for Development: A Needs Assessment for SDG Monitoring and Statistical Capacity Development. https://sustainabledevelopment.un.org/content/documents/2017Data-for-Development-Full-Report.pdf
United Nations Statistics Division. n.d. SDG Indicators database. https://unstats.un.org/sdgs/indicators/database/
United Nations. n.d. SDG Indicators Database. https://unstats.un.org/sdgs/indicators/database/
World Bank. n.d.a. World Development Indicators. https://databank.worldbank.org/source/world-development-indicators
_____. n.d.b. PovcalNet. Data downloaded 4 October 2019. http://iresearch.worldbank.org/PovcalNet/povOnDemand.aspx
