|
|
|
The Open Data Inventory (ODIN) 2017 assessed the coverage and openness of official statistical data in 180 countries, representing over 99 percent of the world’s population. Assessments were carried out between June and October of 2017. All results are based on the data available from countries’ principal NSO websites and linked official websites during the assessment period. Assessment results were reviewed and, when necessary, revised between October and December 2017. To ensure that datasets were not overlooked, officials from national statistical offices were invited to suggest additional data sources before their results were finalized. (See the ODIN 2017 Annual Report.for an overview analysis of results.)
ODIN 2017 includes nearly all high-income and OECD countries and most low- and middle-income countries. The assessments analyzed datasets in 21 data categories that are the most pertinent to managing and monitoring progress on the Sustainable Development Goals (SDGs) and the social, economic, and environmental development of a country. Figure 1 shows the scores of countries included in ODIN 2017 grouped by quintiles. Countries not included are shown in gray.
ODIN scores are calculated as a percentage of the maximum score obtainable. The median ODIN country score for 2017 is 37.4, meaning that fewer than half the countries satisfy more than 38 percent of the ODIN criteria for data coverage and openness across all data categories. National scores range from 80 for Denmark to 3 for Chad.
The highest scoring country was Denmark. Four new countries moved into the top ten list –Netherlands, Bulgaria, Mexico, and Slovenia – displacing Czech Republic, Lithuania, Estonia, and the United States. Rwanda, ranked 41st in the world, was the highest scoring low-income country.
All ODIN results are available at odin.opendatawatch.com. The ODIN website now includes an option for customized weighting of data categories and elements. With this option, users can create their own indexes or redefine the scores and subscores produced by ODIN.
What is ODIN?
The Open Data Inventory (ODIN) is an evaluation of the coverage and openness of data provided on the websites maintained by national statistical offices (NSOs). The overall ODIN score is an indicator of how complete and open an NSO’s data offerings are. The summary scores for social, economic, and environmental statistics and summary scores for coverage and openness provide a picture of the national statistical systems’ strengths and weaknesses.
When and how was ODIN 2017 conducted?
The ODIN 2017 assessments were carried out between July and October 2017. To conduct these assessments, Open Data Watch hired and trained a group of researchers skilled in various languages to complete the first round of assessments. Following the initial assessment, each country underwent two layers of reviews and, if necessary, revisions between October and December 2017.
What is ODIN’s purpose?
ODIN helps identify critical gaps, promote open data policies, improve data access, and encourage dialogue between NSOs and data users. NSOs and their development partners can use ODIN as part of a strategic planning process and as a measuring rod for the development of the statistical system. ODIN provides valuable information to data users within the government and private sectors and to the public about the availability of important statistical series. In addition to the ratings of coverage and openness in twenty-one statistical categories, ODIN assessments record the online location of key indicators in each data category, permitting quick access to 56 indicators.
Why assess national statistical offices?
ODIN assessments begin with the websites maintained by national statistical offices because, in most countries, the NSO is the lead agency of the national statistical system, coordinating its work with other governmental bodies that produce official statistics. If an official national data source can be reached from the NSO’s website, it is included in the ODIN assessment. NSOs, as producers and caretakers of official statistics, have a special obligation to maximize their public benefit. NSOs can and should be the leading advocates for and providers of high quality, official statistics to government, the public, and the international community.
How is open data defined?
There is general agreement on the core meaning of open data. As summarized in the Open Definition, version 2.1s, “Knowledge is open if anyone is free to access, use, modify, and share it — subject, at most, to measures that preserve provenance and openness.” This definition has been operationalized in the International Open Data Charter. In practical terms, open data should be machine readable in non-proprietary formats, accompanied by descriptive metadata and export options that allow customization and bulk download, and free to be used and reused for any purpose without limitations other than acknowledgement of the original source. These requirements have been incorporated in the five elements of the ODIN openness assessment.
What data categories are included?
ODIN assessments review published statistics in twentyone categories, grouped as social statistics, economic and financial statistics, and environmental statistics. The default ODIN overall score weights the three groups equally. In each category, representative indicators were selected because they are frequently needed for public policies or private initiatives and because they provide evidence of underlying statistical processes for which statistical offices are responsible. The data categories in ODIN 2017 are:
Social Statistics
- Population and Vital Statistics
- Education Facilities
- Education Outcomes
- Health Facilities
- Health Outcomes
- Reproductive Health
- Gender Statistics
- Crime and Justice Statistics
- Poverty Statistics
Economic Statistics
- National Accounts
- Labor Statistics
- Price Indexes
- Government Finance
- Money and Banking
- International Trade
- Balance of Payments
Environmental Statistics
- Land Use
- Resource Use
- Energy Use
- Pollution
- Built Environment
More information about the indicators in each data category and how data categories were scored can be found in tables 4a-4u.
How did NSOs participate in ODIN 2017?
Open Data Watch invited all countries’ national statistical offices to provide feedback on the datasets recorded in ODIN. Forty-eight national statistical offices participated in this NSO review process, where they had the opportunity to provide suggestions and feedback on datasets. For these participating countries, slight changes were made to the assessment and review process: upon completion of the first assessment and the first layer of reviews, NSOs were given the change to review the data sources and provide feedback. If necessary, revisions arising from their feedback underwent an additional layer of review in additional to a final review to ensure all changes align with ODIN methodology.
Methodology Changes
New data category
The prosecution of crime and operation of a justice system are important functions of justice. In 2017 ODIN added a new category of crime and justice statistics, changing the number of data categories from 20 to 21. The representative indicators and disaggregations for this data category were selected after consultation with the United Nations Office of Drugs and Crime (UNODC). For more information on the scoring of this data category, see page 27 of this report.
New indicators
New indicators were added in ODIN 2017 to better represent the types of data most often sought by data users. The new indicators are listed below, along with their corresponding data categories. These additions may have changed the minimum amount of data needed to score full points for coverage within the data categories. The changes reflect the growing number of datasets critical to monitoring sustainable development.
- Data on child marriages (gender statistics)
- Stunting, wasting, or obesity rates (health outcomes)
- Interest rates (money and banking)
- Data on protected lands (land use)
- Data on timber harvests or deforestation (resource use)
NSO engagement
Beginning April 2017, Open Data Watch formally invited 180 NSOs to participate in the review of their country assessments. Upon confirmation, NSO representatives received a list of data sources for each indicator assessed in ODIN and were invited to suggest additional sources of data. All feedback was reviewed and taken into consideration if it met the methodology guidelines. Sixty-five NSOs confirmed participation and 48 ultimately submitted feedback. All countries were contacted at least three times between April and June 2017.
The NSO review process was introduced to ODIN 2017 as a response to the growing demand from NSOs to better understand ODIN methodology and their scores, as expressed at the 48th session of the United Nations Statistics Commission. It is also a chance for Open Data Watch to learn more about the country specific challenges to opening data and to use this feedback to strengthen the usefulness of ODIN to NSOs.
Change in education statistics
Within education statistics, the disaggregation of data by “school stage“ must now show three stages. One stage must be primary, general, or have a similar description. The second two stages must be secondary or tertiary. In ODIN 2016, no such requirement existed. This change has resulted in a decrease of many countries’ scores for education statistics. The change aligns our methodology more closely to the United Nations Educational, Scientific and Cultural Organization’s (UNESCO) International Standard Classification of Education.
Change in reproductive health statistics
Assessments of indicators for reproductive health are based on definitions by the World Health Organization (WHO). Data presented using alternative definitions are no longer accepted. The following is a list of current indicators in this category along with their definitions.
- (6.1) Maternal mortality ratio is the number of maternal deaths per 100,000 live births.
- (6.2) Infant mortality rate is the probability of a child born in a specific year or period to die before reaching the age of one, per 1,000 live births.
- (6.3) Under-5 mortality rate is the probability of a child born in a specific year or period to die before reaching the age of five, per 1,000 live births.
- (6.4) Fertility rate is the number of births per 1,000 women.
- (6.5) Contraceptive prevalence rate is the percentage of women who are currently using, or whose sexual partner is currently using, at least one method of contraception, regardless of the method used. It is usually reported for married or in-union women aged 15 to 49.
- (6.6) Adolescent birth rate, also known as age-specific fertility rate, is the number of births per 1,000 women aged 15-19
Scoring openness element: download options
In ODIN 2016, data categories could receive a maximum score on this openness element if all data had (1) an API or bulk download and (2) had user-selected download options. In ODIN 2017, a data category receives a maximum score if all indicators have (1) bulk download and (2) an API or user-selectable download options. This change was made to give bulk downloads increased emphasis, especially given their prominence in the Open Definition. APIs are not a substitute for bulk downloads and are better suited for certain users and scenarios. APIs and user-select download options both allow users to customize data exports to access a small part of the total available data, which is why these features are interchangeable for scoring purposes.
Bulk download definition
Bulk download is defined at the indicator level. In ODIN 2016, it was defined at the data category level. The current definition is: “The ability to download all available data recorded in ODIN for a particular indicator (all years, disaggregations, and subnational data) in one file, or multiple files that can be downloaded simultaneously.” The effect of this change is that countries with a small amount of data published will not have their openness score penalized, so long as that data are available in bulk. For countries who publish primarily in PDF files and saw their openness score increase since ODIN 2016, this is likely the reason.
Terms of use scoring
In ODIN 2017, any policy that prohibits commercial use is classified as restrictive and receives a score of zero for this element. Last year, a policy with this clause was classified as semi-restrictive and scored ½ point (out of a maximum of 1 point). As explained by the Open Data Institute, “A non-commercial provision is problematic primarily because of a lack of clarity around what constitutes ‘commercial’ usage.”t In some cases, a restriction on commercial use can be interpreted as usage for personal use only. This methodology change was made in ODIN 2017 to better align to the Open Definition and to address one of the most common restrictive clauses seen in website’s terms of use. Any restrictions that offer only exclusive access to data violates the open license requirements of free use and redistribution and act as a serious impediment to openness.
Metadata definition
In ODIN 2016, metadata were defined as either specific or general. Specific metadata was required in ODIN 2016 to receive a maximum score. Specific metadata was defined as specific details about the definition of the indicator or the method of data collection and compilation for that indicator. General metadata was defined as information about a large survey or group of data of which the indicator is part.
In ODIN 2017, metadata are classified as complete or incomplete. There are three specific criteria for complete metadata: (1) definition of the indicator, definition of key terms used in the indicator description (as applicable), or description of how the indicator was calculated; (2) publication date (date of upload), data compilation date, or date dataset was last updated; and (3) name of data source (which agency collected or is responsible for the data). Incomplete metadata are defined as metadata that only meets one or two of the above criteria.
This change was made to increase the objectivity of scoring this element, as well as ensure the common components of many metadata standards were being met. These components focus on descriptions about the product and processes of the data, not the format, technology used, or administrative components. As a result, some countries who received full points for metadata may see their scores decreased in ODIN 2017 since only one criterion was required for a maximum score.
Data Categories and Indicators
The following sections explain the what sources and types of data are assessed in ODIN.
Data sources
ODIN assesses the coverage and openness of statistics available from websites maintained by national statistical offices. Websites maintained by private or non-governmental agencies or international agencies are not included in the assessment. Websites maintained by other units of the national government or by sub-national governmental units are included if and only if they can be reached from the national statistical office website.
For example, if the national accounts are maintained by the central bank, then data would be included in the ODIN assessment only if the NSO’s website provides a link to the central bank’s website or if the NSO reproduces the data on its own website. ODIN is premised on the belief that NSOs can and should take responsibility for providing access to all official statistics. Additionally data that requires payment or registration to access is not considered in ODIN.
Data categories
ODIN assesses macro data. By this we mean data from surveys or administrative sources that have been aggregated above the level of individual respondents. We focus on these because they are the final product released by the NSO or other official agencies. They are used most frequently for policy making and for tracking policy outcomes. Microdata from censuses and surveys are very important, but require a different approach to assessing their openness, primarily because of privacy concerns.
Twenty-one categories of data are included in the ODIN assessment. Tables 1-3 list the data categories, the representative indicators, required disaggregation, and explanatory notes for each indicator. For the construction of summary measures, the data categories are grouped as social statistics, economic statistics, and environmental statistics.
Table 1. ODIN social data categories, 2017
| Data Categories | Representative Indicators | Disaggregation | Notes |
| (1) Population & Vital Statistics | (1.1) Population by 5-year age groups (1.2) Crude birth rate (1.3) Crude death rate |
(1.1) Sex; marital status (1.2) Sex; marital status (1.3) Sex |
(1.1) Population data not disaggregated by 5-year age groups can be recorded if available at the admin 1 or 2 level, but national data must also be available in a separate file. Data not disaggregated by 5-year age groups only impacts the administrative coverage scoring element. (1.1) Estimates and projections are accepted. (1.2, 1.3) Birth and death registration figures are not accepted, but number of births/deaths are accepted. Births and deaths presented as sex ratios are not accepted as disaggregated by sex. |
| (2) Education Facilities | (2.1) Number of schools or classrooms (2.2) Number of teaching staff (2.3) Annual education budget |
(2.1) Age; school stage; school type (2.2) Age; school stage; school type (2.3) School stage; functional categories (such as teachers, supplies, administration, etc.). |
(2.1-2.3) School stage at a minimum refers to primary, plus two other stages (ideally, secondary and post-secondary/tertiary). The two other stages must not be subsets of primary education. See UNESCO guidelines for more information on school stages. (2.1, 2.2) School type may refer to technical schools, public, private, etc. Age groups count as age disaggregation. (2.3) Education budget must reference corresponding year. GDP contribution to education is not accepted. Ministry of Education expenditures or allocations are accepted. |
| (3) Education Outcomes | (3.1) Enrollment rate (3.2) Completion or graduation rate (3.3) Competency exam results |
(3.1) Sex; school stage; age; school type. (3.2) Sex; school stage; age; school type. (3.3) Sex |
(3.1-3.3) Attainment, attendance, and literacy rates are not accepted. Age groups count as age disaggregation. (3.1-3.2) School stage at a minimum refers to primary, plus two other stages (ideally, secondary and post-secondary/tertiary). The two other stages must not be subsets of primary education. See UNESCO guidelines for more information on school stages. (3.1) Number of students is accepted for this indicator. (3.3) Matriculation exam results are accepted. |
| (4) Health Facilities | (4.1) Number of health facilities (4.2) Number of beds or health budget (4.3) Number of health care staff |
(4.1) Facility type (4.2) Facility type; department type; functional categories (for budget data only), (4.3) Facility type; department type; staff type |
(4.1-4.3) Facility type generally include clinics, hospitals, pharmacies, dental clinics, public/private hospitals, etc. (at least two must be present). Department type generally include surgery, ophthalmology, internal medicine, pediatric care, dentistry, etc. (at least two must be present). Some countries may have different designations and this is analyzed on a case by case basis. 4.1-4.3) Figures for the public sector only are accepted. Figures for the private sector only will be reviewed on a case by case basis. (4.2) Ministry of Health expenditures or allocations are accepted. |
| (5) Health Outcomes | (5.1) Immunization rate (5.2) Disease prevalence/incidence (5.3) Stunting, wasting, or obesity rate. |
(5.1) Age; sex (5.2) Age; sex (5.3) Age; sex |
(5.1, 5.3) Rates for children/infants is accepted as age disaggregation if the age group is defined. In general, age groups count as age disaggregation. Absolute figures are not accepted. (5.2) Data must show figures for at least two diseases unless dataare for HIV/AIDS, then that alone is sufficient. (5.2) Mortality rates disaggregated by disease type is not accepted. (5.3) Undernourishment rate is not accepted. Malnourishment rate is accepted. Underweight or overweight rates are accepted. Wasting is also referred to as low weight-to-height. Obesity/overweight is also referred to as high weight-to-height. Stunting is also referred to as low height-for-age. |
| (6) Reproductive Health | (6.1) Maternal mortality ratio/rate (6.2) Infant mortality rate (6.3) Under-5 mortality rate (6.4) Fertility rate (6.5) Contraceptive prevalence rate (6.6) Adolescent birth rate |
(6.1) Optional (6.2) Sex (6.3) Sex (6.4) Optional (6.5) Optional (6.6) Optional |
(6.1-6.6) Absolute figures cannot replace rates/ratios (as specified). See WHO definitions for each indicator. (6.3) Mortality rates disaggregated by age/age groups are accepted so long as rates for under 5 years of age can be extracted. (6.6) Birth rates disaggregated by age/age groups are accepted so long as rates for 15-19 years of age can be extracted. |
| (7) Gender Statistics | (7.1) Data on violence against women (7.2) Proportion of women in government or management (7.3) Data on child marriages |
(7.1) Violent offense type (7.2) Optional (7.3) Optional |
(7.1) Data on people’s opinions/ attitudes about violence against women are not accepted. (7.1) Data must show at least two violent offense types. Data must either show victimization rates for both men and women, or specify data are only for women (even if the offense is sexual assault, rape, or domestic violence). (7.2) Share of women in employment in the non-agricultural sector is not accepted. Number of women in certain professions may be accepted on a case by case basis. Data must be presented as a rate. Absolute figures are only accepted if the data are disaggregated by two sexes. (7.3) “Child” is defined as under 18 years of age. Countries may define “child” marriages differently and this data are accepted if maximum age is under 18. Data may also be called “early marriages.” Age of first marriage is accepted if figures are not averages, medians, or modes. |
| (8) Crime Statistics | (8.1) Homicide rate/ count (8.2) Crime rate/count 8.3) Persons in prison/ incarceration rate |
(8.1) Sex of victim; age of victim; sex of perpetrator; age of perpetrator; victim/ perpetrator relationship (8.2) Crime type; age of victim; sex of victim; sex of perpetrator; age of perpetrator; victim/perpetrator relationship (8.3) Sentenced/ Un-sentenced; age; sex |
((8.1-8.3) Age groups count as age disaggregation. (8.1) The definition of homicide may vary by country, but should include murder; honor killings; serious assaults leading to death; death as a result of terrorist activities, etc. as defined in the International Classification of Crime for Statistical Purposes (ICCS). Homicides may also be called “intentional homicides” or included as a line item under “Crimes against person.” (8.2) Crime type can be specific, such as assault, rape, burglary, kidnapping, etc., or be broad categories such as “crimes against person, property, and other.” (8.2) Disaggregation by victim’s characteristics (sex and age) is only relevant to crimes against persons. (8.3) Disaggregation by sentenced/ un-sentenced is a classification of status. Other similar classifications are accepted. |
| (9) Poverty & Income Statistics | (9.1) Poverty rate/ incidence (9.2) Distribution of income by deciles |
(9.1) Optional (9.2) Optional |
(9.1) Figures on at-risk poverty, low income rates, or similar are accepted. Data based on international or national poverty lines are accepted. Data that are not accepted include: Gini coefficients and proportion of population with low disposable income. (9.2) Distribution can be disaggregated by more than 10 categories as well. |
Table 2. ODIN economic data categories, 2017
| Data Category | Representative Indicators | Disaggregation | Notes |
| (10) National Accounts | (10.1) GDP (production approach) or gross value added (10.2) GDP (expenditure approach) |
(10.1) Industrial classification; current prices; constant prices (10.2) Major expenditure categories; current prices; constant prices |
(10.1) Industrial classes include agriculture, mining, manufacturing, etc. (10.2) Major expenditure categories include final consumption expenditure, government expenditure on goods and services, gross fixed capital formation, etc. |
| (11) Labor Statistics | (11.1) Employment rate (11.2) Unemployment rate |
(11.1) Sex; age; industry; occupation type (11.2) Sex; age |
(11.1, 11.2) Active and idle figures are not accepted as employment or unemployment. Figures must be rates. (11.1, 11.2) Youth unemployment is not accepted. (11.1, 11.2). Data disaggregated by age must include all the population age 15 or older. (11.1, 11.2). Age groups count as age disaggregation. |
| (12) Price Indexes | (12.1) Consumer price index (CPI) (12.2) Producers price index (PPI) |
(12.1) Optional (12.2) Optional |
(12.1) If only CPI for the capital city is available, this is accepted, but points are not awarded for admin 1 or 2. (12.2) This indicator may also be called Output Price Index. PPI disaggregated by industry or commodity qualifies only if an aggregate for all industries/commodities is also presented. Industrial PPI is not accepted as it does not include construction and services. |
| (13) Government Finance | (13.1) Actual revenues (13.2) Actual expenditures |
(13.1) Revenue source (13.2) Administrative, economic, or functional classification |
(13.1) Revenue sources may include: taxes, social contributions, grant, other. The division of oil/ non-oil revenues or tax/non-tax revenues is not sufficient unless further disaggregation under those items is also available. (13.2) Administrative classes may include specific ministries, departments, etc. Economic classes are typically presented as: expense, compensation of employees, use of goods and services, consumption of fixed capital, interest, subsidies, grant, social benefits, etc. Functional classes are typically presented as: general public services, defense, public order and safety, economic affairs, environmental protection, health, education, etc. (13.1-13.2) Budgeted revenues and expenditures are accepted for most recent year’s data. |
| (14) Money & Banking | (14.1) Money supply (14.2) Interest rates |
(14.1) M1, M2, or M3 (14.2) Rate type |
(14.1) M1 includes cash (banknotes and coins), plus overnight deposits. M2 (also called quasi money), includes M1, deposits with an agreed maturity of up to two years and deposits redeemable at notice of up to three months. M3 is M2, plus repurchase agreements, money market fund shares/units and debt securities up to two years. (14.2) Rate types can include short or long term government security rates (treasury bills, bond yields), policy orientated interest rates (central bank lending or discount rates), money market, or interbank rates. |
| (15) International Trade | (15.1) Merchandise exports (15.2) Merchandise imports |
(15.1) Major product categories (agricultural products, fuels/ mining, manufactures, etc.). (15.2)Major product categories (agricultural products, fuels/ mining, manufactures, etc.) |
(15.1, 15.2) Product categories may be called commodity groups. Information available on a country’s Balance of Payments is not accepted for any indicator in this category. |
| (16) Balance of Payments/td> | (16.1) Current account (16.2) Capital and Financial account |
(16.1) Goods and services, income, and current transfers (16.2) Direct investment or international investment position |
(16.1, 16.2) Disaggregation of current and capital and financial accounts are based on the IMF’s Balance of Payments guide. |
Table 3. ODIN environmental data categories, 2017
| Data Category | Representative Indicators | Disaggregation | Notes |
| (17) Land Use | (17.1) Data on land use or land cover
(17.2) Data on protected lands |
(17.1) Urban/rural; agricultural use (crop type); environmental zones
(17.2) Optional |
(17.1, 17.2) Data should can presented in physical units (hectares, sq. km, etc.) or percentage of total land.
(17.1) Environmental zones refer to classifications as defined in the UN’s Guidelines of International Classifications for Agricultural Statistics. Environmental zones for land use may include: agriculture, forestry, aquaculture, etc. Environmental zones for land cover may include: crops, trees, mangroves, grassland, etc. (17.2) Refer to IUCN’s information on protected areas for more information. Data may be on “protected areas” if terrestrial areas are separate from other area types. Data on protected areas may include: national parks, wildlife refuge, etc. |
| (18) Resource Use | (18.1) Data on fishery harvests
(18.2) Data on timber harves(18.3) Data on major mining or extractive activitiests or deforestation (18.4) Water supply and/ or consumption |
(18.1) Optional
(18.2) Optional (18.3) Type of mining activity (18.4) Optional |
(18.1-18.4) Data should be presented in physical units or value.
(18.1, 18.2, 18.3) Contribution to GDP for given resources are not accepted. (18.1) Data may also be on fishery landings and catches. (18.2) Land use/cover data with a disaggregation category of forestry is not accepted, nor is data on areas affected by forest fires as this is only one factor contributing to deforestation. (18.3) Type of mining activities may include gas, petroleum, coal, etc. |
| (19) Energy Use
|
(19.1) Consumption of energy | (19.1) End-use sector; industrial sector | (19.1) Data should be in physical units, such as kilowatt hours of electricity or tons of oil equivalent for fossil fuels.
Type of energy must be specified. Energy types include electricity, natural gas, renewables, coal/lignite, oil. End-use sectors may include categories: construction, agriculture, transport, industry, residential, etc. Multiple end-use or industrial sectors must be presented. |
| (20) Pollution
|
(20.1) CO2 orother greenhouse gas (GHG) emissions
(20.2) Emissions of air or water pollutants |
(20.1) Optional
(20.2) Optional |
(20.1, 20.2) Data should be in physical units and GHGs and pollutants should be specifically named.
Production and consumption based accounting of emissions are both accepted. (20.1) Data on ozone depleting substances are accepted for this indicator. (20.2) Common air pollutants include: PM10, PM2.5, O3, NO2, CO, and SO2. |
| (21) Built Environment
|
(21.1) Proportion of people with access to water
(21.2) Proportion of people with access to sanitation (21.3) Housing quality indicators |
(21.1) Optional
(21.2) Optional (21.3) Housing type; number of rooms; construction material, and other indicators on a case by case basis |
(21.1) Data on access to clean/ drinking water or improved water sources is accepted.
(21.2) Data on access to improved sanitation is accepted. (21.3) Housing type is defined broadly and is analyzed on a case by case basis. |
Data Coverage
The data categories are assessed against five elements of coverage shown in Tables 4-6. Each element has a possible score of 1, 0.5, or 0, indicating that the data in a category satisfy the criteria for that element, partially satisfy them, or fail to satisfy them or the data are entirely missing. Thus, a country has a maximum potential score of 210: 105 for data coverage and 105 for data openness.
The scoring scheme is deliberately coarse. A finer scoring grid (say from 1 to 10) would inevitable invite greater subjectivity on the part of assessors and create problems when comparing results produced by different assessors or at different times.
The coverage elements are as follows:
- Coverage Element 1: Indicator Coverage and Disaggregation
- Coverage Element 2: Data Availability for Last 5 Years
- Coverage Element 3: Data Availability for Last 10 Years
- Coverage Element 4: First Administrative Level
- Coverage Element 5: Second Administrative Level
Coverage Element 1: Indicator coverage and disaggregation
The first element requires assessors to locate representative indicators within each data category and determine whether important topical disaggregations are available. Guidelines for scoring each data category are shown in Tables 4a-4u for this element. The representative indicators and disaggregations are also listed in these tables for reference.
Coverage elements 2-5 cannot exceed the score of coverage element 1 for any data category. However, the elements of data openness are scored on the basis of available data, which may receive a maximum score regardless of coverage scores.
If no data are available for a category, all elements are scored 0.
Additionally, though ODIN assesses information available at the first or second administrative level, data must be first and foremost be available at the national level. If national level data for an indicator is not available, no points are rewarded for that indicator.
Below are designated pages for each data category, their representative indicators and topical disaggregations, the scoring criteria for coverage element 1, as well as any methodology changes that may affect scoring coverage element 1 between ODIN 2016 and 2017.
Population & Vital Statistics
Table 4a. Population & Vital Statistics, coverage element 1
| Data Category | Representative Indicators | Disaggregation | >Scoring Guidelines | Notes |
| (1) Population & Vital Statistics | (1.1) Population by 5-year age groups
(1.2) Crude birth rate (1.3) Crude death rate |
(1.1) Sex; marital status
(1.2) Sex; marital status (1.3) Sex |
To receive full point: Must have all indicators disaggregated by sex. To receive half point: Must have at least (1.1) with one disaggregation, or (1.2) and (1.3) with one disaggregation each. |
(1.1) Population data not disaggregated by 5-year age groups can be recorded if available at the admin 1 or 2 level, but national data must also be available in a separate file. Data not disaggregated by 5-year age groups only impacts the administrative coverage scoring element. (1.1) Estimates and projections are accepted. (1.2, 1.3) Birth and death registration figures are not accepted, but number of births/deaths are accepted. Births and deaths presented as sex ratios are not accepted as disaggregated by sex. |
Indicator examples
(1.1) Population by 5-year age group
Example 1: Population at the first day of the quarter by region, sex, age, and marital status, Denmark
(1.2) Crude birth rate/number of births
Example 1: Live births by sex. Albania
(1.3) Crude death rate, number of deaths
Example 1: Deaths by sex, Albania
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Education Facilities
Table 4b. Education Facilities, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (2) Education Facilities
|
(2.1) Number of schools or classrooms
(2.2) Number of teaching staff (2.3) Annual education budget |
(2.1) Age; school stage; school type
(2.2) Age; school stage; school type (2.3) School stage; functional categories (such as teachers, supplies, administration, etc.). |
To receive full point: Must have all three indicators disaggregated as follows: (2.1) and (2.2) by school stage (see notes) and one other disaggregation, as well as (2.3) with one disaggregation.
To receive half point: Must have at least (1.1) with one disaggregation, or (1.2) and (1.3) with one disaggregation each. |
(2.1-2.3) School stage at a minimum refers to primary, plus two other stages (ideally, secondary and post-secondary/tertiary). The two other stages must not be subsets of primary education. See UNESCO guidelines for more information on school stages.
(2.1, 2.2) School type may refer to technical schools, public, private, etc. Age groups count as age disaggregation. (2.3) Education budget must reference corresponding year. GDP contribution to education is not accepted. Ministry of Education expenditures or allocations are accepted. |
Indicator examples
(2.1) Number of schools or classrooms
Example 1: Number of schools by school type and school stage, United States
(2.2) Number of teaching staff
Example 1: Number of teachers by school type, United States
(2.3) Annual education budget
Example 1: Annual education expenditures by school type, United States
Methodology changes
There are no changes to the indicators, however some of the disaggregations have changed.
Disaggregation Change to “School Stages”: In ODIN 2016, assessors were permitted to cite data as disaggregated by school stage if at least two school stages were present. Generally, this meant data was only disaggregated for primary and secondary school. However, in ODIN 2017, three school stages must be included, namely primary, plus two other stages (ideally, secondary and postsecondary/tertiary). If three stages are not listed, the datasets should be removed and rescored.
Disaggregation Change to “Age” and “School Type”: In ODIN 2016, assessors were permitted to cite data as disaggregated by school type if at least one school stage has such disaggregation. However, in ODIN 2017, at least two school stages, disaggregated by school type, must be present.
(2.3) Budget Data: In ODIN 2016, assessors were told to look for budget amounts (or allocations), but expenditure amounts are also accepted this year. Additionally, in the previous ODIN, budget year was counted as a disaggregation; this year it is not.
Scoring First Coverage Element: No changes have been made to the criteria for 0.5 points. However, because of the changes noted above, some datasets used in the previous ODIN no longer qualify.
Population & Vital Statistics
Table 4c. Education Outcomes, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (3) Education Outcomes
|
(3.1) Enrollment rate
(3.2) Completion or graduation rate (3.3) Competency exam results |
(3.1) Sex; school stage; age; school type.
(3.2) Sex; school stage; age; school type. (3.3) Sex |
To receive full point: Must have all indicators disaggregated as follows: (3.1) and (3.2) by sex and one other disaggregation, as well as (3.3) by sex.
To receive half point: Must have (3.1) or (3.2) with two disaggregations or (3.3) disaggregated by sex. Two indicators with only one disaggregation each is not enough. |
(3.1-3.3) Attainment, attendance, and literacy rates are not accepted. Age groups count as age disaggregation.
(3.1-3.2) School stage at a minimum refers to primary, plus two other stages (ideally, secondary and post-secondary/tertiary). The two other stages must not be subsets of primary education. See UNESCO guidelines for more information on school stages. (3.1) Number of students is accepted. (3.3) Matriculation exam results are accepted. |
Indicator examples
(3.1) Enrollment rates
Example 1a: Number of pupils by school stage, school type, Norway
Example 1b: Number of pupils in upper-secondary school by sex and school type, Norway
Example 1c: Number of students in higher education by sex and school type, Norway
(3.2) Completion or graduation rates
(3.3) Competency exam results
Example 1: Average score points on national tests by sex, Norway
Methodology changes
Disaggregation Change to “School Stages”: In ODIN 2016, assessors were permitted to cite data as disaggregated by school stage if at least two school stages were present. Generally, this meant data was only disaggregated for primary and secondary school. However, in ODIN 2017, three school stages must be included, namely primary, plus two other stages (ideally, secondary and postsecondary/tertiary).
Disaggregation Change to “Age,” “Sex,” and “School Type”: In ODIN 2016, assessors were permitted to cite data as disaggregated by school type if at least one school stage has such disaggregation. However, in ODIN 2017, at least two school stages, disaggregated by school type, must be present.
(3.3) Competency Exam Results: In ODIN 2016, the required disaggregation for exam results were identical to indicators 3.1 and 3.2.
However, in ODIN 2017, sex is the only required disaggregation for exam results.
Scoring First Coverage Element: Because of the changes noted above, some datasets used in the previous ODIN no longer qualify.
Education Facilities
Table 4d. Health Facilities, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (4) Health Facilities
|
(4.1) Number of health facilities
(4.2) Number of beds or health budget (4.3) Number of health care staff |
(4.1) Facility type
(4.2) Facility type; department type; functional categories (for budget data only), (4.3) Facility type; department type; staff type |
To receive full point: Must have two indicators with one disaggregation each.
To receive half point: Must have at least one indicator with one disaggregation. |
(4.1-4.3) Facility type generally include clinics, hospitals, pharmacies, dental clinics, public/private hospitals, etc. (at least two must be present). Department type generally include surgery, ophthalmology, internal medicine, pediatric care, dentistry, etc. (at least two must be present). Some countries may have different designations and this is analyzed on a case by case basis.
(4.1-4.3) Figures for the public sector only are accepted. Figures for the private sector only will be reviewed on a case by case basis. (4.2) Ministry of Health expenditures or allocations are accepted. |
Indicator examples
(4.1) Number of health facilities
Example 1: Number of health facilities by facility type, Azerbaijan
(4.2) Number of beds or health budget
Example 1: Number of beds by department type, Azerbaijan
(4.3) Number of health care staff
Example 1: Number of health care staff by staff type, Azerbaijan
Methodology changes
There are no changes to the indicators, however some of the disaggregations and scoring have changed.
Disaggregation change for (4.2) number of beds or health budget: In the previous ODIN, facility type was the only available disaggregation for this indicator. However, in ODIN 2017, “department type” and “functional categories” are included as acceptable disaggregations for number of beds or health budget.
Scoring First Coverage Element: Because of the changes noted above, some datasets that qualify for use in the current ODIN may not have been recorded last year. These new datasets may affect the score.
Health Outcomes
Table 4e. Health Outcomes, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (5) Health Outcomes
|
(5.1) Immunization rate
(5.2) Diseases prevalence/incidence (5.3) Stunting, wasting, or obesity rate |
(5.1) Age; sex
(5.2) Age; sex. (5.3) Age; sex |
To receive full point: Must have all indicators disaggregated by sex.
To receive half point: Must have at least (1.1) with one disaggregation, or (1.2) and (1.3) with one disaggregation each. |
(5.1, 5.3) Rates for children/ infants is accepted as age disaggregation if the age group is defined. In general, age groups count as age disaggregation. Absolute figures are not accepted.
(5.2) Data must show figures for at least two diseases unless data are for HIV/AIDS. (5.2) Mortality rates disaggregated by disease type is not accepted. (5.3) Undernourishment rate is not accepted. Malnourishment rate is accepted. Underweight or overweight rates are accepted. Wasting is also referred to as low weight-to-height. Obesity/overweight is also referred to as high weight-to-height. Stunting is also referred to as low height-for-age. |
Indicator examples
(5.1) Immunization rates
Example 1: Immunization rates by age group and sex, Canada
(5.2) Disease prevalence/incidence
Example 1: Disease prevalence by age group and sex, Canada
(5.3) Stunting, wasting, or obesity rate
Methodology changes
This data category now includes a new indicator: (5.3) Stunting, wasting, and obesity rates. Along with the creation of a new indicator, there were slight changes to existing indicators.
Indicator change to disease prevalence/incidence: (5.2) Disease prevalence/incidence. In ODIN 2016, we only accepted communicable disease prevalence. However, in ODIN 2017, we accept disease prevalence as a whole—meaning both communicable and non-communicable diseases.
Scoring First Coverage Element: Changes have been made to the criteria for 0.5 points. In ODIN 2016, it was acceptable to provide two indicators with no disaggregation, or one indicator with one disaggregation. For ODIN 2017, the criterion to receive 0.5 points was revised to mandate at least one indicator with disaggregation.
Reproductive Health
Table 4f. Reproductive Health, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (6) Reproductive Health
|
(6.1) Maternal mortality ratio/rate
(6.2) Infant mortality rate (6.3) Under-5 mortality rate (6.4) Fertility rate (6.5) Contraceptive prevalence rate (6.6) Adolescent birth rate |
(6.1) Optional
(6.2) Sex (6.3) Sex (6.4) Optional (6.5) Optional (6.6) Optional |
To receive full point: Must have at least five indicators with one disaggregation each (unless disaggregation is optional).
To receive half point: Must have at least two disaggregated indicators (unless disaggregation is optional). One must be a mortality rate. . |
(6.1-6.6) Absolute figures cannot replace rates/ratios (as specified). See WHO definitions for each indicator.
(6.3) Mortality rates disaggregated by age/age groups are accepted so long as rates for under 5 years of age can be extracted. (6.6) Birth rates disaggregated by age/age groups are accepted so long as rates for 15-19 years of age can be extracted. |
Indicator examples
(6.1) Maternal mortality ratio/rate
Example 1: Maternal mortality rate, Jordan
(6.2) Infant mortality rate
Example 1: Infant mortality rate by sex, Lebanon
(6.3) Under-5 mortality rate
Example 1: Under-5 mortality rate by sex, Lebanon
(6.4) Fertility rate
Example 1: Total fertility rate, Jordan
(6.5) Contraceptive prevalence rate
Example 1: Contraceptive prevalence rate, Lebanon
(6.6) Adolescent birth rate
Example 1: Adolescent birth rate, Jordan
Methodology changes
Indicator definitions: The World Health Organization (WHO) definitions for all indicators are more strictly enforced.
Gender Statistics
Table 4g. Gender Statistics, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (7) Gender Statistics
|
(7.1) Data on violence against women
(7.2) Proportion of women in government or management (7.3) Data on child marriages |
(7.1) Violent offense type
(7.2) Optional (7.3) Optional |
To receive full point: Must have all indicators with one disaggregation (unless optional).
To receive half point: Must have at least one indicator with one disaggregation (unless optional). . |
(7.1) Data on people’s opinions/ attitudes about violence against women are not accepted.
(7.1) Data must show at least two violent offense types. Data must either show victimization rates for both men and women, or specify data are only for women (even if the offense is sexual assault, rape, or domestic violence). (7.2) Share of women in employment in the non-agricultural sector is not accepted. Number of women in certain professions may be accepted on a case by case basis. Data must be presented as a rate. Absolute figures are only accepted if the data are disaggregated by two sexes. (7.3) “Child” is defined as under 18 years of age. Countries may define “child” marriages differently and this data are accepted if maximum age is under 18. Data may also be called “early marriages.” Age of first marriage is accepted if figures are not averages, medians, or modes. |
Indicator examples
(7.1) Data on violence against women
Example 1: Victims as a result of domestic violence by violence offense type, Azerbaijan
(7.2) Proportion of women in government or management positions
Example 1: Number and sex distribution of parliamentary elections, Azerbaijan
(7.3) Data on child marriages
Example 1: Marriages by age groups of groom and bride, Azerbaijan
Methodology changes
This data category now includes a new indicator: (7.3) Data on child marriages. Along with the creation of a new indicator, there were slight changes to existing indicators.
Disaggregation change to “violence against women”: For indicator (7.1) Data on violence against women, data must show at least two violent offense types. Previously, disaggregation for this indicator was option. If data on violence against women cited in the previous ODIN does not have this disaggregation, the dataset should be removed and rescored.
Scoring First Coverage Element: No changes have been made to the criteria for 0.5 points. However, because of the changes noted above, some datasets used in the previous ODIN no longer qualify. Thus, scores may need to be adjusted.
Crime & Justice Statistics
Table 4h. Crime & Justice Statistics, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (8) Crime & Justice
Statistics
|
(8.1) Homicide rate/ count
(8.2) Crime rate/count (8.3) Persons in prison/ incarceration rate |
(8.1) Sex of victim; age of victim; sex of perpetrator; age of perpetrator; victim/perpetrator relationship
(8.2) Crime type; age of victim; sex of victim; sex of perpetrator; age of perpetrator; victim/perpetrator relationship (8.3) Sentenced/ Un-sentenced; age; sex |
To receive full point: Must have two indicators with two disaggregations each. Each indicator must be disaggregated by sex (of victim, where relevant). (8.2) must be disaggregated by crime type.
To receive half point: Must have one indicator with two disaggregations or two indicators with one disaggregation each. |
(8.1-8.3) Age groups count as age disaggregation.
(8.1) The definition of homicide may vary by country, but should include murder; honor killings; serious assaults leading to death; death as a result of terrorist activities, etc. as defined in the International Classification of Crime for Statistical Purposes (ICCS). Homicides may also be called “intentional homicides” or included as a line item under “Crimes against person.” (8.2) Crime type can be specific, such as assault, rape, burglary, kidnapping, etc., or be broad categories such as “crimes against person, property, and other.” (8.2) Disaggregation by victim’s characteristics (sex and age) is only relevant to crimes against persons. (8.3) Disaggregation by sentenced/un-sentenced is a classification of status. Other similar classifications are accepted. |
Indicator examples
(8.1) Homicide rate/count
Example 1: Homicide rate/count by age of victim, sex of victim, and victim/perpetrator relationship, Mauritius
(8.2) Crime rate/count
(8.3) Persons in prison/incarceration rate
Example 1: Prison population by age, Mauritius
Methodology changes
There are no methodology changes. This is a new category.
Poverty & Income Statistics
Table 4i. Poverty & Income Statistics, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (9) Poverty & Income
Statistics |
(9.1) Poverty rate/ incidence
(9.2) Distribution of income by deciles |
(9.1) Optional
(9.2) Optional |
To receive full point: Must have both indicators.
To receive half point: Must have at least one indicator. |
(9.1) Figures on at-risk poverty, low income rates, or similar are accepted. Data based on international or national poverty lines are accepted. Data that are not accepted include: Gini coefficients and proportion of population with low disposable income.
(9.2) Distribution can be disaggregated by more than 10 categories as well. |
Indicator examples
(9.1) Poverty rate/incidence
Example 1: Poverty head count ratio (%), Sri Lanka
(9.2) Distribution of income by deciles
Example 1: Household income by deciles, Sri Lanka
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year. However, for indicator 8.1, figures on at-risk-of poverty and low income rates are now accepted for all countries.
National Accounts
Table 4j. National Accounts, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (10) National Accounts
|
(10.1) GDP
(production approach) or gross value added (10.2) GDP (expenditure approach) |
(10.1) Industrial classification; current prices; constant prices
(10.2) Major expenditure categories; current prices; constant prices |
To receive full point: Must have all indicators disaggregated as follows: (10.1) by industrial classification and (10.2) by major expenditure categories. Any data in the most recent 5 years (2012 onward) must be presented on at least quarterly basis to receive a full point.
disaggregation. To receive half point: Must have at least one indicator with one disaggregation. Data can be presented on a monthly, quarterly, or annual basis. |
(10.1) Industrial classes include agriculture, mining, manufacturing, etc.
(10.2) Major expenditure categories include final consumption expenditure, government expenditure on goods and services, gross fixed capital formation, etc. |
Indicator examples
(10.1) GDP (production approach) or gross value added
(10.2) GDP (expenditure approach)
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Labor Statistics
Table 4k. Labor Statistics, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (11) Labor Statistics
|
(11.1) Employment rate
(11.2) Unemployment rate |
(11.1) Sex; age; industry; occupation type
(11.2) Sex; age |
To receive full point: Must have all indicators disaggregated as follows: (11.1) by sex and age and (11.2) by sex and age
To receive half point: Must have at least one indicator with one disaggregation. |
(11.1, 11.2) Active and idle figures are not accepted as employment or unemployment. Figures must be rates.
(11.1, 11.2) Youth unemployment is not accepted. (11.1, 11.2). Data disaggregated by age must include all the population age 15 or older. (11.1, 11.2). Age groups count as age disaggregation. |
Indicator examples
(11.1) Employment rate
Example 1: Labour force status in percentage by employment status, age and sex, Denmark
(11.2) Unemployment rate
Example 1: Labour force status in percentage by employment status, age and sex, Denmark
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Price Indexes
Table 4l. Population & Vital Statistics, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (12) Price Indexes
|
(12.1) Consumer price index (CPI)
(12.2) Producers price index (PPI) |
(12.1) Optional
(12.2) Optional |
To receive full point: Must have all indicators. Any data in the most recent 5 years (2012 onward) must be presented on a quarterly or monthly basis to receive a full point.
To receive half point: Must have at least one indicator. Data can be presented on a annual, quarterly, or monthly frequency. |
(12.1) If only CPI for the capital city is available, this is accepted, but points are not awarded for admin 1 or 2.
(12.2) This indicator may also be called Output Price Index. PPI disaggregated by industry or commodity qualifies only if an aggregate for all industries/ commodities is also presented. Industrial PPI is not accepted as it does not include construction and services. |
Indicator examples
(12.1) Consumer price index (CPI)
Example 1: Monthly consumer price index, Tanzania
(12.2) Producer price index (CPI)
Example 1: Monthly producer price index, Tanzania
Methodology changes
(12.1) CPI Change: ODIN 2017 now accepts consumer price indexes for the capital city as national level data, but points are not awarded for the first or second administrative level in this case.
Government Finance
Table 4m. Government Finance, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (13) Government
Finance
|
(13.1) Actual revenues
(13.2) Actual expenditures |
(13.1) Revenue source
(13.2) Administrative classification; economic classification; functional classification |
To receive full point: Must have all indicators with one disaggregation each.
To receive half point: Must have at least one indicator with one disaggregation. |
(13.1) Revenue sources may include: taxes, social contributions, grant, other. The division of oil/ non-oil revenues or tax/non-tax revenues is not sufficient unless further disaggregation under those items is also available.
(13.2) Administrative classes may include specific ministries, departments, etc. Economic classes are typically presented as: expense, compensation of employees, use of goods and services, consumption of fixed capital, interest, subsidies, grant, social benefits, etc. Functional classes are typically presented as: general public services, defense, public order and safety, economic affairs, environmental protection, health, education, etc. (13.1-13.2) Budgeted revenues and expenditures are accepted for most recent year’s data. |
Indicator examples
(13.1) Actual revenues
Example 1: Government actual revenues by revenue source (table 14-3), Afghanistan
(13.2) Actual expenditures
Example 1: Budget actual expenditure by administrative classification (table 14-2), Afghanistan
Methodology changes
There are no changes to the indicators, however some of the disaggregations and scoring have changed.
Disaggregation for (13.1) Actual revenues: Revenue source disaggregation that states only tax and non-tax revenues, oil and nonoil revenues, or any other binary division, is not sufficient unless further disaggregation is available under those categories.
Money & Banking
Table 4n. Money & Banking, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (14) Money & Banking
|
(14.1) Money supply
(14.2) Interest rates |
(14.1) M1; M2; M3
(14.2) Rate type |
To receive full point: Must have all indicators disaggregated. Must have at least three rates.
To receive half point: Must have one disaggregated indicator. If indicator is (14.2), only one rate is needed. |
(14.1) M1 includes cash
(banknotes and coins), plus overnight deposits. M2 (also called quasi money), includes M1, deposits with an agreed maturity of up to two years and deposits redeemable at notice of up to three months. M3 is M2, plus repurchase agreements, money market fund shares/units and debt securities up to two years. (14.2) Rate types can include short or long term government security rates (treasury bills, bond yields), policy orientated interest rates (central bank lending or discount rates), money market, or interbank rates. |
Indicator examples
(14.1) Money supply
Example 1: Money supply (M1, M2 and M3, table 13.1.1, pages 155-156), Rwanda
(14.2) Interest rates
Example 1: Interest rates structure (table 13.1.8, page 168), Rwanda
Methodology changes
Money supply disaggregation: Both M1 and M2 were required in ODIN 2016 to be recorded at all. However, in ODIN 2017, either M1, M2, or M3 is accepted.
New indicator ( 14.2) Interest rates: This indicator replaces the previous indicator, Total Credit, which was included in ODIN 2015 and 2016.
.
International Trade
Table 4o. International Trade, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (15) International Trade
|
(15.1) Merchandise exports
(15.2) Merchandise imports |
(15.1) Major product categories (agricultural products, fuels/mining, manufactures, etc.).
(15.2)Major product categories (agricultural products, fuels/mining, manufactures, etc.). |
To receive full point: Must have (15.1) and (15.2) imports by major product categories. Any data in the most recent 5 years (2012 onward) must be presented on a quarterly or monthly basis to receive a full point.
To receive half point: Must have one indicator with one disaggregation. Data can be presented on an annual, quarterly, or monthly basis. |
(15.1, 15.2) Product categories may be called commodity groups.
Information available on a country’s Balance of Payments is not accepted for any indicator in this category. |
Indicator examples
(15.1) Merchandise exports
Example 1: Quarterly merchandise exports by major product categories (table 2), Vanuatu
(15.2) Merchandise imports
Example 1: Quarterly merchandise exports by major product categories (table 4), Vanuatu
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Balance of Payments
Table 4p. Balance of Payments, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (16) Balance of
Payments
|
(16.1) Current account
(16.2) Capital and Financial account |
(16.1) Goods and services, income, and current transfers
(16.2) Direct investment or international investment position |
To receive full point: Must have (16.1) and (16.2) with one disaggregation each.
To receive half point: Must have (16.1) or (16.2) with one disaggregation. |
(16.1, 16.2) Disaggregation of current and capital and financial accounts are based on the IMF’s Balance of Payments guide. |
Indicator examples
(16.1) Current account
Example 1: Balance of payments, current account (goods, services, income, current transfers), India
(16.2) Capital and financial account
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Land Use
Table 4q. Land Use, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (17) Land Use
|
(17.1) Data on land use or land cover
(17.2) Data on protected lands |
(17.1) Urban/rural; agricultural use (crop type); environmental zones
(17.2) Optional |
To receive full point: Must have all indicators disaggregated by sex.
To receive half point: Must have at least (1.1) with one disaggregation, or (1.2) and (1.3) with one disaggregation each. |
(17.1, 17.2) Data should can presented in physical units (hectares, sq. km, etc.) or percentage of total land.
(17.1) Environmental zones refer to classifications as defined in the UN’s Guidelines of International Classifications for Agricultural Statistics. Environmental zones for land use may include: agriculture, forestry, aquaculture, etc. Environmental zones for land cover may include: crops, trees, mangroves, grassland, etc. (17.2) Refer to IUCN’s information on protected areas for more information. Data may be on “protected areas” if terrestrial areas are separate from other area types. Data on protected areas may include: national parks, wildlife refuge, etc. |
Indicator examples
(17.1) Data on land use or land cover
Example 1: Land cover by environmental zones, United Kingdom
Example 2: Land use by agricultural use, United Kingdom
(17.2) Data on protected lands
Example 1: Data on protected areas, United Kingdom
Methodology changes
New indicator: This data category now includes the indicator, (17.2) Data on protected lands.
Scoring First Coverage Element: Changes have been made to the criteria for a full point in order to include data on the new indicator.
Pollution
Table 4r. Pollution, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (18) Resource Use | (18.1) Data on fishery harvests
(18.2) Data on timber harvests or deforestation (18.3) Data on major mining or extractive activities (18.4) Water supply and/ or consumption |
(18.1) Optional
(18.2) Optional (18.3) Type of mining activity (18.4) Optional |
To receive full point: Must have three indicators, disaggregated (unless optional).
To receive half point: Must have at least two indicators, disaggregated (unless optional). |
(18.1-18.4) Data should be presented in physical units or value.
(18.1, 18.2, 18.3) Contribution to GDP for given resources are not accepted. (18.1) Data may also be on fishery landings and catches. (18.2) Land use/cover data with a disaggregation category of forestry is not accepted, nor is data on areas affected by forest fires as this is only one factor contributing to deforestation. (18.3) Type of mining activities may include gas, petroleum, coal, etc. |
Indicator examples
(18.1) Data on fishery harvests
Example 1: Fish production (table 1.21, page 70), Pakistan
(18.2) Data on timber harvests/deforestation
Example 1: Output on major forest products (table 1.20, page 69), Pakistan
(18.3) Data on major mining or extractive activities
Example 1: Mineral production of Pakistan (table 7.3, pages 168-173), Pakistan
(18.4) Water supply and/or consumption
Example 1: Overall water availability (table 1.14, page 65), Pakistan
Methodology changes
Change to Indicator (18.2): This indicator has been updated to include timber harvests as well as deforestation.
Scoring First Coverage Element: No changes have been made to the criteria for 0 .5 points. However, because of the changes noted above, some datasets not used in the previous ODIN now qualify. Thus, scores may need to be adjusted.
Energy Use
Table 4s. Energy Use, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (19) Energy Use | (19.1) Consumption of energy | (19.1) End-use sector; industrial sector | To receive full point: Must have (19.1) on three energy types with one disaggregation each.
To receive half point: Must have (19.1) on two energy type with one disaggregation each. |
(19.1) Data should be in physical units, such as kilowatt hours of electricity or tons of oil equivalent for fossil fuels.
Type of energy must be specified.Energy types include electricity, natural gas, renewables, coal/lignite, oil. End-use sectors may include categories: construction, agriculture, transport, industry, residential, etc. Multiple end-use or industrial sectors must be presented. |
Indicator examples
(19.1) Consumption of energy (must specify type)
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Pollution
Table 4t. Pollution, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (20) Pollution | (20.1) CO2 or other greenhouse gas (GHG) emissions
(20.2) Emissions of air or water pollutants |
(20.1) Optional
(20.2) Optional |
To receive full point: Must have all indicators. CO2 must be specified.
To receive half point: Must have at least one indicator. |
(20.1, 20.2) Data should be in physical units and GHGs and pollutants should be specifically named.
Production and consumption based accounting of emissions are both accepted. (20.1) Data on ozone depleting substances are accepted for this indicator. (20.2) Common air pollutants include: PM10, PM2.5, O3, NO2, CO, and SO2. |
Indicator examples
(20.1) CO2 or other GHG
Example 1: Accounts of emissions to the atmosphere, Spain
(20.2) Emissions of air or other water pollutants
Example 1: Accounts of emissions to the atmosphere, Spain
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Built Environment
Table 4u. Built Environment, coverage element 1
| Data Category | Representative Indicators | Disaggregation | Scoring Guidelines | Notes |
| (21) Built Environment
|
(21.1) Proportion of people with access to water
(21.2) Proportion of people with access to sanitation (21.3) Housing quality indicators |
(21.1) Optional
(21.2) Optional (21.3) Housing type; number of rooms; construction material, and other indicators on a case by case basis |
To receive full point: Must have
(21.1) and (21.2); as well as (21.3) with one disaggregation. To receive half point: ust have either (21.1) and (21.2), or (21.3) with at least one disaggregation. |
(21.1) Data on access to clean/ drinking water or improved water sources is accepted.
(21.2) Data on access to improved sanitation is accepted. (21.3) Housing type is defined broadly and is analyzed on a case by case basis.
|
Indicator examples
(21.1) Proportion of people with access to water
Example 1: Percent distribution of households by source of drinking water (table 2.1, page 12), Sierra Leone
(21.2) Proportion of people with access to sanitation
Example 1: Percent distribution of households by type of toilet facilities (table 2.2, page 14), Sierra Leone
(21.3) Housing quality indicators
Methodology changes
There are no changes in the indicators or their disaggregations between this year and last year.
Coverage Element 2: Data availability for the last five years
Coverage element 2 measures whether data are available in a data category over the last five years. Scores for this element cannot be greater than the score for coverage element 1. Scores are given by data category, not indicator.
For the categories National Accounts, Price Indexes, and International Trade, if data are presented on a quarterly basis, 3 out of 4 quarters for a particular year must be available to award credit. If data are presented on a monthly basis, 7 out of 12 months for a particular year must be present to award credit.
Countries may use calendar or fiscal years to present data. Common fiscal years include 1 July – 30 June, 1 April-31 March. Less common fiscal years are used in Afghanistan, Nepal, Iran, and the US.
Table 5a. Scoring criteria, coverage element 2
| Scoring Options | Notes |
| 1 point if all published data are available for 3 of the last 5 years. | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data. However, a full point cannot be awarded unless a full point was awarded for coverage element 1.
The three years must occur within 2012-2016 (or 2011/2012-2015/2016). If the indicator shows data for a non-calendar year (i.e. 2012/2013), this should be counted as one year. Overlapping years (such as 2012 and 2012/13 for a single indicator) should not be double counted. |
| .5 points if some published data are available for 1-2 of the last 5 years. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country.
The 1-2 years must occur within 2012-2016 (or 2011/2012-2015/2016). If the indicator shows data for a non-calendar year (i.e. 2012/2013), this should be counted as one year. |
| 0 points if no published data are unavailable for last 5 years. | A 0 is only given if no data in the last five years is available for any indicator within a data category, or if only monthly/quarterly data exists, but not for a majority of any months/quarters in the last five years. |
Coverage Element 3: Data availability for the last ten years
Coverage element 2 measures whether data are available in a data category over the last ten years. Scores for this element cannot be greater than the score for coverage element 1. Scores are given by data category, not indicator.
For the categories National Accounts, Price Indexes, and International Trade, if data are presented on a quarterly basis, 3 out of 4 quarters for a particular year must be available to award credit. If data are presented on a monthly basis, 7 out of 12 months for a particular year must be present to award credit.
Countries may use calendar or fiscal years to present data. Common fiscal years include 1 July – 30 June, 1 April-31 March. Less common fiscal years are used in Afghanistan, Nepal, Iran, and the US.
Table 5b. Scoring criteria, coverage element 3
| Scoring Options | Notes |
| 1 point if all published data are available for 6 of the last 10 years. | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data. However, a full point cannot be awarded unless a full point was awarded for coverage element 1.
The six years must occur within 2007-2016 (or 2006/2007-2015/2016). If the indicator shows data for a non-calendar year (i.e. 2012/2013), this should be counted as one year. Overlapping years (2012 and 2012/13) should not be double counted. |
| .5 points if some published data are available for 3-5 of the last 5 years. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country.
The 3-5 years must occur within 2007-2016 (or 2006/2007-2015/2016). If the indicator shows data for a non-calendar year (i.e. 2012/2013), this should be counted as one year. Overlapping years (2012 and 2012/13) should not be double counted. |
| 0 points if all published data are unavailable for 2 or fewer of the last 10 years. | A 0 is only given if no data in the last ten years is available for any indicator within a data category, or if only monthly/quarterly data exists, but not for a majority of any months/quarters in the last five years. |
Coverage Element 4: First administrative level
Coverage element 4 measures whether data are available at the first administrative level. Scores for this element cannot be greater than the score for coverage element 1. Scores are given by data category, not indicator.
Additionally, data disaggregated at the first administrative level is only scored if national level data also exists for that indicator.
Often, national data are included in the same file as subnational data as an aggregate figure.
To identify the first administrative levels, ODIN largely draws on the ISO 3166-2 standard. In many countries, first administrative levels refer to governorates, regions, or provinces. However, this is not always the case. For instance, in China, four municipalities are included in the first administrative level (Beijing, Chongqing, Shanghai, and Tianjin).
For indicator 1.1, population data at the first administrative level are accepted even they are not disaggregated by 5-year age groups.
Money & Banking, International Trade, and Balance of Payments are not scored for this element.
Table 5c. Scoring criteria, coverage element 4
| Scoring Options | Notes |
| 1 point if all published data in a data category are available at first administrative level. | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data. However, a full point cannot be awarded unless a full point was awarded for coverage element 1. |
| .5 points if some published data in a data category are available at first administrative level. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country. |
| 0 points if no data are available at this level. | A 0 is only given if no data in a data category is available at this level. |
Coverage Element 5: Second administrative level
Coverage element 5 measures whether data are available at the second administrative level. Scores for this element cannot be greater than the score for coverage element 1. Scores are given by data category, not indicator.
Additionally, data disaggregated at the second administrative level are only scored if national level data also exists for that indicator. Often, national data are included in the same file as subnational data as aggregate figures.
No official list exists for the second administrative level classifications. If geographical disaggregation exists that does not qualify as first administrative level, assume that the data are disaggregated to the second administrative level as long as the classification appears to be a subset of the first administrative level. It is possible to score points on this element even if no data are disaggregated at the first administrative level. Incomplete second administrative level data are not scored. For instance, if municipalities are a country’s second administrative level and the country publishes data on 20 municipalities for an indicator (but the country has a total of 42 municipalities), then no credit is given.
National Accounts, Government Finance, Money & Banking, International Trade, and Balance of Payments are not scored for this element.
For indicator 1.1, population data at the second administrative level is accepted even if it is not disaggregated by 5-year age groups.
Money & Banking, International Trade, and Balance of Payments are not scored for this element.
Table 5d. Scoring criteria, coverage element 5
| Scoring Options | Notes |
| 1 point if all data in a data category are available at second administrative level. | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data. However, a full point cannot be awarded unless a full point was awarded for coverage element 1. |
| .5 points if some published data are available at second administrative level. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country. |
| 0 points if no data are available at this level. | A 0 is only given if no data in a data category is available at this level. |
Methodology changes
Experience from the previous two ODIN assessments and feedback from countries suggested that very small countries were unlikely to designate administrative areas below the first administrative level. Therefore ODIN 2017 does not score the second administrative element for countries with a surface area of 1,500 sq. km. or less. Fourteen countries in ODIN 2017 qualified for this exclusion: Andorra, Anguilla, Hong Kong, Kiribati, Macao, Maldives, Malta, Marshall Islands, Micronesia, Sao Tome and Principe, Seychelles, Singapore, St. Lucia, St. Vincent and the Grenadines.
Data Openness
The data categories are assessed against five elements of openness shown in Tables 6a-6e. Each element has a possible score of 1, 0.5, or 0, indicating that the data in a category satisfy the criteria for that element, partially satisfy them, or fail to satisfy them or the data are entirely missing. Thus, a country has a maximum potential score of 210: 105 for data coverage and 105 for data openness.
The scoring scheme is deliberately coarse. A finer scoring grid (say from 1 to 10) would inevitable invite greater subjectivity on the part of assessors and create problems when comparing results produced by different assessors or at different times.
The openness elements are as follows:
- Openness Element 1: Machine Readability
- Openness Element 2: Non-proprietary Format
- Openness Element 3: Download Options
- Openness Element 4: Metadata Availability
- Openness Element 5: Free/Unrestricted Terms of Use
Openness elements 1 through 5 assess the openness of data in a category using criteria derived from the Open Definition. Scores for coverage and openness were considered independently. If only one indicator for a certain category was published but that indicator was published in a fully open manner, it was given full points for openness. Scores for openness could, therefore, exceed the scores for coverage in the same category, but in practice this rarely happens. Data categories that received a coverage score of 0, were also scored 0 for all elements of openness.
Openness elements 1 and 2 assess whether data are downloadable in machine readable, non-proprietary formats. Open data should be available to anyone in convenient and readily modifiable form. Openness element 3 asks whether users can download data in bulk, or whether they can download a subset of the data via an API or other user-select option. The alternative is often that data are only available in predetermined tables. The availability of metadata (openness element 4) is of importance in providing users with information on how the data were collected and compiled. Clear licensing terms/Terms of Use (openness element 5) state what users may do with the data and permit for reuse of data with some restrictions; fully open data may be used and reused without restriction other than providing attribution to the original source.
Starting in the next section, tables for each openness element show the criteria for each possible score.
Openness Element 1: Machine readability
Openness element 1 measures whether data are available in a machine readable format such as XLS, XLSX, CSV, and JSON. Machine-readable file formats allow users to easily process data using a computer. When data are made available in formats that are not machine readable, users cannot easily access and modify the data, which severely restricts the scope of the data’s use. In many cases PDF versions of datasets within reports can be useful to users, as the text in conjunction with the tables gives context and explanation to the figures which helps less technical users understand the data. Because of this, ODIN assessments do not penalize countries for making datasets available in PDF or other non-machine readable formats, unless these formats are the only option for exporting data.
Scores are not penalized for having identical datasets in both machine readable and non-readable formats.
Compression formats do not affect machine readability scores, only non-proprietary scores (see Table 6b).
Scores are given by data category, not indicator.
Table 6a. Scoring criteria, openness element 1
| Scoring Options | Notes |
| 1 point if all published data are available in a machine readable format (such as XLS, XLSX, CSV, Stata, SAS, SPSS, JSON and so forth). | “All publsihed data” is defined as every indicator, disaggregation, and year published by a country, not all possible data.
CDF, RDF, XML, and TXT files are also classified as machine readable. PDFs, picture files, and HTML formats are not machine readable. |
| .5 points if some published data are available in machine-readable format. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country.
If published data are available in both a machine readable format and in a non-machine readable format, a full point should be awarded for machine readability. If, however, the data in the machine readable format is less complete, a half point should be awarded. |
| 0 points if all published data are not available in machine-readable format. | A 0 is only given if no data are available in a machine readable format. |
Openness Element 2: Non-proprietary format
Coverage element 2 measures whether data are in non-proprietary formats such as XLSX, DOCX, CSV, XML, HTML, and JSON. Non-proprietary formats are important because they allow users to access data without requiring the use of a costly, proprietary software that may prevent some users from accessing the data. Because the PDF format is non-proprietary, countries publishing PDF files could receive full marks for this element, even when they are marked down for not providing machine-readable files.
The availability of XLSX files has become more common, likely because it is the default format used by newer versions of Microsoft
Excel (2007 and onward). The XLSX format is in the public domain. However, many countries still publish data in XLS files. Although XLS files can be opened with some open source software, such as OpenOffice and LibreOffice, the format is based on BIFF (Binary Interchange File Format), which has its encoding published, but its use is restricted by various licenses.
Scores are given by data category, not indicator.
Table 6b. Scoring criteria, openness element 2
| Scoring Options | Notes |
| 1 point if all published data are available in non-proprietary format (such as XLSX, DOCX, CSV, XML, HTML, and JSON). | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data.
TXT files are also treated as non-proprietary. XLS, Stata, SAS, SPSS, DOC and PPT files are proprietary. |
| .5 points if some published data are available in non-proprietary format. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country.
If published data are available in both a proprietary format and in a non-proprietary format, a full point should be awarded. If, however, the data in the non-proprietary format is less complete, a half point should be awarded. |
| 0 points if no data are available in a non-proprietary format. | A 0 is only given if no data in a data category is available in this format or if all data files are compressed in RAR format. ZIP compression is non-proprietary and does not affect a country’s score. |
Openness Element 3: Download options
Openness element 3 measures whether data are available with three different download options: bulk download, API, and userselect options.
A bulk download is defined at the indicator level as: The ability to download all data recorded in ODIN for a particular indicator (all years, disaggregations, and subnational data) in one file, or multiple files that can be downloaded simultaneously. Bulk downloads are a key component of the Open Definition, which requires data to be “provided as a whole . . . and downloadable via the internet.”
User-selectable download options are defined as: Users must be able to select an indicator and at least one other dimension to create a download or table. These dimensions could include time periods, geographic disaggregations, or other recommended disaggregations. An option to choose the file export format is not enough.
API stands for Application Programming Interface. Ideally, APIs should be clearly displayed. ODIN assumes APIs are available for the NSOs entire data collection, unless clearly stated. ODIN assessors do not register for use or test API functionality. For more information on APIs, see this guide.
Scores are given by data category, not indicator.
Table 6c. Scoring criteria, openness element 3
| Scoring Options | Notes |
| 1 point if all published data has a bulk download option exists and an API or user-selectable download option. | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data.
CDF, RDF, XML, and TXT files are also classified as machine readable. PDFs, picture files, and HTML formats are not machine readable. |
| .5 points if some published data are available in machine-readable format. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country.
If published data are available in both a machine readable format and in a non-machine readable format, a full point should be awarded for machine readability. If, however, the data in the machine readable format is less complete, a half point should be awarded. |
| 0 points if all published data are not available in machine-readable format. | A 0 is only given if no data are available in a machine readable format. |
Methodology changes
The definition of bulk download was revised in ODIN 2017, which is due to the Open Definition’s rewording which softened the language around bulk downloads, stating instead that data “must be provided as a whole…and should be downloadable via the Internet”. Though ODIN still refers to this option as bulk download, we define bulk download at the indicator level in ODIN 2017.
In ODIN 2016, bulk downloads needed to include multiple indicators.
Additionally, APIs are now interchangeable with user-select options. In ODIN 2016, it was interchangeable with bulk download.
This change was made to give bulk downloads increased emphasis, especially given their prominence in the Open Definition. APIs are not a substitute for bulk download and are better suited for certain users and scenarios. APIs and user-select download options both allow users to customize data exports to access a small part of the total available data, which is why these features are interchangeable for scoring purposes.
Openness Element 4: Metadata availability
Openness element 4 measures whether metadata are made available. Scores are given by data category, not indicator.
Metadata are defined at the indicator level as information about how the data are defined/calculated and collected. ODIN classifies metadata into three categories: (1) Not Available, (2) Incomplete, and (3) Complete.
The following must be available to classify metadata as complete:
- Definition of the indicator, or definition of key terms used in the indicator description (as applicable), or how the indicator was calculated.
- Publication (date of upload), compilation date (date on front of report is not sufficient), or date dataset was last updated.
- Name of data source (what agency collected the data).
If the metadata only have one or two of the above elements, they are scored as incomplete.
Table 6d. Scoring criteria, openness element 4
| Scoring Options | Notes |
| 1 point if all published data have complete metadata. | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data.
Metadata may be included in the same file as the data or it can be included elsewhere on the website (this may happen if the data are in a regularly published report, survey, or census). |
| .5 points if some published data have complete or incomplete metadata, or all published data have incomplete metadata. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country. |
| 0 points if all published data have no metadata. |
Metadata examples
Example 1: Complete metadata for indicator (3.1) enrollment rate, Armenia
Methodology changes
The classifications of metadata have changed in ODIN 2017. Previously, metadata were classified as “specific” or “general.” Specific metadata was defined as that which “provides specific details about the definition of the indicator or the method of data collection and compilation for that indicator.” General metadata was defined as that which “provides information about a large survey or group of data of which the indicator is part”.
These definitions do not directly translate to the current definitions of complete or incomplete. However, if metadata was classified as “specific” last year, it would score this year, at minimum, as incomplete. Metadata classified as “general” last year may not meet the criteria for incomplete.
Openness Element 5: Free/unrestrictive terms of use
Openness element 5 measures whether data are available with an open terms of use. Generally, terms of use (TOU) will apply to an entire website or data portal (unless otherwise specified). In these cases, all data found on the same website and/or portal will receive the same score. If a portal is located on the same domain as the NSO website, the terms of use on the NSO site will apply. If the data are located on a portal or website on a different domain, another terms of use will need to be present. For a policy/ license to be accepted as a terms of use, it muat clearly refer to the data found on the website. Terms of use that refer to nondata content (such as pictures, logos, etc.) of the website are not considered. A copyright symbol at the bottom of the page is not sufficient. A sentence indicating a recommended citation format is not sufficient.
Terms of use are classified the following ways: (1) Not Available, (2) Restrictive, (3) Semi-Restrictive, and (4) Open.
If the TOU contains one or more restrictive clauses, it receives 0 points and is classified as “restrictive.” Restrictive clauses include:
Noncommercial provisions: These clauses forbid use for commercial purposes. As explained by the Open Data Institute, “A noncommercial provision is problematic primarily because of a lack of clarity around what constitutes ‘commercial’ usage.” In some cases, a limitation to noncommercial use can be interpreted as allowing personal use only, as happened in a 2014 German court case.
Requires prior permission: These clauses require users to seek permission, in any format, before using data.
Registration requirements: These clauses require users to register or provide personal information about themselves in order to accept the terms of use conditions.
If the TOU contains one “semi-restrictive” clauses, it receives 0.5 points. If the terms of use includes two or more of these clauses, it is classified as “Restrictive” and receives 0 points. Semi-restrictive clauses include:
Onerous attribution: These clauses require an onerous attribution, such as requiring users to publish large amounts of metadata or other technical notes when using the data.
Prohibits misleading use: Any language that prohibits the use of data in a “misleading” or similar manner is classified as semi-restrictive because it is unclear and may be enforced arbitrarily to discriminate against users.
Vague language: Any TOU that doesn’t meet the criteria of fully open, but also does not include any restrictive or semirestrictive clauses, is classified as “semi-restrictive” and receive 0.5 points.
A TOU is considered open and receives a full point if it states that data are licensed for free use and reuse for commercial and noncommercial use with, at most, an obligation to attribute data to original source. Some important notes:
- CC BY (any version), CC BY-SA (any version), and CC0 licenses receive a full point.
- If data are identified as in the “public domain” without any semi-restrictive or restrictive clauses, a full point is awarded.
- If data can be used “freely,” that is interpreted as “without restrictions” and would receive a full point if there are no semi-restrictive or restrictive clauses. This is different than free of charge.
- Clauses that require the user to state that the data were updated or modified are generally not seen as restrictive and do not affect the TOU score, but this is determined on a case by case basis.
Table 6e. Scoring criteria, openness element 5
| Scoring Options | Notes |
| 1 point if all published data have a terms of use classified as open. | “All published data” is defined as every indicator, disaggregation, and year published by a country, not all possible data.
See above for the criteria for an open terms of use. |
| .5 points if some published data have a terms of use classified as open, or if some published data has a terms of use classified as semi-restrictive or if all published data has a terms of use classified as semi-restrictive. | “Some published data” is defined as any subset of indicators, disaggregations, or years published by a country.
See above for the criteria for semi-restrictive terms of use. |
| 0 points If no terms of use are found or if all published data have a terms of use classified as restrictive. | See above for the criteria for restrictive terms of use. |
Terms of use examples
Example 1: Open terms of use, Finland
Example 2: Semi-restrictive terms of use (includes onerous attribution), Philippines
Example 3: Restrictive terms of use (includes noncommercial provision), Hong Kong
Methodology changes
The classification of TOUs has changed slightly in ODIN 2017. Noncommercial provisions are now considered restrictive rather than semi-restrictive. The terms “public domain” and “freely” are now interpreted as open, so long they do not refer solely to the lack of required payment.
Aggregate Scores
ODIN scores are summarized along both dimensions of the ODIN assessment: by categories and by elements. In addition, subscores are computed for the combined categories of social statistics, economic statistics, and environmental statistics and for the combined elements of coverage and openness. The overall score aggregates all scores across both dimensions. For convenience, all aggregate scores are represented as percentages of the maximum possible score.
Standardized scores
The aggregate scores shown in ODIN tables and charts have been standardized. Scores are standardized by dividing by the maximum score achievable and multiplying by 100. For most subscores, the maximum score is the product of the number of data categories and the number of elements included. However, some of the elements of geographic disaggregation have been excluded from the economic categories.
Specifically, it is assumed that the national accounts and government finance statistics will not be available at the second administrative level and that money and banking, international trade, and balance of payments statistics will not be available at the first or second administrative levels. Additionally, countries with a surface area of 1,500 sq. km. or less are not scored at the second administrative level for any category. Therefore, for most countries, the maximum, unweighted score for five data coverage elements across all seven economic categories is 27 not 35 and the maximum achievable score over all data categories and elements is 192 not 200. Standardized scores involving any of these categories are reweighted to give them full weight.
Because of this discrepancy, subscores over data categories or across elements involving economic statistics will not “add up” consistently, but the treatment of each subscore is internally consistent.
Weighting
Because the three principal topical groupings (social, economic, and environmental) contain different numbers of data categories, aggregates computed over these categories would be implicitly weighted by the number of categories in each grouping. To neutralize this effect, the data categories are reweighted so that each group has equal weight in aggregates computed over all categories. The reweighting does not affect aggregates computed within each grouping. All elements have equal weights in all aggregates.
ODIN Online has an option for downloading both the raw and weighted scores for further analysis.
The ODIN website includes scores for every round of ODIN and provides many ways for users to interact and manipulate scores. The following is a brief description of each page of the ODIN website and its functionality.
ODIN home
- The Home page displays a map of the world, showing in color the countries included in the 2017 ODIN assessment. Colors indicate the range of their overall ODIN score by quintiles. Countries in gray were not include in the 2017 ODIN assessments. The view can be toggled between years by selecting from the drop-down menu above the map.
- Clicking on a country brings up an information box with the country’s aggregate scores and rank. Clicking on the country name takes users to the Country Profile page.
Rankings
- The Rankings page displays the overall score and aggregate subscores for data coverage and openness for all countries. The display can be sorted by country name, region, or scores by clicking on the table headers.
- The view defaults to ODIN 2017 data but can be changed by selecting a year from the dropdown menu.
- To download the dataset as seen on screen, click the download button in the top right corner and select the export file format.
Country profile
- The Country Profile page provides the most detailed information on a country’s ODIN scores. Summary scores are shown for the 21 (20 in 2015 and 2016) data categories aggregated over the elements of coverage and openness and for the 10 elements of coverage and openness aggregated over the social, economic, and data categories. Graphs provide regional and global comparisons.
- The data locator tab on the Country Profile page provides a description of the sources used for ODIN assessments.
ODIN Online
- A PDF report of a country’s findings can be downloaded from this page by clicking “Download country report.” This report shows a summary of results, provides a brief narrative with a list of recommendations, and provides other useful information to better understand the country’s context.
- The Country Profile page also provides a twoyear comparison when any pair of 2015, 2016, or 2017 are selected.
Regional comparison
- The Regional Comparison page gives users the option to compare country scores by geographic regions or income groups within geographic regions. Users choose options from the top drop-down menu to display data.
- To download the dataset as seen on screen, click the download button in the top right corner and select the export file format.
- ODIN countries are grouped by continents and regions defined by the United Nations Statistics Division’s M49 listing of macro geographical regions. Country codes are three-character ISO codes. ODIN also includes the Republic of Kosovo and Taiwan with the respective ISO codes of XKX and TWN, which are not included in the UN list. Three-character regional codes were created for use in ODIN and are not part of the M49 listing.
- ODIN countries have also been classified by the World Bank’s income groups.
Country comparison
- The Country Comparison page allows users to tabulate aggregate scores for one or more countries. The overall score and five scores aggregated over categories and elements are displayed.
- First select the year, followed by regions or subregions from which to select countries; then select some or all of the countries.
- To download the dataset as seen on screen, click the download button in the top right corner and select the export file format.
Data download
- The Data Download page provides access to the full ODIN dataset at the item level. Two types of scores can be selected: Raw or weighted. Raw scores are the original scores recorded by the assessors. Choose weighted if you want to use standard ODIN weights or custom weights, as you would on any other page. Standard ODIN weighted scores have been multiplied by a weighting matrix that gives greater weight to the environmental and economic data categories to compensate for the fewer number of categories in the overall score. Standardized scores are derived from the weighted scores by dividing by the sum of their weights and multiplying by 100. The item level standard weighted scores differ from the raw scores by a factor of 100. Weighting only affects aggregate scores.
- First select regions or sub-regions and then select countries. The entire database can be selected by choosing all years, regions, and countries.
- The aggregate subscores for social, economic, and environmental categories and subscores for coverage and openness elements can be selected for downloading.
Reports
- The Reports page gives access to the ODIN Annual Report, ODIN Methodology Report, and one-page country and regional briefs in PDF format.
Custom weighting
- Users can apply custom weighting to each page of ODIN. At the top of each page users will see what kind of weighting scheme is being used to show results. ODIN standard weights will display by default.
- To change to custom weighting, click the blue button that says “change weights.”
- Once you click “change weights” a pop up box will appear. ODIN default weights will show automatically (shown in the image below).
- Click “use custom weights” at the top of the window to input custom weights.
- After selecting custom weights, toggle between the two tabs, “category weights” and “element weights” to enter any value between 0-10. You can choose weights for 21 data categories and ten elements of coverage and openness. Weights across categories are normalized independently of the weights across elements.
- A weight of 0 will omit that category/element from scoring.
- When finished, scroll to the bottom of the page and press “Save Weights.” These weights will now be applied throughout the website for the remainder of the session. On each page, it will indicate custom weights have been
- To clear custom weights and return to ODIN stanrd weights, select “use default ODIN weights,” as shown in the image above.
- Weights are saved only during the current session. If users close their session, they will have to input their weights again.
* * * * * odin.opendatawatch.com * * * *
Acknowledgements
The Open Data Inventory is a team effort. We are pleased to acknowledge the help of all who contributed to this work.
The Open Data Inventory is managed by Jamison Crowell assisted by Tawheeda Wahabzada, under the direction of Eric Swanson. Amelia Pittman helped with the development of maps and charts. Other contributors include: Deirdre Appel, Caleb Rudow, Shaida Badiee, Misha Belkindas, Reza Farivari, and Martin Getzendanner.
ODIN Assessors
Laura Batista, Walter Leiva Bonilla, Sarah Cover, Gabrielle Hibbert, Daniel Hiller, Niña Ledonio,
Yuan Lei, Dominic Scerbo, and Marie-Charlotte Pierre.
ODIN Reviewers
Jamison Crowell, Chandrika Kaul, Tawheeda Wahabzada, and Walter Leiva Bonilla.
Website and publication design
District Design Group
Website development
Akron, Inc.
Funding Provided by the William and Flora Hewlett Foundation
