In recent years, breakthrough technologies in artificial intelligence (AI) and the use of satellite imagery made it possible to disrupt the way we collect, process, and analyze data. Facilitated by the intersection of new statistical techniques and the availability of (big) data, it is now possible to create hypergranular estimates.

National statistical offices (NSOs) could be at the forefront of this change. Conventional tasks of statistical offices, such as the coordination of household surveys and censuses, will remain at the core of their work. However, just like AI can enhance the capabilities of doctors, it also has the potential to make statistical offices better, faster, and eventually cheaper.

Still, many countries struggle to make this happen. In a COVID-19 world marked by constrained financial and statistical capacities, making innovation work for statistical offices is of prime importance to create better lives for all. PARIS21 and World Data Lab have joined forces to support innovation in statistical offices and make them fit for this purpose, including Colombia’s national statistical office. If we enrich existing surveys and censuses with geospatial data, it will be possible to generate very granular and more up-to-date demographic and poverty estimates.

In the case of Colombia, this novel method facilitated a scale-up from existing poverty estimates that contained 1,123 data points to 78,000 data points, which represents a 70-fold increase. This results in much more granular estimates highlighting Colombia’s heterogeneity between and within municipalities.

The averages for each municipality still contain big variances as poverty depends on many more factors than geography.

Read the full article about improving national statistical offices' data analysis by Juan Daniel, Katharina Fenz, François Fonteneau, and Simon Riedl at Brookings.