Case Study

 Office for National Statistics, UK

Modernising statistical publication with the Office for National Statistics

Key outcomes

2×

Faster to publication than previous census

60m

Rows of confidential data

100m+

Tables can be built from data

The Challenge

The Office for National Statistics (ONS) conducts a census of the England and Wales population every 10 years. The census dataset is large, has complex geographies, and 400 variables, with billions of possible tables potentially available. For the 2021, the ONS had a vision to transform the publication of census data and release far more data than before, much more quickly:

  • Create cross-tabulations from microdata with disclosure checks automated in real-time, with data returned to the user in less than a second.

  • Allow users to build any table they want on-demand, instead of having to select from a list of precomputed tables.

  • Provide a single online tool where a user can build or find any census cross-tabulation they want.

Approach

Building on an early version of Cantabular we iteratively developed the disclosure control algorithms needed by the ONS and integrated them into our software, optimising their performance to ensure queries would always run in less than a second.

We worked closely with ONS staff and users through regular prioritisation meetings and show and tells to ensure needs were met and to respond quickly to feedback.

We developed data pipelines, APIs and additional features to support the integration of Cantabular into Census output products.

We provided support and training to ONS staff to enable the deployment of the software within ONS’s firewall.

The Solution

Cantabular was extended with additional functionality to apply disclosure control techniques in real-time when generating tabular census outputs on-demand.

We provided the ONS with a metadata service to allow the incorporation of reference metadata along with the data, in both English and Welsh, and accessible in a single API.

We invented a disclosure rules language that allows the ONS to write disclosure checks specific to their census data without having to share the exact rules and their parameters with us.

Cantabular powered the creation of 2021 England and Wales tabular census outputs, whether generating them on-demand from microdata or acting as a data repository for them.

Outcomes

Secure Tabulations

Cantabular provides safe, flexible access to tabulations from over 60 million rows of confidential microdata from the 2021 England and Wales census.

Countless Tables

Cantabular provides safe, flexible access to tabulations from over 60 million rows of confidential microdata from the 2021 England and Wales census.

Complete Customisation

100s of millions of possible tables can be built by users from the published census data.

Faster publishing

ONS staff deployed the software themselves in their own environment and created their own user interface powered by Cantabular's API.

It allows us to release billions of anonymised census statistics far more quickly than ever before.

Jen Woolford

Director of Population Statistics, ONS

Blog

New ways to access Census 2021 data

Disclosure control approaches

As well as providing a mechanism for automating disclosure checks, we implemented three different disclosure control algorithms for the ONS:

Schedule a demo

Use Cantabular on your next project and publish mandatory data automatically, securely and with full control. Contact us to arrange a convenient date.

Request a demo See pricing