Automating publication of the 2021 Northern Ireland Census

Towards the end of June, the Northern Ireland Statistics and Research Agency (NISRA) launched a new product, a “Flexible Table Builder”, allowing users to create their own outputs from the 2021 Northern Ireland Census, powered by our software Cantabular.

This launch is hot on the heels of the ONS “Create a custom dataset” product launch in March, also powered by Cantabular.

NISRA’s product uses Cantabular’s own user interface which is designed specifically to give experienced users of statistical data a high level of flexibility and control over exactly what data they get and how it is represented.

Screenshot: NISRA’s new “Flexible Table Builder” product

Automating publication

Our work on the Northern Ireland census is a great example of how Cantabular can be used to automate the publication of statistical outputs, from the processed survey results all the way through to Excel downloads and interactive visualisations.

Diagram: Cantabular’s architecture for publishing statistical outputs

Automating publication in this way has significant benefits to organisations that take this approach:

Speed: the time taken to publish results after collection can be dramatically reduced, significantly increasing the value of the collected data to society.
Flexibility: users can create their own outputs, rather than being limited to a fixed set of pre-defined tables, reducing the number of expensive-to-process ad-hoc requests.
Repeatability and efficiency: an automated process has more throughput, is less error-prone, easier to audit, and when changes are required they can be made once and applied simultaneously to all outputs.

We’ve written more on this blog about exactly how we’ve automated different aspects of this process, including cross-tabulation and perturbation, output disclosure checks and metadata management.

Helping users to build their own table

With the Northern Ireland 2021 census processed and available in our software, users of the data can now use this flexible table builder to create outputs on demand that weren’t previously possible, with privacy protections applied in real-time.

For example, I can:

Create a table of Ethnic group by age by full-time student status at Local Government District level for all Northern Ireland
Add a filter to remove age groups we aren’t interested in
Increase the geographic detail by changing to District Electoral Area level
Change the geographic coverage to just look at electoral areas in Belfast
And finally pivot the data to change the layout of the table
The final table is then available in Excel, CSV or JSON format.

This is in a user interface designed to be easy to use on desktop and mobile, accessible and keyboard navigable and with hackable URLs for expert users.

Screenshot: A custom table from NISRA’s “Flexible Table Builder”

Enabling interactive exploration of data

Cantabular’s user interface helps users create and download tables of data, but its APIs, on top of its flexible architecture, allow for powerful interactive visualisations to be created.

As an example, we’ve created a mapping application that allows users to visualise characteristics of the Northern Ireland population by comparing prevalence of them in individual areas at a range of geographic levels:

Using data for the public good

Our mission at Sensible Code is to help the public sector use data for the public good. After a lot of work with NISRA and ONS, we’re very proud to see the Northern Ireland census results published using our software, and we’re looking forward to seeing how users make use of the data.