Projecting belief – our model explained

Jack Russell

Jack Russell

— 7 min read
Some graph paper with a line chart plotted on it, with a metal ruler and pens on a wooden desk.
Photo by Isaac Smith / Unsplash

Our launch article made a projection that the share of the UK population identifying as Christian would fall to one third by 2031, mainly driven by an increase in the number of people with no religion. This article explains the model behind that projection, which is based on population forecasts from the Office for National Statistics (ONS) and some simple extrapolations of trends in religious affiliation. We’ll explain these approaches and explain how we combined them to create the ensemble model used for the final projections.

I’m aiming to make this accessible to as wide an audience as possible, so I’ll try to keep technical details to a minimum here and focus on the most important concepts. If you do want to access the detail, I'll be making all the source data and code behind the model available to view and download on github - please check it out and get in touch if you have any feedback.

High-level model structure

Our model combines two sets of projections. The first set from the ONS projects the UK population by country, age and sex until 2122(!). We used the country projections for 2022–2031, apportioning these to smaller Local District Authority areas (LDAs) using a basic attribution model.

The second set of models is our own extrapolation-based projection of the share of religious affiliation for each sub-population by age (actually birth cohort), sex and LDA, described in the main body of this article.

These two sets of projections are combined by assigning each projected population an associated projection for its religious mix, to give granular projections of population counts by religion, LDA, sex and age. Higher-level projections can then be calculated by aggregating the granular projections at the appropriate level.

Extrapolation is fundamentally about extending patterns or trends that have been observed in the past into the future. This is just like drawing a line through a series of dots on a graph, extending the line beyond where the dots end. Its basic assumption is that the patterns observed so far will continue in a similar way.

There could be several ways to draw that line though, and given different assumptions about the underlying dynamics in play, each may be more or less appropriate. Our model combines three approaches: two of these extrapolate trends directly, using linear and exponential growth assumptions respectively. The third uses a transition-matrix based approach, which can be thought of as a form of “process extrapolation”. Our final model combines each of these projections in an ensemble that is optimised for accuracy when back-tested on historical data.

Linear extrapolation

Linear extrapolation is the simplest approach of all. It assumes that the absolute rate of change will stay the same, which is equivalent to drawing a best-fit straight line through the historical points and extending that straight line forwards. So if a religious group has been declining by 1% every year in absolute terms, linear extrapolation will project it to continue declining at exactly that rate. If it halves in one year, it will disappear the next.

Linear assumptions can be appropriate in cases where affiliation change is relatively slow and steady and where the trend is not approaching saturation points or natural limits (0% or 100%).

💡
Nerd note: As long as individual populations don’t bump into these limits, projections made using linear extrapolation will not require normalisation, as the net sum of changes for all groups will remain at zero and the total population will be preserved.

Exponential extrapolation

Exponential extrapolation assumes that the relative rate of change will stay the same. If a population doubles one year, it assumes that it will double again the next. If it halves, then by next year, the population will have halved again to a quarter of the original level.

This can be appropriate for social phenomena that spread through networks of people. As more people adopt a technology or idea, the exposure to that idea increases. If a consistent proportion of people change their affiliation after exposure, an exponential pattern will emerge.

💡
Nerd note: Projections of population shares for multiple categories made using exponential extrapolation require normalisation to ensure that they sum up to 100%. This normalisation means that the absolute growth rates are not exponential - it is the relative shares of the populations which change exponentially. The result is equivalent to transforming the odds by a constant ratio in the case of a two-category population, or multiplying the dirichlet parameter vector by a fixed transformation vector in the general multi-category case.

Transition matrix projection

Finally, we use a transition-matrix based approach. Whereas the previous approaches extrapolate the proportion of people affiliated with each religious category directly, this approach makes an assumption about the underlying process involved – that people with any given religious identity have a constant likelihood-per-unit-time (or rate) of swapping to each of the other categories. These rates of transition between each pair of categories are encoded in a matrix.

The transition matrix we use in this case could be described as “lazy” in that it encodes the least amount of switching possible to explain the observed change in population shares. All categories that are declining are assumed to be losing adherents to all categories that are growing, in proportion to the population share respectively being lost or gained. While this is won't be true in practice, it provides a decent approximation which enables us to project a single step forward. This approach makes the least radical projections of change, as net movements between categories decrease over time as the population shares approach an equilibrium.

Model granularity

Trends in religious affiliation vary geographically and by sex and age. Uniquely in the survey world, census data volumes are so large that they allow robust modelling of subsamples cut by multiple attributes, enabling us to build separate models in this case for over 15,000 granular micro-populations: for each of 387 local district authorities (LDAs) across the UK, we make separate projections for female and male populations within each five-year birth cohort available.

As an example, one micro-population is males born between 1991–1995 living in Cornwall; another is females born between 1976-1980 living in Belfast.

For each micro-population, we extrapolated the specific, local changes in religious affiliation observed for this group between 2011–2021 using the linear, exponential, and transition matrix-based approaches described above, to project the change we might expect to see over the next ten years. This approach implicitly assumes that local trends will continue.

We also calculated the national trend for each sex/age combination using the same techniques and applied it to each group. In contrast to the local projections, these national projections represent the change we would expect to see if the pattern of changing affiliations in the LDA "reverted to the national mean".

Combining these two approaches yields a total of six projections for each micro-population.

Building the ensemble model

Ensemble models typically combine several simple models. They can produce more accurate and robust predictions because the errors made by individual models often cancel out when aggregated. This is a bit like asking a group of people for an estimate and taking the average – you’re more likely to get a good answer than relying on one person alone.

You might trust some of these people more than others though, in which case you might pay more attention (assign more weight) to their estimates. With machine learning, the relative weight assigned to each model is normally determined in a "training" phase which aims to find weights that optimise prediction performance on a training set known data.

Our final ensemble model uses an optimally-weighted combination of the six projections described above, trained by projecting the 2001–2011 trends (available for England & Wales only) forward to 2021, and comparing these projections to the actual 2021 outcomes.

💡
Nerd note: the ensemble weights were optimised to minimise the mean squared error using a training set containing a random 50% sample of LDAs across England & Wales. The real-world performance of the ensemble was then estimated using a test set containing the remaining LDAs.

Predicting the affiliation of under-tens

If you've been paying close attention to the modelling approach so far, you may have realised that it won't work to predict the religious affilation of 2031's under-tens. These future children wouldn't have been alive in 2021, so there would have been no baseline to project forward from.

For this group, we recognise that their parents or carers will fill the census in for them. Making use of the same set of models that we used to extrapolate temporal trends within individual birth cohorts, in this case we model the difference between age groups, as a proxy for modelling the difference between parental affiliations and those of the children on whose behalf they are responding. To keep the model (slightly) simpler, we pick the best performing cohort-model pair as opposed to creating an ensemble. As with the projections for over-tens, full details are on github.

Combining with the ONS population data

Once the percentage shares of each religion had been projected for each micro-population, we combined these with the population projections from ONS to give a detailed set of projections of population counts by religion, LDA, sex and age.

💡
Nerd note: The ONS figures are only supplied at the country level, so first we needed an approach to apportion these to LDAs. To do this, we first made our own crude LDA-level population projections, using historical "survival rates" which compared birth cohort populations within each LDA to the same cohorts ten-years later. (These are not true survival rates as they encompass migration into and out of LDAs, as well as mortality rates.) New births were projected using the last 5 years of birth data. These crude predictions were then scaled uniformly across LDAs until they matched the ONS totals in aggregate.

The final step - aggregating and displaying projections

Our final set of granular projections was used to build the graphs in the main article, using appropriate aggregations across geographic and demographic attributes. But that article just scratched the surface of what they might tell us – there are countless other ways to slice and dice the data.

If you'd like to interrogate the projections to check out whether they map to your local knowledge, or to test your own hypotheses, the data is available on github alongside the raw source data (from ONS and the census surveys) and the code used to produce our projections.

Any questions?

I've tried to summarise the key points of my approach in this article, and the full code is available to be examined on github. However if you have any questions or constructive suggestions, please do get in touch at info@believethedata.org. And please subscribe if you'd like to be notified next time I've got an analysis to share.

Thanks for reading!

Read more