“All Models are Wrong – But Some are Useful”

We use simulation and modelling to replicate real world systems and processes – enabling us to better explain how something works, or to predict what happens when something changes. Simulation and modelling are well used in transport, with digital replications of the real world being used to explore the impact of new infrastructure and new services on congestion, the environment and the economy.

The models that have made the news recently are epidemiological – mathematical representations of systems just like we use in transport and infrastructure, but concerned with the spread of disease (COVID-19) through a population and the impact on health.

We can draw lots of parallels between epidemiological and transport models. Neither are constrained purely to physical systems – we must capture human behaviour and the interactions between people. Social systems are inherently chaotic, complex and filled with uncertainty. These models are also used extensively for decision making, particularly when other evidence is lacking.

There’s often a lot at stake when we model these systems – the economy, the environment, health, jobs and welfare.

Undeniably, models have never been more important.

Uncertainty, risk and assumption

The statistician George Box is reported to have said “All models are wrong, but some are useful”. In this case, when we refer to models as being “wrong”, we aren’t talking about errors or mistakes in software – we’re talking about the simplifications that are necessary when dealing with large populations and complex systems. We don’t claim to know exactly how every individual might respond to a new bus route, nor do we know how every individual will respond to an illness or a course of treatment. We use observations from the real world and fill in the gaps based on our expert understanding of the system to help us gain better understanding of things that haven’t happened yet. .

We almost always need these simplifications and assumptions when dealing with complex systems – such as how people travel or how diseases propagate – or else we wouldn’t be able to create useful models with the data available.

It’s not our aim to create a perfect replica of the system – instead we want something we can trust to help us explain and make decisions.

Making models useful

If we know models aren’t perfect, how do we make sure they are useful?

Modellers of transportation systems have always stressed the importance of proper interpretation and communication of results. Fundamental to this is understanding what a model is for, what it isn’t for, and what assumptions have been made. Crucially, we shouldn’t be communicating model results based around only one version of the future – we need to produce multiple scenarios, varying inputs and assumptions, always asking our models “what if?”. We know that different assumptions lead to different results, different interpretations and, crucially, different decisions.

We shouldn’t bury these assumptions in our models never to be discussed – we need to make them a central part of our reporting, alongside a wider range of potential futures.

The new normal

The COVID-19 pandemic has fundamentally changed the way we travel and how we move goods.

Distribution, volume, trip lengths, mode – things that are fundamental to how we model transport systems are not the same as they were just a few weeks ago. We don’t yet know how long this will last, the permanency of behavioural change, or the fallout for the economy and society at large. But we can question some of the emerging trends in mobility and explore what that means for the future of transport and infrastructure. Will workers travel less for commuting and business reasons? What if people avoid crowded buses and trains in favour of walking, cycling and private cars? What does this mean for car ownership, for ridesharing and for on-demand mobility?

The future is complex and uncertain. Any model that claims to have “the answer” will almost certainly be wrong – but we can use our models to ask “what if?” and explore what these different outlooks could mean for us.