Prediction in complex systems using agent-based models

Guest post by Corinna Elsenbroich & Gary Polhill

Should we ask people to stay at home during a pandemic?
Or just let the disease run its course?

The COVID-19 crisis forced governments to make difficult decisions at short notice that they then had to justify to their electorate. In many cases, these decisions were informed by computer simulations.

An advanced kind of computer simulation, known as agent-based modelling, proved particularly helpful in evaluating different options where it was used. In agent-based models, there is a virtual representation of an artificial population of human beings, each so-called ‘agent’ going about its simulated daily life, and, critically, affecting, and being affected by, other agents.

So, if one agent becomes “infected”, and spends too long near another agent not yet immune, then the computer simulation can “infect” the other agent. Furthermore, agent-based models can simulate social networks, families, friends, work colleagues, and take into account which people are likely to spend too long near another to transmit infections. Agent-based models can also simulate interactions with wider social environments. If one agent not wearing a mask finds themselves in an area where all the other agents are wearing masks, the simulated agent can decide whether to put their mask on (by allowing themselves to be influenced by the social norm), or remain mask-free (because their identity outweighs the norm, or because they cannot wear a mask for medical reasons).

Each agent has their own ‘story’, and the computer can simulate how these stories intertwine to form the narrative of the artificial population’s interaction with a communicable disease and measures to prevent its spread.

The pandemic was a vivid example of the challenges of governing complex systems. Complex systems are studied by scholars in various disciplines, including mathematics, physics, economics, sociology, computer science, geography, ecology and biology. They are fundamental to life, from the cellular to international relations levels, and as fascinating as they are challenging. The reasons why they are called ‘complex’ are the reasons that make them difficult to govern. Some of these reasons include:

• They are ‘nonlinear’. Using some made-up numbers for the purposes of illustration, nonlinearity means that if a government spends £1Bn to save the first 100,000 lives, they might have to spend £5Bn to save the next 100,000, but only £500M for the 100,000 after that. Nonlinearity is challenging mathematically; a lot of ‘classical’ mathematics (including a 200-year-old algorithm now laughably rebranded as ‘machine learning’) assumes linearity. It is from nonlinearity that we get the concept of a ‘tipping point’: the difference in habitability between 1C and 1.5C of global warming is not the same as the difference between 1.5C and 2C.
• They have ‘fat-tailed’ distributions. A mathematical law called the ‘central limit theorem’ is often used to justify assuming everything has a normal distribution. Because of this, a lot of statistics is focused on working with that distribution. In complex systems, however, the law of large numbers, on which the central limit theorem depends, does not always apply. Distributions can have ‘fat-tails’, meaning that the probabilities of extreme events are higher than if a normal distribution is assumed. Underestimating the probability of an extreme event is risky for a government, and potentially fatal to some of its population.
• They are sensitive to local circumstances. Mathematicians call this ‘non-Markovian’ or non-ergodic, and again, find themselves unable to rely on a large body of work that can be applied very successfully when there is not such sensitivity. The practical outcome is that a policy that works in one place may not work in another.
• They are not at equilibrium. Even now, for some ecologists and economists, the assertion that living systems are not at equilibrium is controversial. Systems apparently remaining in similar (or cycling) states is instead referred to in complex systems language as ‘homeostasis’. The important difference with equilibrium is that homeostasis requires energy, and so by definition is not at equilibrium. For example, your body tries to maintain its blood temperature at the same level (around 36.5C), but has different mechanisms to do this depending on whether the weather is hot or cold, and dry or humid. Mathematically, not being at equilibrium means that calculus becomes a less useful tool. For government, it may mean that after a perturbation, a society will not necessarily return to the way it lived before.
• They are evolutionary. Complex systems can adapt, innovate and learn. This means that a measure that worked historically may not work now. Indeed, even the language used to describe what people do and how they differ can change. In medical circles, we no longer speak of ‘humours’ or ‘miasmas’, but of white blood cells, bacteria and viruses, and their mutations and variants.

Agent-based modelling grew out of studying complex systems as a way of helping scientists understand them better. But that has not led to the community of practitioners being as willing to use their agent-based models to make predictions. Quite the opposite, in fact. Many practitioners, on the basis of their understanding, regard prediction in complex systems as impossible, and point to other important and useful applications of agent-based models.

All these challenges to classical mathematics make prediction in complex systems much harder. Even those who don’t regard prediction as impossible use guarded language like ‘rough forecasting’, or ‘anticipated outcomes’.

However, claiming that prediction is impossible does not help the policy-maker decide what to do about a pandemic, nor to justify the expense and curtailment of liberties to the people. Worse, there is still a significant community of researchers quite willing to ignore complexity altogether, and to apply methods to make predictions and claim them as such that rely on assumptions that are false in complex systems. (In some circumstances, over short time periods, these methods can work because complex systems don’t always behave in complex ways.) Agent-based models have been argued to have an important role in helping people make decisions in complex systems.

It might be that agent-based modellers need to find ways of participating in discussions about governing complex systems, in circles where prediction is part of the narrative, while still being true to their understanding. Rather than remaining a taboo, prediction is something agent-based modellers need to face. In a special issue of the International Journal of Social Research Methodology, we have collected contributions that aim to open up a conversation about prediction with agent-based models. They reflect a diversity of opinion as varied as the backgrounds of people in the community of practitioners.

Our beleaguered global governments, wearily emerging from the pandemic, find themselves facing an escalated war in Europe, polarized societies, economic instability, persistent misinformation spread on social media, a sixth mass-extinction, and ever-more frequent extreme weather events. Each of these issues is complex, multidimensional and multi-scale, and any solution (including doing nothing) has uncertain, unintended, cascading consequences. If agent-based modelling can help with such challenging decision-making, then it should.

The full editorial Agent-based Modelling as a Method for Prediction for Complex Social Systems is freely available International Journal of Social Research Methodology

Corinna Elsenbroich is Reader of Computational Modelling in Social and Public Health Science at University of Glasgow. Follow @CElsenbroich on Twitter and read more research via ORCID

J. Gareth Polhill (known as Gary Polhill) is a Senior Research Scientist in the Information and Computational Sciences Department at The James Hutton Institute. Follow @GaryPolhill⁩ ⁦on Twitter and read more research via ORCID

Why Questions Like “Do Networks Matter?” Matter to Methodology: How Agent-Based Modelling Makes It Possible to Answer

By Edmund Chattoe-Brown

“Scientists tend not to ask themselves questions until they can see the rudiments of an answer in their minds. Embarrassing questions tend to remain unasked or, if asked, to be asked rudely” -Peter Medawar.

Disciplines and research methods are often arbitrarily divided by the assumptions they make about the social world. Economics is based almost exclusively on its own definition of rationality which is a minority interest (and widely regarded with scepticism) in almost all other social sciences. Statisticians focus on finding “big picture” patterns in “usual suspects” variables while qualitative researchers emphasise the role of agency, interaction, and context. While this state of affairs may be adequate under the normal academic divisions of labour, it creates a particular problem for interdisciplinary research and research intended for policy. In interdisciplinary research, different (and often entrenched) assumptions must somehow be reconciled so that the outcome really is collective insight rather than simply a ragbag of disconnected “business as usual” sub-projects. In policy research, we need to be confident of all the things that actually seem to reduce crime, not just the subset that criminologists (or economists or sociologists or statisticians or ethnographers) decide that their field should attend to.

But if we want to address this problem scientifically, we need an approach that can represent different organising beliefs about the social world fairly and effectively (which quantitative and qualitative approaches, for example, notoriously cannot do with each other’s insights). If we can represent two different views of the social world using the same framework, we can then examine how much difference it makes if we assume one thing rather than another. The argument of my article (after laying out the nature of this problem) is that a form of computer simulation known as Agent-Based Modelling (Chattoe-Brown 2019) can be developed to offer such an approach. Agent-Based Modelling is increasingly recognised as a technique that offers distinctive advantages to social science in representing process and fundamental heterogeneity (not just in “variables” but also in behaviour) and  in analysing systems where simple individual interactions can lead to counter-intuitive aggregates, so-called complex systems displaying emergence (Chattoe-Brown 2013). This representational richness, based on describing social processes explicitly, allows the technique to avoid “technical” assumptions (made purely on analytical grounds) and to focus instead on the effective use of different sorts of data to justify building models in one way rather than another. (It is thus not only the technology that is distinctive but its associated methodology and relationship with different sorts of data.)

Therefore, most of the article is devoted to laying out and analysing a “worked example” concerning the social aspects of disease transmission, illustrating how Agent-Based Models operate in general and how they can be designed to answer the kind of questions that separate different fields of research. For example, does the presence or absence of social networks “matter” to the behaviour of systems? Some areas (like Social Network Analysis) take it for granted that networks do matter while others like large scale statistical analysis (with no less empirical success) analyse social behaviours without reference to network variables. To address this question, then, we can design an Agent-Based Model In which the social network can be “switched off” while all other aspects of the social process described remain the same. Any differences in the resulting behaviour of the system, therefore, necessarily arise from the presence (or absence) of social networks alone. We are effectively controlling for model assumptions independently. The result obtained from analysing this example is that static social networks matter considerably to the dynamics of disease transmission while evolving social networks make little additional difference. (Like a lot of social science, these results might be considered unsurprising with hindsight but that tells us more about hindsight than it does about the social world!)

Although the article uses the single example of networks as an aspect of social process, another aim of the article is to point out that many important social science debates tend to hinge on mere assertions endorsed by different disciplines which this approach could make a constructive contribution to addressing. For example, is decision behaviour rational, adaptive, habitual, or imitative as different disciplines assert? This debate is unlikely to progress scientifically without a technique for exploring how different kinds of decision making may give rise to distinctive patterns in data that we could discover. The same applies to the opposition between the statistical quest for “big patterns” and the qualitative emphasis on detail. Can suitably designed variants of Agent-Based Models show when “detail matters” and when it may “wash out” to leave big patterns? This sort of approach would be particularly valuable in analysing educational attainment, for example, where individual, interactional, and structural elements are all clearly in play. Being able to move these different positions forward from a “is, isn’t, is too” style of argument should be a major contribution to interdisciplinarity and more effective policy.

Of course, since writing this article, the importance of being able to draw on the best evidence from all relevant disciplines and methods has been made hugely more topical by the COVID pandemic. To tackle a real problem (which in this case is literally a matter of life and death), we need ways of understanding how geography, networks and social behaviour interact with diseases, the physics of PPE and surface contamination and many other aspects of the social process (like who cares for children when schools are closed). Simply biting off parts of the problem using existing approaches and studying them in isolation will almost certainly not be enough to produce effective policy. This article thus shows yet another way in which Agent-Based Modelling can make a distinctive contribution to advancing social science.

See full IJSRM article here.

References

Chattoe-Brown, Edmund (2013) ‘Why Sociology Should Use Agent Based Modelling’, Sociological Research Online, 18(3), article 3, August. doi:10.5153/sro.3055

Chattoe-Brown, Edmund (2019) ‘Agent Based Models’, in Atkinson, Paul, Delamont, Sara, Cernat, Alexandru, Sakshaug, Joseph W. and Williams, Richard A. (eds.) SAGE Research Methods. doi:10.4135/9781526421036836969