Staff Opinion: What’s up with these COVID-19 models changing all the time?

This reflection by Saul Thorkelson and Hannah Henry ’19 is part of the Community Voices op-ed series for the BPI Public Health Journal. Through the COVID-19 crisis, BPI alumni, staff, and faculty will be posting reflections about their work and response to the virus here on the BPI Blog. 

The coronavirus pandemic is hard to deal with for innumerable reasons; one of them is, simply, not knowing what we most want to know. Is it safe to go outside? How dangerous is it to visit a friend? When will we be able to get haircuts? Why can’t we go back to work if the spread is slowing down here?

Crucially: how long will this last, and how many people are going to get sick before the virus is under control?

No one can predict the future. Instead, researchers and government officials use models to project the likely course of the disease. These models give us solid answers — numbers and dates— and answers are comforting. They give us security by providing the sense that there is an end to these disconcerting, destabilized times. That said, they can seem unreliable, especially when they don’t predict what ends up happening or new models give us dramatically different predictions.

But projection models aren’t meant to give concrete answers to concrete questions. This is not just because the information included in models, such as current number of cases, contagiousness, incubation period, fatality rates, and recovery time, is incomplete (to put it mildly). It’s also because statistical models are designed to extrapolate: to provide the best estimates based on likely scenarios. They are based on assumptions of continuity–that today’s number of new cases, today’s fatality rates, and today’s rules about businesses being open will stay relatively constant. 

Let’s think for a moment about the assumptions. Think of this like predicting the path of a storm: here comes Hurricane Etta, tearing up through the Gulf of Mexico towards the Carolinas — but wait! Forecasters receive new information: the winds are changing, the storm system’s turning north up the coast, things are looking worse for southern New England now. When you hear there’s a hurricane coming, you know the path could change, that it might be milder (or more severe) than the forecasters are saying. Either way, it’s helpful to know how to react, to prepare for what’s coming. This is true of epidemiological projections, too. 

There are limits to this analogy, though. We can’t do much as far as changing the course of the hurricane (although we do make decisions about how to mitigate these hazards). We don’t control the winds and the sea temperatures. But, on the population level, we have agency over the course of the pandemic. People are carriers of the virus. It spreads from one person to the next, or it doesn’t. Individuals’ decisions matter, but are constrained by the options each has available to them. Our collective fate is both biological and political; it lies in large part with the decisions made by government officials. Where one city makes the call that it’s safe to allow children to use playgrounds, another might not. Epidemiologists cannot predict that; they can only consider the set of plausible scenarios and model them based on the best information about the virus available.

So the models of the pandemic don’t give us precise forecasts because their inputs — calculated using current data — might be off, and because they are based on assumptions about how well the past matches the future.

Think of how a shortage of testing kits means that nobody knows exactly how many cases there are in Brooklyn (or in Atlanta, or where you live). This is an example of imperfect, but needed, inputs to models. As for assumptions, scientists have to make do with not knowing how much difference it makes that people live in a building with two apartments or 40, or in a house with a wide lawn, or, for that matter, in the woods with no visible neighbors. They can’t model every single variation in every place. So, what governors in other states have done over the past weeks was based in large part on seeing the crisis unfold in New York. That isn’t a mistake. It’s just based on the unknowns and the fact that our expectations about the future derive from what we know about the present and the past occurring in particular locations, which we then use as the basis for projections. 

What we knew about fatality rates a month ago came mainly from observing the spread of the coronavirus in China, Iran, and Italy. The Italian population has proportionally more elderly people, who are more often living at home than in nursing homes, and many people live in villages rather than in the dense neighborhoods of New York City.

COVID-19 case reports in the US recently suggested that we had reached a plateau in the number of new cases per day, and in the hardest-hit areas like New York, the number of new cases per day is now declining. Considering these trends, it would be reasonable to think that things are getting safer overall and that we should shift towards more businesses being open and fewer precautions in everyday life. Indeed, some governors have decided to lift stay at home orders. As a result of their decisions, there will be more interaction and more transmission of the virus. Epidemiological models with new, higher projected numbers of COVID-19 cases and fatalities were revised to account for those decisions.

We should not consider models to be wrong if they differ from what we later observe, because they are not meant to be correct. They are meant to inform us of the most likely scenarios based on available data and particular assumptions. Nor is it an indication that an earlier model was wrong if a new one predicts a very different path of the pandemic. The people reading and responding to their models are moving targets: audiences and policymakers react. Their reaction means that the most likely scenarios will change, too. We have seen this many times in the past weeks, and we can expect to see it again as we and government policymakers make choices about how to manage this crisis.

Dr. Saul Thorkelson is BPI Site Director for Taconic Correctional Facility and a faculty member at Bard Microcollege at BPL. Saul is a sociologist and demographer whose research centers on immigrant identity and integration.

Hannah Henry ’19 is a Data and Research Assistant for BPI and tutor in biology and statistics. She completed a BA in Biology with a concentration in global public health.