95% confidence predictions regarding coronaplague

by philg

The model of coronaplague with the best graphics says that Massachusetts, home to 2% of Americans, will suffer 12% of total COVID-19 deaths (8,219 out of 68,841, through August 4, 2020).

How sure can we be of this result?

“Learning as We Go: An Examination of the Statistical Accuracy of COVID19 Daily Death Count Predictions” (Marchant, et al., April 8, 2020) says “not very”:

Our results suggest that the IHME model substantially underestimates the uncertainty associated with COVID19 death count predictions. We would expect to see approximately 5% of the observed number of deaths to fall outside the 95% prediction intervals. In reality, we found that the observed percentage of death counts that lie outside the 95% PI to be in the range 49% – 73%, which is more than an order of magnitude above the expected percentage. Moreover, we would expect to see 2.5% of the observed death counts fall above and below the PI. In practice, the observed percentages were asymmetric, with the direction of the bias fluctuating across days.

So… the model was “wrong” (actual fell outside predicted 95 percent range) as much as 73 percent of the time!

8 thoughts on “95% confidence predictions regarding coronaplague”

    • Thanks for the link. This guy would have shut down the world economy at least four times between 2013 and 2017, no? We had outbreaks of avian influenza and warnings from the WHO. https://en.wikipedia.org/wiki/Influenza_A_virus_subtype_H7N9

      He says “when the potential risk is an extinction event, you get all hands on deck, even if the models prove to be false.” H7N9 had just as much potential to render humans extinct as coronavirus, no?

      (Now that it has broken out, unlike H7N9, if coronavirus is an “extinction event”, why aren’t the Swedes all dead? They’ve taken minimal action to avoid extinction.)

      The article celebrates Massachusetts, on track to be the worst-hit state in the U.S.: “And in Boston, the governor’s office announced the nation’s first community-wide effort to contact trace.”

      “The threat of increased travel creating a tipping point in a pandemic is nothing new to Yaneer; he has been warning the world about it for fifteen years. His concern stems from a decades-old model he created that showed that with the advent of widely-available air travel, local outbreaks would, over time, pose an existential threat to humankind.”

      Isn’t this language overheated? Even the Black Death, which killed 50% of the population in a lot of Europe, was not an “existential threat to humankind”. Is he saying that with modern technology and sanitation we’re likely to suffer a pandemic that is worse than the Black Death?

  2. I have tried to read Yaneer Ban-Yam’s writings before, but a few paragraphs in, decided he was pulling a Sokal and now just ignore it.

    I tracked the IHME data for California deaths diligently and it has done reasonably well for predicting deaths for today plus (1,2,3) days for the past month. We’ll see if the accuracy continues, as after the curve clearly turned down over the past few days it is predicting an upturn in the next few days.

    Trying to predict ICU beds seems pretty far-fetched because of so many unpredictable human factors in getting someone to that state.

  3. The sneaky long infectious incubation time of COVID-19, combined with the high R0 is very dangerous, we would be in big trouble if , for example, it acted like AIDS and destroyed people’s immune systems. And it has not really been long enough to know if there are long term serious health consequences. The Spanish Flu supposedly had some long term neurological effects (Parkinsonism) in some of the people who recovered, which only manifested years later.

    https://www.the-scientist.com/features/can-the-flu-and-other-viruses-cause-neurodegeneration–65498

    • > if … if … if … supposedly … in some …

      Thanks for a fine example of Coronahysteria, Henry. Let’s worry about hypothetical terrors. They’re more important than definite consequences.

Comments are closed.