Trade-offs between individual and ensemble forecasts of an emerging infectious disease

Oidtman, R. J.; Omodei, E.; Kraemer, M. U. G.; Casteneda-Orjuela, C. A.; Cruz-Rivera, E.; Misnaza-Castrillon, S.; Cifuentes, M. P.; Rincon, L. E.; Canon, V.; de Alarcon, P.; Espana, G.; Huber, J. H.; Hill, S. C.; Barker, C. M.; Johansson, M. A.; Manore, C. A.; Reiner, R. C.; Rodriguez-Barraquer, I.; Siraj, A. S.; Frias-Martinez, E.; Garcia-Herranz, M.; Perkins, A.

When new pathogens emerge, numerous questions arise about their future spread, some of which can be addressed with probabilistic forecasts. The many uncertainties about the epidemiology of emerging pathogens can make it difficult to choose among model structures and assumptions, however. To assess the potential for uncertainties about emerging pathogens to affect forecasts of their spread, we evaluated the performance of a suite of 16 forecasting models in the context of the 2015-2016 Zika epidemic in Colombia. Each model featured a different combination of assumptions about the role of human mobility in driving transmission, spatiotemporal variation in transmission potential, and the number of times the virus was introduced. All models used the same core transmission model and the same iterative data assimilation algorithm to generate forecasts. By assessing forecast performance through time using logarithmic scoring with ensemble weighting, we found that which model assumptions had the most ensemble weight changed through time. In particular, spatially coupled models had higher ensemble weights in the early and late phases of the epidemic, whereas non-spatial models had higher ensemble weights at the peak of the epidemic. We compared forecast performance of the equally-weighted ensemble model to each individual model and identified a trade-off whereby certain individual models outperformed the ensemble model early in the epidemic but the ensemble model outperformed all individual models on average. On balance, our results suggest that suites of models that span uncertainty across alternative assumptions are necessary to obtain robust forecasts in the context of emerging infectious diseases.