Using Digital Surveillance Tools for Near Real-Time Mapping of the Risk of International Infectious Disease Spread: Ebola as a Case Study

Sangeeta Bhatia; Britta Lassmann; Emily Cohn; Malwina Carrion; Moritz U.G. Kraemer; Mark Herringer; John Brownstein; Larry Madoff; Anne Cori; Pierre Nouvellet

Although our model was not always able to robustly predict the future incidence in the three mainly affected countries, we found that it allowed to robustly predict the presence or absence of cases in all countries in Africa up to a week in advance. For each week and each country in Africa, our model generated an alert if the predicted incidence (using a predetermined percentile of the forecast interval) was greater than 0. We classified an alert for a given week as a true alert when the observed incidence was also greater than 0, as a false alert when no cases were observed, and as a missed alert if cases were observed but were not predicted by the model. Using different percentiles of the forecast (e.g., the median or the 95 th percentile) yielded different rates of true, false and missed alerts, which were summarised in a ROC curve ( Fig 4A) . Overall, our model achieved high sensitivity (i.e., true alert rate) but variable specificity (i.e., 1 -false alert rate). Maximising the average between sensitivity and specificity led to 93.7% sensitivity and 96.0% specificity at 42.5% threshold ( Fig 4A) . The sensitivity of the model remained high over longer forecast horizon while the specificity deteriorated with more false alerts being raised 4 week ahead (Fig 47) . Both the sensitivity and the specificity of the model remained high when the analysis was restricted to all countries in Africa other than the three majorly affected countries (Guinea, Liberia and Sierra Leone) . The average of sensitivity and specificity was maximum at 92.5% threshold in this case with 85.7% sensitivity and 81.7% specificity (Fig 48) . The model exhibited high sensitivity (83.3%) and specificity (82.0%) in predicting presence of cases in weeks following a week with no observed cases in all countries in Africa (Fig 49) at 93% threshold (similarly chosen to maximise the average between sensitivity and specificity). Out of the 9 one-week ahead missed alerts in this case, 3 are in Liberia towards the end of the epidemic, after Liberia had been declared Ebola free on two separate occasions [22, 23] . The serial interval distribution that we have used does not account for very long intervals between infections such as that associated with sexual transmission. Using the latest available data on pairs of primary and secondary infections and models that allow for more heterogeneity in the distribution of cases e.g., using Negative Binomial distributions could potentially improve the assessment of risk of spread in such cases [24, 25] .