Recurrent Neural Reinforcement Learning for Counterfactual Evaluation of Public Health Interventions on the Spread of Covid-19 in the world

Ge, Q.; Hu, Z.; Zhang, K.; Li, S.; Lin, W.; Jin, L.; Xiong, M.

As the Covid-19 pandemic soars around the world, there is urgent need to forecast the expected number of cases worldwide and the length of the pandemic before receding and implement public health interventions for significantly stopping the spread of Covid-19. Widely used statistical and computer methods for modeling and forecasting the trajectory of Covid-19 are epidemiological models. Although these epidemiological models are useful for estimating the dynamics of transmission of epidemics, their prediction accuracies are quite low. Alternative to the epidemiological models, the reinforcement learning (RL) and causal inference emerge as a powerful tool to select optimal interventions for worldwide containment of Covid-19. Therefore, we formulated real-time forecasting and evaluation of multiple public health intervention problems into off-policy evaluation (OPE) and counterfactual outcome forecasting problems and integrated RL and recurrent neural network (RNN) for exploring public health intervention strategies to slow down the spread of Covid-19 worldwide, given the historical data that may have been generated by different public health intervention policies. We applied the developed methods to real data collected from January 22, 2020 to June 28, 2020 for real-time forecasting the confirmed cases of Covid-19 across the world. We forecasted that the number of laboratory confirmed cumulative cases of Covid-19 will pass 26 million as of August 14, 2020.