Comparing early outbreak detection algorithms based on their optimized parameter values

Wang, Xiaoli; Zeng, Daniel; Seale, Holly; Li, Su; Cheng, He; Luan, Rongsheng; He, Xiong; Pang, Xinghuo; Dou, Xiangfeng; Wang, Quanyi

BACKGROUND: Many researchers have evaluated the performance of outbreak detection algorithms with recommended parameter values. However, the influence of parameter values on algorithm performance is often ignored. METHODS: Based on reported case counts of bacillary dysentery from 2005 to 2007 in Beijing, semi-synthetic datasets containing outbreak signals were simulated to evaluate the performance of five outbreak detection algorithms. Parameters’ values were optimized prior to the evaluation. RESULTS: Differences in performances were observed as parameter values changed. Of the five algorithms, space–time permutation scan statistics had a specificity of 99.9% and a detection time of less than half a day. The exponential weighted moving average exhibited the shortest detection time of 0.1 day, while the modified C1, C2 and C3 exhibited a detection time of close to one day. CONCLUSION: The performance of these algorithms has a correlation to their parameter values, which may affect the performance evaluation.