Development and Evaluation of an AI System for COVID-19

Cheng Jin; Weixiang Chen; Yukun Cao; Zhanwei Xu; Xin Zhang; Lei Deng; Chuansheng Zheng; Jie Zhou; Heshui Shi; Jianjiang Feng

The copyright holder for this preprint . https://doi.org/10.1101/2020.03.20.20039834 doi: medRxiv preprint cy of the AI system was comparable to that of experienced radiologists from the outbreak center, who achieved higher sensitivity (94.70%), specificity (91.39%) and AUC (98.05%). Among the five professional readers in the radiology department, only one was able to produce a higher diagnostic accuracy than the AI system. This automatic, high-precision, non-invasive diagnostic system was developed to provide clinicians with easy-to-use tools. Given the chest CT of a suspected patient as input, the AI system can automatically output the diagnosis result. In the reader study, the average reading time of radiologists was 6.5 min, while that of AI system was 2.73 s, which can significantly improve the productivity of radiologists. Meanwhile, we found that 71% (15/21) of errors made by radiologists could be corrected by AI system. It means that AI system can be used as an effective secondary reader to provide reference suggestions when the radiologist is not sure about the case or when multiple radiologists are inconsistent. In general, AI can be adapted to different requirements. According to the highly sensitive settings, it can screen out suspicious patients for confirmation by doctors; In accordance with the highly specific settings, it can warn possible diagnosis errors made by the doctor; or an optimal threshold value is chosen according to the prior probability of infectious diseases and the local prevention and control strategy.