The accuracy of teleradiologists in diagnosing COVID-19 based on a French multicentric emergency cohort

Nivet, Hubert; Crombé, Amandine; Schuster, Paul; Ayoub, Thomas; Pourriol, Laurent; Favard, Nicolas; Chazot, Alban; Alonzo-Lacroix, Florian; Youssof, Emile; Ben Cheikh, Alexandre; Balique, Julien; Porta, Basile; Petitpierre, François; Bouquet, Grégoire; Mastier, Charles; Bratan, Flavie; Bergerot, Jean-François; Thomson, Vivien; Banaste, Nathan; Gorincour, Guillaume

OBJECTIVES: To evaluate the accuracy of diagnoses of COVID-19 based on chest CT as well as inter-observer agreement between teleradiologists during on-call duty and senior radiologists in suspected COVID-19 patients. MATERIALS AND METHODS: From March 13, 2020, to April 14, 2020, consecutive suspected COVID-19 adult patients who underwent both an RT-PCR test and chest CT from 15 hospitals were included in this prospective study. Chest CTs were immediately interpreted by the on-call teleradiologist and were systematically blind reviewed by a senior radiologist. Readings were categorised using a five-point scale: (1) normal; (2) non-infectious findings; (3) infectious findings but not consistent with COVID-19 infection; (4) consistent with COVID-19 infection; and (5) typical appearance of COVID-19 infection. The diagnostic accuracy of chest CT and inter-observer agreement using the kappa coefficient were evaluated over the study period. RESULTS: In total, 513 patients were enrolled, of whom 244/513 (47.6%) tested positive for RT-PCR. First readings were scored 4 or 5 in 225/244 (92%) RT-PCR+ patients, and between 1 and 3 in 201/269 (74.7%) RT-PCR− patients. The data were highly consistent (weighted kappa = 0.87) and correlated with RT-PCR (p < 0.001, AUC(1st-reading) = 0.89, AUC(2nd-reading) = 0.93). The negative predictive value for scores of 4 or 5 was 0.91–0.92, and the PPV for a score of 5 was 0.89–0.96 at the first and second readings, respectively. Diagnostic accuracy was consistent over the study period, irrespective of a variable prevalence rate. CONCLUSION: Chest CT demonstrated high diagnostic accuracy with strong inter-observer agreement between on-call teleradiologists with varying degrees of experience and senior radiologists over the study period. KEY POINTS: • The accuracy of readings by on-call teleradiologists, relative to second readings by senior radiologists, demonstrated a sensitivity of 0.75–0.79, specificity of 0.92–0.97, NPV of 0.80–0.83, and PPV of 0.89–0.96, based on “typical appearance,” as predictive of RT-PCR+. • Inter-observer agreement between the first reading in the emergency setting and the second reading by the senior emergency teleradiologist was excellent (weighted kappa = 0.87). ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s00330-020-07345-z) contains supplementary material, which is available to authorized users.