Observer agreement and clinical significance of chest CT reporting in patients suspected of COVID-19

Debray, M.-P.; Tarabay, H.; Males, L.; Chalhoub, N.; Mahdjoub, E.; Pavlovsky, T.; Visseaux, B.; Bouzid, D.; Borie, R.; Wackenheim, C.; Crestani, B.; Rioux, C.; Saker, L.; Choquet, C.; Mullaert, J.; Khalil, A.

Objectives: To assess inter-observer agreement and clinical significance of chest CT reporting in patients suspected of COVID-19. Methods: From 16th to 24th March 2020, 241 consecutive patients addressed to hospital for COVID-19 suspicion had both chest CT and SARS-CoV-2 RT-PCR. Eight observers (2 thoracic and 2 general senior radiologists, 2 junior radiologists and 2 emergency physicians) retrospectively categorized each CT into one out of 3 categories (evocative, compatible for COVID-19 pneumonia, and not evocative or normal). Observer agreement for categorization between all readers and pairs of readers with similar experience was evaluated with the Kappa coefficient. The results of a consensus categorization were correlated to RT-PCR. Results: Observer agreement across the 3 categories was good between all readers (kappa value 0.68 95%CI 0.67-0.70) and good to very good between pairs of readers (0.64-0.85). It was very good (0.81 95%CI 0.79-0.83), fair (0.32 95%CI 0.29-0.34) and good (0.74 95%CI 0.71-0.76) for the categories evocative, compatible and not evocative or normal, respectively. RT-PCR was positive in 97%, 50% and 27% of cases classified in the respective categories. Observer agreement was lower (p=0.045) and RT-PCR positive cases were less frequently categorized evocative in presence of an underlying pulmonary disease (p<0.001). Conclusion: Inter-observer agreement for chest CT reporting using categorization of findings is good in patients suspected of COVID-19. Among patients considered for hospitalization in an epidemic context, CT categorized evocative is highly predictive of COVID-19, whereas the predictive value of CT decreases between the categories compatible and not evocative.