Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans

Xuehai He; Xingyi Yang; Shanghang Zhang; Jinyu Zhao; Yichen Zhang; Eric Xing; Pengtao Xie

To further solve this problem, we propose an Self-Trans approach, which integrates contrastive self-supervision [3] into the transfer learning process. Self-supervised learning (SSL) [36] , [39] , [41] , [42] is a learning paradigm which aims to capture the intrinsic patterns and properties of input data (e.g., CT images) without using human-provided labels. The basic idea of SSL is to construct some auxiliary tasks solely based on the data itself without using human-annotated labels and force the network to learn meaningful representations by performing the auxiliary tasks well. Typical selfsupervised learning approaches generally involve two aspects: constructing auxiliary tasks and defining loss functions. The auxiliary tasks are designed to encourage the model to learn meaningful representations of input data without utilizing human annotations. The loss functions are defined to measure the difference between a model's prediction and a fixed target, the similarities of sample pairs in a representation space (e.g., contrastive loss), or the difference between probability distributions (e.g., adversarial loss). In this work, we design the auxiliary tasks based on the contrastive loss [41] , [42] , [51] to provide self-supervision for the transfer learning process. To be specific, the auxiliary task is to judge whether two images created via random data augmentation are augments of the same original image. We build a large and consistent dictionary on-the-fly based on the contrastive loss to fulfill this auxiliary task. To fully explore the structure and information of the CT images, we apply Self-Trans on both external large-scale lung CT datasets and our collected COVID19-CT dataset. 1) Contrastive learning for self-supervision: Given an original image in the dataset, contrastive self-supervised learning (CSSL) [51] performs data augmentation of this image and obtains two augmented images: x q and x k , where the first one is referred to as query and the second one as key. Two networks f q (·; θ q ) and f k (·; θ k ), referred to as the query encoder and the key encoder and parameterized by weights θ q and θ k , are used to obtain latent representations -q = f q (x q ; θ q ) and k = f k (x k ; θ k ) -of the query and key images respectively. A query and a key belonging to the same image are labeled as a positive pair. A query and a key belonging to different images are labeled as a negative pair. The auxiliary task is: given a (query, key) pair, judging whether it is positive or negative.