Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans

Xuehai He; Xingyi Yang; Shanghang Zhang; Jinyu Zhao; Yichen Zhang; Eric Xing; Pengtao Xie

Given a target task (e.g., diagnosing COVID-19 from CT scans in our case) that has limited training data, transfer learning aims to leverage large-scale data and human-provided labels from other source tasks to learn expressive and generalizable feature representations to help with the learning of the target task. A commonly used approach [45] is to pretrain a deep neural network -which is used for feature extraction -on large datasets in the source tasks by fitting the humanannotated labels therein, then fine-tune this pretrained network on the target task. In our case, we can take a classic neural architecture such as ResNet [46] and its weights pretrained on large-scale image classification datasets such as ImageNet, then fine-tune it on the COVID19-CT dataset, with the goal of transferring the images and classes labels in ImageNet into our task for mitigate the deficiency of COVID-19 CTs. When applying this strategy, we should keep several caveats in mind. First, the image data in source tasks have a large domain discrepancy with COVID-19 CTs. For example, the ImageNet images mostly belong to categories in the general domain, such as cat, dog, chair, etc. whereas the images in our task are CTs. The visual appearance, size, resolution of ImageNet images are quite different from chest CTs. As a result, the visual representations learned on ImageNet may not be able to represent CT images well, which casts doubts on the transferability from other sources of images to COVID-19