Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans

Xuehai He; Xingyi Yang; Shanghang Zhang; Jinyu Zhao; Yichen Zhang; Eric Xing; Pengtao Xie

We observe that for CRNet which is a small-sized network, the effect of transfer learning is marginal. This is probably because a small-sized network is more robust to overfitting, therefore it has a smaller need of using transfer learning for combating overfitting. On the contrary, for large-sized neural architectures such as VGG-16, initialization with pretrained weights makes a huge difference. The reason is that large-sized networks are more prone to overfitting, especially considering that our dataset is fairly small. Under such circumstances, transfer learning has a better chance to play its value. Table IV shows the performance of the DenseNet-169 backbone with weights (1) randomly initialized, (2) pretrained on ImageNet, (3) pretrained on LMN, and (4) pretrained first on ImageNet, then on LMN. From this table, we make the following observations. First, transfer learning on either ImageNet or LMN improves performance, which further demonstrates the efficacy of transfer learning. Second, the performance of the network pretrained on ImageNet has no significant difference with that pretrained on LMN (the former has slightly better accuracy but worse F1). The two datasets have both advantages and disadvantages. ImageNet has more images and more classes than LMN, which enables learning more powerful and generalizable feature representations. The downside of ImageNet is that its images have a large domain discrepancy with the CTs in COVID19-CT whereas the images in LMN are all CTs. The advantages and disadvantages of these two datasets make them similar in providing transfer learning values. Pretraining first on ImageNet then on LMN achieves better performance than just pretraining on ImageNet. This shows that using data with complementary properties (e.g., size, domain similarity) can generate a synergistic effect in transfer learning.