Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans

Xuehai He; Xingyi Yang; Shanghang Zhang; Jinyu Zhao; Yichen Zhang; Eric Xing; Pengtao Xie

The results of transfer learning from ImageNet are shown in Table III (columns marked with "Trans."), where we first train the networks from scratch on ImageNet and then finetune them on the COVID19-CT dataset. Comparing these results with those achieved by randomly initialized networks (columns marked with "Rand." in Table III) , we can see pretraining on ImageNet significantly improves classification performance. This demonstrates the effectiveness of transfer learning, which leverages large-scale images and their class labels in source tasks to help with the learning of the target task. In certain cases, the benefits of transfer learning are highly significant. For example, for VGG16, when trained with random initialization, it performs the poorest. Transfer learning helps it to improve accuracy by 10% (absolute improvement), F1 by 8% (absolute), and AUC by 8% (absolute), exceeding the performance of networks (e.g., ResNet) that have more sophisticated architectures designed for preventing overfitting, even when these networks are pretrained as well.