Imaging Biomarker Knowledge Transfer for Attention-Based Diagnosis of COVID-19 in Lung Ultrasound Videos

Lum, T.; Mahdavi, M.; Frenkel, O.; Lee, C.; Jafari, M. H.; Dezaki, F. T.; Woudenberg, N. V.; Gu, A. N.; Abolmaesumi, P.; Tsang, T.

The use of lung ultrasound imaging has recently emerged as a quick, cost-effective, and safe method for diagnosis of patients with COVID-19. Challenges with training deep networks to identify COVID-19 signatures in lung ultrasound data are that large datasets do not yet exist;disease signatures are sparse, but are spatially and temporally correlated;and signatures may appear sporadically in ultrasound video sequences. We propose an attention-based video model that is specifically designed to detect these disease signatures, and leverage a knowledge transfer approach to overcome existing limitations in data availability. In our design, a convolutional neural network extracts spatially encoded features, which are fed to a transformer encoder to capture temporal information across the frames and focus on the most important frames. We guide the network to learn clinically relevant features by training it on a pulmonary biomarker detection task, and then transferring the model’s knowledge learned from this problem to achieve 80% precision and 87% recall for COVID-19. Our results outperform the state-of-the-art model on a public lung ultrasound dataset. We perform ablation studies to highlight the efficacy of our design over previous state-of-the-art frame-based approaches. To demonstrate that our approach learns clinically relevant imaging biomarkers, we introduce a novel method for generating attention-based video classification explanations called Biomarker Attention-scaled Class Activation Mapping (Bio-AttCAM). Our analysis of the activation map shows high correlation with the key frames selected by clinicians. © 2021, Springer Nature Switzerland AG.