Domain Knowledge Driven Multi-modal Segmentation of Anatomical Brain Barriers to Cancer Spread

Zou, Xiaoyang; Dou, Qi

It is important to accurately segment anatomical brain barriers to cancer spread with multi-modal images, in order to assist definition of the clinical target volume (CTV). In this work, we explore a multi-modal segmentation method largely driven by domain knowledge. We apply 3D U-Net as the backbone model. In order to reduce the learning difficulty of deep convolutional neural networks, we employ a label merging strategy for the symmetrical structures which have both left and right labels, to highlight the structural information regardless of the locations. Moreover, considering the existence of visual preference for certain modality and mismatches in co-registration, we adopt a multi-modality ensemble strategy for multi-modal learning to enable the models better driven by domain knowledge of this task, which is different from fully data-driven methods, like early fusion strategy for multi-modal images. By contrast, multi-modality ensemble strategy yields better segmentation results. Our method achieved an average score of 0.895 on MICCAI 2020 Anatomical Brain Barriers to Cancer Spread Challenge’s final test dataset (https://abcs.mgh.harvard.edu/.). Detailed methodologies and results are described in this technical report (This work was done when X. Zou did remote internship with CUHK.).