Joint polyp detection and segmentation with heterogeneous endoscopic data

Endoscopy is commonly used for the early diagnosis of colorectal cancer. However, the endoscope images are usually obtained under different illumination conditions, at various sites of the digestive tract, and from multiple medical centers. The collected heterogeneous dataset is a challenging problem in developing automatic and accurate segmentation and detection models. To address these issues, we propose comprehensive polyp detection and segmentation in endoscopic scenarios with novel insights and strategies. For the detection task, we perform joint optimization of classification and regression with adaptive training sample selection strategies in order to deal with the heterogeneous problem. Our detection model achieves 1st place in both first and second rounds of EndoCV 2021 polyp detection challenge. Specifically, the proposed detection framework achieves full-scores (1.0) on AP_large and AP_middle in the 1st round, and 0.8986 ± 0.1920 of score-d on the 2nd round. For the segmentation task, we employ HRNet as our backbone and propose a low-rank module to enhance the generalization ability across multiple heterogeneous datasets. Our segmentation model achieves 0.7771 ± 0.0695 score and ranked 4th place in EndoCV 2021 polyp segmentation challenge.