TY - GEN
T1 - Prototypical Interaction Graph for Unsupervised Domain Adaptation in Surgical Instrument Segmentation
AU - Liu, Jie
AU - Guo, Xiaoqing
AU - Yuan, Yixuan
N1 - Funding Information:
This work was supported by National Natural Science Foundation of China (62001410), Hong Kong Research Grants Council (RGC) Early Career Scheme grant 21207420 (CityU 9048179) and Hong Kong RGC Collaborative Research Fund grant C4063-18G (CityU 8739029).
Publisher Copyright:
© 2021 Springer Nature Switzerland AG
PY - 2021/9/23
Y1 - 2021/9/23
N2 - Surgical instrument segmentation is fundamental for the advanced computer-assisted system. The variability of the surgical scene, a major obstacle in this task, leads to the domain shift problem. Unsupervised domain adaptation (UDA) technique can be employed to solve this problem and adapt the model to various surgical scenarios. However, existing UDA methods ignore the relationship among different categories, hindering the model learning discriminative features from a global view. Additionally, the adversarial strategy utilized in these methods only narrows down the domain gap at the end of the network, leading to the poor feature alignment. To tackle above mentioned problems, we advance a semantic-prototype interaction graph (SePIG) framework for surgical instrument type segmentation to grasp the category-level relationship and further align the feature distribution. The proposed framework consists of prototypical inner-interaction graph (PI-Graph) and prototypical cross-interaction graph (PC-Graph). In PI-Graph, EM-Grouping module is designed to generate multi-prototypes representing the semantic information adequately. Then, propagation is performed upon these multi-prototypes to communicate semantic information inner each domain. Aiming at narrowing down the domain gaps, the PC-Graph constructs hierarchical graphs upon multi-prototypes and category centers, and conducts dynamic reasoning to exchange the correlated information among two domains. Extensive experiments on the EndoVis Instrument Segmentation 2017 → 2018 scenarios demonstrate the superiority of our SePIG framework compared with state-of-the-art methods. Code is available at https://github.com/CityU-AIM-Group/SePIG.
AB - Surgical instrument segmentation is fundamental for the advanced computer-assisted system. The variability of the surgical scene, a major obstacle in this task, leads to the domain shift problem. Unsupervised domain adaptation (UDA) technique can be employed to solve this problem and adapt the model to various surgical scenarios. However, existing UDA methods ignore the relationship among different categories, hindering the model learning discriminative features from a global view. Additionally, the adversarial strategy utilized in these methods only narrows down the domain gap at the end of the network, leading to the poor feature alignment. To tackle above mentioned problems, we advance a semantic-prototype interaction graph (SePIG) framework for surgical instrument type segmentation to grasp the category-level relationship and further align the feature distribution. The proposed framework consists of prototypical inner-interaction graph (PI-Graph) and prototypical cross-interaction graph (PC-Graph). In PI-Graph, EM-Grouping module is designed to generate multi-prototypes representing the semantic information adequately. Then, propagation is performed upon these multi-prototypes to communicate semantic information inner each domain. Aiming at narrowing down the domain gaps, the PC-Graph constructs hierarchical graphs upon multi-prototypes and category centers, and conducts dynamic reasoning to exchange the correlated information among two domains. Extensive experiments on the EndoVis Instrument Segmentation 2017 → 2018 scenarios demonstrate the superiority of our SePIG framework compared with state-of-the-art methods. Code is available at https://github.com/CityU-AIM-Group/SePIG.
KW - Graph convolution
KW - Surgical instrument type segmentation
KW - Unsupervised domain adaptation
UR - http://www.scopus.com/inward/record.url?scp=85116467143&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-87199-4_26
DO - 10.1007/978-3-030-87199-4_26
M3 - Conference proceeding
AN - SCOPUS:85116467143
SN - 9783030871987
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 272
EP - 281
BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2021
A2 - de Bruijne, Marleen
A2 - Cattin, Philippe C.
A2 - Cotin, Stéphane
A2 - Padoy, Nicolas
A2 - Speidel, Stefanie
A2 - Zheng, Yefeng
A2 - Essert, Caroline
PB - Springer Nature
T2 - 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
Y2 - 27 September 2021 through 1 October 2021
ER -
