TY - JOUR
T1 - Super-Resolution Enhanced Medical Image Diagnosis with Sample Affinity Interaction
AU - Chen, Zhen
AU - Guo, Xiaoqing
AU - Woo, Peter Y.M.
AU - Yuan, Yixuan
N1 - Funding Information:
This work was supported by the Hong Kong Research Grants Council (RGC) Early Career Scheme under Grant 21207420 (CityU 9048179), by the National Natural Science Foundation of China under Grant 62001410, and by the Shenzhen-Hong Kong Innovation Circle Category D Project SGDX2019081623300177 (CityU 9240008).
Publisher Copyright:
© 2021 IEEE.
PY - 2021/5
Y1 - 2021/5
N2 - The degradation in image resolution harms the performance of medical image diagnosis. By inferring high-frequency details from low-resolution (LR) images, super-resolution (SR) techniques can introduce additional knowledge and assist high-level tasks. In this paper, we propose a SR enhanced diagnosis framework, consisting of an efficient SR network and a diagnosis network. Specifically, a Multi-scale Refined Context Network (MRC-Net) with Refined Context Fusion (RCF) is devised to leverage global and local features for SR tasks. Instead of learning from scratch, we first develop a recursive MRC-Net with temporal context, and then propose a recursion distillation scheme to enhance the performance of MRC-Net from the knowledge of the recursive one and reduce the computational cost. The diagnosis network jointly utilizes the reliable original images and more informative SR images by two branches, with the proposed Sample Affinity Interaction (SAI) blocks at different stages to effectively extract and integrate discriminative features towards diagnosis. Moreover, two novel constraints, sample affinity consistency and sample affinity regularization, are devised to refine the features and achieve the mutual promotion of these two branches. Extensive experiments of synthetic and real LR cases are conducted on wireless capsule endoscopy and histopathology images, verifying that our proposed method is significantly effective for medical image diagnosis.
AB - The degradation in image resolution harms the performance of medical image diagnosis. By inferring high-frequency details from low-resolution (LR) images, super-resolution (SR) techniques can introduce additional knowledge and assist high-level tasks. In this paper, we propose a SR enhanced diagnosis framework, consisting of an efficient SR network and a diagnosis network. Specifically, a Multi-scale Refined Context Network (MRC-Net) with Refined Context Fusion (RCF) is devised to leverage global and local features for SR tasks. Instead of learning from scratch, we first develop a recursive MRC-Net with temporal context, and then propose a recursion distillation scheme to enhance the performance of MRC-Net from the knowledge of the recursive one and reduce the computational cost. The diagnosis network jointly utilizes the reliable original images and more informative SR images by two branches, with the proposed Sample Affinity Interaction (SAI) blocks at different stages to effectively extract and integrate discriminative features towards diagnosis. Moreover, two novel constraints, sample affinity consistency and sample affinity regularization, are devised to refine the features and achieve the mutual promotion of these two branches. Extensive experiments of synthetic and real LR cases are conducted on wireless capsule endoscopy and histopathology images, verifying that our proposed method is significantly effective for medical image diagnosis.
KW - Medical image diagnosis
KW - semantic consistency
KW - super resolution
UR - http://www.scopus.com/inward/record.url?scp=85100455903&partnerID=8YFLogxK
U2 - 10.1109/TMI.2021.3055290
DO - 10.1109/TMI.2021.3055290
M3 - Journal article
C2 - 33507866
AN - SCOPUS:85100455903
SN - 0278-0062
VL - 40
SP - 1377
EP - 1389
JO - IEEE Transactions on Medical Imaging
JF - IEEE Transactions on Medical Imaging
IS - 5
M1 - 9339901
ER -