TY - JOUR
T1 - A Self-Supervised Deep Denoiser for Hyperspectral and Multispectral Image Fusion
AU - Wang, Zhicheng
AU - Ng, Michael K.
AU - Michalski, Joseph
AU - Zhuang, Lina
N1 - Funding information:
The work of Michael K. Ng was supported in part by the Hong Kong Research Grants Council (HKRGC) General Research Fund (GRF) under Grant 12300519, Grant 17201020, and Grant 17300021; in part by the HKRGC Collaborative Research Fund (CRF) under Grant C1013-21GF and Grant C7004-21GF; and in part by the Joint National Natural Science Foundation of China (NSFC) and RGC under Grant N-HKU769/21. The work of Joseph Michalski was supported by the HKRGC CRF under Grant C7004-21G. The work of Lina Zhuang was supported by the NSFC under Grant 42001287. (Corresponding author: Lina Zhuang.)
Publisher Copyright:
© 1980-2012 IEEE.
PY - 2023/8/10
Y1 - 2023/8/10
N2 - The plug-and-play (PnP) technique enables us to plug image priors into an alternating direction method of multipliers (ADMM) framework for solving a regularized optimization problem. Deep image priors have shown their flexibility and robustness in solving several image inverse problems. Hyperspectral image (HSI) super-resolution problem is an ill-posed inverse problem that aims to obtain a high-resolution HSI (HR-HSI) by combining the information of low-resolution HSI (LR-HSI) and HR multispectral image (HR-MSI) simultaneously. This article proposes an HSI and MSI fusion framework termed E2E-Fusion, plugged with a self-supervised deep learning prior called Eigenimage2Eigenimage. First, the spectral low-rank structure of HSIs is exploited via subspace representations of spectra vectors. Meanwhile, benefiting from the high quality of the first eigenimage (i.e., representation coefficients), we design a self-supervised deep eigenimage guidance network image prior, E2E. By using the PnP technique, we plugged the E2E prior into the ADMM fusion framework to update the optimal objective function iteratively. The numerical experimental results both on the simulated datasets and real datasets demonstrate that the proposed method performs better than state-of-the-art fusion methods.
AB - The plug-and-play (PnP) technique enables us to plug image priors into an alternating direction method of multipliers (ADMM) framework for solving a regularized optimization problem. Deep image priors have shown their flexibility and robustness in solving several image inverse problems. Hyperspectral image (HSI) super-resolution problem is an ill-posed inverse problem that aims to obtain a high-resolution HSI (HR-HSI) by combining the information of low-resolution HSI (LR-HSI) and HR multispectral image (HR-MSI) simultaneously. This article proposes an HSI and MSI fusion framework termed E2E-Fusion, plugged with a self-supervised deep learning prior called Eigenimage2Eigenimage. First, the spectral low-rank structure of HSIs is exploited via subspace representations of spectra vectors. Meanwhile, benefiting from the high quality of the first eigenimage (i.e., representation coefficients), we design a self-supervised deep eigenimage guidance network image prior, E2E. By using the PnP technique, we plugged the E2E prior into the ADMM fusion framework to update the optimal objective function iteratively. The numerical experimental results both on the simulated datasets and real datasets demonstrate that the proposed method performs better than state-of-the-art fusion methods.
KW - Deep plug-and-play (PnP) prior
KW - hyperspectral image (HSI)
KW - image fusion
KW - self-supervised learning
UR - http://www.scopus.com/inward/record.url?scp=85167803779&partnerID=8YFLogxK
U2 - 10.1109/TGRS.2023.3303921
DO - 10.1109/TGRS.2023.3303921
M3 - Journal article
AN - SCOPUS:85167803779
SN - 0196-2892
VL - 61
JO - IEEE Transactions on Geoscience and Remote Sensing
JF - IEEE Transactions on Geoscience and Remote Sensing
M1 - 5520414
ER -