Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation

Xingxing Yang; Jie Chen; Zaifeng Yang

doi:10.1109/VCIP59821.2023.10402624

Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation

Xingxing Yang
, Jie Chen^*
, Zaifeng Yang

^*Corresponding author for this work

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

5 Citations (Scopus)

Abstract

NIR-to-RGB spectral domain translation is a challenging task due to the mapping ambiguities and existing methods show limited learning capacities. To address these challenges, we propose to colorize NIR images via a multi-scale progressive feature embedding network (MPFNet), with the guidance of grayscale image colorization. Specifically, we first introduce a domain translation module that translates NIR source images into the grayscale target domain. By incorporating a progressive training strategy, the statistical and semantic knowledge from both task domains are efficiently aligned with a series of pixel-/feature-level consistency constraints. Besides, a multi-scale progressive feature embedding network is designed to improve learning capabilities. Experiments show that our MPFNet outperforms state-of-the-art counterparts by 2.55dB in the NIR-to-RGB spectral domain translation task in terms of PSNR.

Original language	English
Title of host publication	2023 IEEE International Conference on Visual Communications and Image Processing (VCIP)
Publisher	IEEE
Number of pages	5
ISBN (Electronic)	9798350359855
ISBN (Print)	9798350359862
DOIs	https://doi.org/10.1109/VCIP59821.2023.10402624
Publication status	Published - Dec 2023
Event	2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023 - Jeju, Korea, Republic of Duration: 4 Dec 2023 → 7 Dec 2023 https://ieeexplore.ieee.org/xpl/conhome/10402600/proceeding

Publication series

Name	IEEE Visual Communications and Image Processing (VCIP)
Publisher	IEEE
ISSN (Print)	1018-8770
ISSN (Electronic)	2642-9357

Conference

Conference	2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023
Country/Territory	Korea, Republic of
City	Jeju
Period	4/12/23 → 7/12/23
Internet address	https://ieeexplore.ieee.org/xpl/conhome/10402600/proceeding

User-Defined Keywords

Near-Infrared image colorization
domain adaptation
Generative Adversarial Network
attention mechanism

Access to Document

10.1109/VCIP59821.2023.10402624

Cite this

@inproceedings{c1448b76947a47b598691ddd37bb4fab,

title = "Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation",

abstract = "NIR-to-RGB spectral domain translation is a challenging task due to the mapping ambiguities and existing methods show limited learning capacities. To address these challenges, we propose to colorize NIR images via a multi-scale progressive feature embedding network (MPFNet), with the guidance of grayscale image colorization. Specifically, we first introduce a domain translation module that translates NIR source images into the grayscale target domain. By incorporating a progressive training strategy, the statistical and semantic knowledge from both task domains are efficiently aligned with a series of pixel-/feature-level consistency constraints. Besides, a multi-scale progressive feature embedding network is designed to improve learning capabilities. Experiments show that our MPFNet outperforms state-of-the-art counterparts by 2.55dB in the NIR-to-RGB spectral domain translation task in terms of PSNR.",

keywords = "Near-Infrared image colorization, domain adaptation, Generative Adversarial Network, attention mechanism",

author = "Xingxing Yang and Jie Chen and Zaifeng Yang",

note = "This research is supported by A*STAR C222812026; 2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023 ; Conference date: 04-12-2023 Through 07-12-2023",

year = "2023",

month = dec,

doi = "10.1109/VCIP59821.2023.10402624",

language = "English",

isbn = "9798350359862",

series = "IEEE Visual Communications and Image Processing (VCIP)",

publisher = "IEEE",

booktitle = "2023 IEEE International Conference on Visual Communications and Image Processing (VCIP)",

address = "United States",

url = "https://ieeexplore.ieee.org/xpl/conhome/10402600/proceeding",

}

Yang, X, Chen, J & Yang, Z 2023, Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation. in 2023 IEEE International Conference on Visual Communications and Image Processing (VCIP). IEEE Visual Communications and Image Processing (VCIP), IEEE, 2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023, Jeju, Korea, Republic of, 4/12/23. https://doi.org/10.1109/VCIP59821.2023.10402624

Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation. / Yang, Xingxing; Chen, Jie; Yang, Zaifeng.
2023 IEEE International Conference on Visual Communications and Image Processing (VCIP). IEEE, 2023. (IEEE Visual Communications and Image Processing (VCIP)).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation

AU - Yang, Xingxing

AU - Chen, Jie

AU - Yang, Zaifeng

N1 - This research is supported by A*STAR C222812026

PY - 2023/12

Y1 - 2023/12

N2 - NIR-to-RGB spectral domain translation is a challenging task due to the mapping ambiguities and existing methods show limited learning capacities. To address these challenges, we propose to colorize NIR images via a multi-scale progressive feature embedding network (MPFNet), with the guidance of grayscale image colorization. Specifically, we first introduce a domain translation module that translates NIR source images into the grayscale target domain. By incorporating a progressive training strategy, the statistical and semantic knowledge from both task domains are efficiently aligned with a series of pixel-/feature-level consistency constraints. Besides, a multi-scale progressive feature embedding network is designed to improve learning capabilities. Experiments show that our MPFNet outperforms state-of-the-art counterparts by 2.55dB in the NIR-to-RGB spectral domain translation task in terms of PSNR.

AB - NIR-to-RGB spectral domain translation is a challenging task due to the mapping ambiguities and existing methods show limited learning capacities. To address these challenges, we propose to colorize NIR images via a multi-scale progressive feature embedding network (MPFNet), with the guidance of grayscale image colorization. Specifically, we first introduce a domain translation module that translates NIR source images into the grayscale target domain. By incorporating a progressive training strategy, the statistical and semantic knowledge from both task domains are efficiently aligned with a series of pixel-/feature-level consistency constraints. Besides, a multi-scale progressive feature embedding network is designed to improve learning capabilities. Experiments show that our MPFNet outperforms state-of-the-art counterparts by 2.55dB in the NIR-to-RGB spectral domain translation task in terms of PSNR.

KW - Near-Infrared image colorization

KW - domain adaptation

KW - Generative Adversarial Network

KW - attention mechanism

U2 - 10.1109/VCIP59821.2023.10402624

DO - 10.1109/VCIP59821.2023.10402624

M3 - Conference proceeding

SN - 9798350359862

T3 - IEEE Visual Communications and Image Processing (VCIP)

BT - 2023 IEEE International Conference on Visual Communications and Image Processing (VCIP)

PB - IEEE

T2 - 2023 IEEE International Conference on Visual Communications and Image Processing, VCIP 2023

Y2 - 4 December 2023 through 7 December 2023

ER -

Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation

Abstract

Publication series

Conference

User-Defined Keywords

Access to Document

Fingerprint

Cite this