Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Jing Jin; Junhui Hou; Jie Chen; Henry Yeung; Sam Kwong

doi:10.1109/ICDSP.2018.8631859

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Jing Jin
, Junhui Hou
, Jie Chen
, Henry Yeung
, Sam Kwong

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

5 Citations (Scopus)

Abstract

Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

Original language	English
Title of host publication	2018 IEEE 23rd International Conference on Digital Signal Processing, DSP 2018
Publisher	IEEE
Pages	1-5
Number of pages	5
ISBN (Electronic)	9781538668115, 9781538668108
ISBN (Print)	9781538668122
DOIs	https://doi.org/10.1109/ICDSP.2018.8631859
Publication status	Published - 19 Nov 2018
Event	23rd IEEE International Conference on Digital Signal Processing, DSP 2018 - Shanghai, China Duration: 19 Nov 2018 → 21 Nov 2018 https://doi.org/10.1109/DSP43099.2018 (Conference proceedings)

Publication series

Name	International Conference on Digital Signal Processing, DSP
Volume	2018-November
ISSN (Print)	1546-1874
ISSN (Electronic)	2165-3577

Conference

Conference	23rd IEEE International Conference on Digital Signal Processing, DSP 2018
Abbreviated title	DSP 2018
Country/Territory	China
City	Shanghai
Period	19/11/18 → 21/11/18
Internet address	https://doi.org/10.1109/DSP43099.2018 (Conference proceedings)

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 9 Industry, Innovation, and Infrastructure

User-Defined Keywords

deep learning
Light field
super-resolution

Access to Document

10.1109/ICDSP.2018.8631859

Cite this

Jin, J., Hou, J., Chen, J., Yeung, H., & Kwong, S. (2018). Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image. In 2018 IEEE 23rd International Conference on Digital Signal Processing, DSP 2018 (pp. 1-5). (International Conference on Digital Signal Processing, DSP; Vol. 2018-November). IEEE. https://doi.org/10.1109/ICDSP.2018.8631859

@inproceedings{aba3e65edde1413ab925347be1e5266b,

title = "Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image",

abstract = "Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.",

keywords = "deep learning, Light field, super-resolution",

author = "Jing Jin and Junhui Hou and Jie Chen and Henry Yeung and Sam Kwong",

note = "This work was supported in part by the CityU Start-up Grant for New Faculty under Grant 7200537/CS and in part by the Hong Kong RGC Early Career Scheme Funds 9048123 (CityU 21211518). Publisher Copyright: {\textcopyright} 2018 IEEE.; 23rd IEEE International Conference on Digital Signal Processing, DSP 2018, DSP 2018 ; Conference date: 19-11-2018 Through 21-11-2018",

year = "2018",

month = nov,

day = "19",

doi = "10.1109/ICDSP.2018.8631859",

language = "English",

isbn = "9781538668122",

series = "International Conference on Digital Signal Processing, DSP",

publisher = "IEEE",

pages = "1--5",

booktitle = "2018 IEEE 23rd International Conference on Digital Signal Processing, DSP 2018",

address = "United States",

url = "https://doi.org/10.1109/DSP43099.2018",

}

Jin, J, Hou, J, Chen, J, Yeung, H & Kwong, S 2018, Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image. in 2018 IEEE 23rd International Conference on Digital Signal Processing, DSP 2018. International Conference on Digital Signal Processing, DSP, vol. 2018-November, IEEE, pp. 1-5, 23rd IEEE International Conference on Digital Signal Processing, DSP 2018, Shanghai, China, 19/11/18. https://doi.org/10.1109/ICDSP.2018.8631859

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image. / Jin, Jing; Hou, Junhui; Chen, Jie et al.
2018 IEEE 23rd International Conference on Digital Signal Processing, DSP 2018. IEEE, 2018. p. 1-5 (International Conference on Digital Signal Processing, DSP; Vol. 2018-November).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

AU - Jin, Jing

AU - Hou, Junhui

AU - Chen, Jie

AU - Yeung, Henry

AU - Kwong, Sam

N1 - This work was supported in part by the CityU Start-up Grant for New Faculty under Grant 7200537/CS and in part by the Hong Kong RGC Early Career Scheme Funds 9048123 (CityU 21211518). Publisher Copyright: © 2018 IEEE.

PY - 2018/11/19

Y1 - 2018/11/19

N2 - Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

AB - Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

KW - deep learning

KW - Light field

KW - super-resolution

UR - https://www.scopus.com/pages/publications/85062776177

U2 - 10.1109/ICDSP.2018.8631859

DO - 10.1109/ICDSP.2018.8631859

M3 - Conference proceeding

AN - SCOPUS:85062776177

SN - 9781538668122

T3 - International Conference on Digital Signal Processing, DSP

SP - 1

EP - 5

BT - 2018 IEEE 23rd International Conference on Digital Signal Processing, DSP 2018

PB - IEEE

T2 - 23rd IEEE International Conference on Digital Signal Processing, DSP 2018

Y2 - 19 November 2018 through 21 November 2018

ER -

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Abstract

Publication series

Conference

UN SDGs

User-Defined Keywords

Access to Document

Other files and links

Fingerprint

Cite this