Enhancing Deep Spectral Super-resolution from RGB Images by Enforcing the Metameric Constraint

Tarek Stiebel, Philipp Seltsam, Dorit Merhof

2020

Abstract

The task of spectral signal reconstruction from RGB images requires to solve a heavily underconstrained set of equations. In recent work, deep learning has been applied to solve this inherently difficult problem. Based on a given training set of corresponding RGB images and spectral images, a neural network is trained to learn an optimal end-to-end mapping. However, in such an approach no additional knowledge is incorporated into the networks prediction. We propose and analyze methods for incorporating prior knowledge based on the idea, that when reprojecting any reconstructed spectrum into the camera RGB space it must be (ideally) identical to the originally measured camera signal. It is therefore enforced, that every reconstruction is at least a metamer of the ideal spectrum with respect to the observed signal and observer. This is the one major constraint that any reconstruction should fulfil to be physically plausible, but has been neglected so far.

Paper Citation

in Harvard Style

Stiebel T., Seltsam P. and Merhof D. (2020). Enhancing Deep Spectral Super-resolution from RGB Images by Enforcing the Metameric Constraint. In Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 4: VISAPP; ISBN 978-989-758-402-2, SciTePress, pages 57-66. DOI: 10.5220/0008950100570066

in Bibtex Style

@conference{visapp20,
author={Tarek Stiebel and Philipp Seltsam and Dorit Merhof},
title={Enhancing Deep Spectral Super-resolution from RGB Images by Enforcing the Metameric Constraint},
booktitle={Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 4: VISAPP},
year={2020},
pages={57-66},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008950100570066},
isbn={978-989-758-402-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 4: VISAPP
TI - Enhancing Deep Spectral Super-resolution from RGB Images by Enforcing the Metameric Constraint
SN - 978-989-758-402-2
AU - Stiebel T.
AU - Seltsam P.
AU - Merhof D.
PY - 2020
SP - 57
EP - 66
DO - 10.5220/0008950100570066
PB - SciTePress