Specularity, Shadow, and Occlusion Removal from Image Sequences using Deep Residual Sets

Monika Kwiatkowski, Olaf Hellwich

2022

Abstract

When taking images of planar objects, the images are often subject to unwanted artifacts such as specularities, shadows, and occlusions. While there are some methods that specialize in the removal of each type of artifact individually, we offer a generalized solution. We implement an end-to-end deep learning approach that removes artifacts from a series of images using a fully convolutional residual architecture and Deep Sets. Our architecture can be used as general approach for many image restoration tasks and is robust to varying sequence lengths and varying image resolutions. Furthermore, it enforces permutation invariance on the input sequence. The architecture is optimized to process high resolution images. We also provide a simple online algorithm that allows the processing of arbitrarily long image sequences without increasing the memory consumption. We created a synthetic dataset as an initial proof-of-concept. Additionally, we created a smaller dataset of real image sequences. In order to overcome the data scarcity of our real dataset, we use the synthetic data for pre-training our model. Our evaluations show that our model outperforms many state of the art methods that are used in related problems such as background subtraction and intrinsic image decomposition.

Paper Citation

in Harvard Style

Kwiatkowski M. and Hellwich O. (2022). Specularity, Shadow, and Occlusion Removal from Image Sequences using Deep Residual Sets. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 118-125. DOI: 10.5220/0010822300003124

in Bibtex Style

@conference{visapp22,
author={Monika Kwiatkowski and Olaf Hellwich},
title={Specularity, Shadow, and Occlusion Removal from Image Sequences using Deep Residual Sets},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP},
year={2022},
pages={118-125},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010822300003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP
TI - Specularity, Shadow, and Occlusion Removal from Image Sequences using Deep Residual Sets
SN - 978-989-758-555-5
AU - Kwiatkowski M.
AU - Hellwich O.
PY - 2022
SP - 118
EP - 125
DO - 10.5220/0010822300003124
PB - SciTePress