Beyond Variational Models and Self-Similarity in Super-Resolution: Unfolding Models and Multi-Head Attention

Ivan Pereira-Sánchez, Ivan Pereira-Sánchez, Eloi Sans, Julia Navarro, Julia Navarro, Joan Duran, Joan Duran

2024

Abstract

Classical variational methods for solving image processing problems are more interpretable and flexible than pure deep learning approaches, but their performance is limited by the use of rigid priors. Deep unfolding networks combine the strengths of both by unfolding the steps of the optimization algorithm used to estimate the minimizer of an energy functional into a deep learning framework. In this paper, we propose an unfolding approach to extend a variational model exploiting self-similarity of natural images in the data fidelity term for single-image super-resolution. The proximal, downsampling and upsampling operators are written in terms of a neural network specifically designed for each purpose. Moreover, we include a new multi-head attention module to replace the nonlocal term in the original formulation. A comprehensive evaluation covering a wide range of sampling factors and noise realizations proves the benefits of the proposed unfolding techniques. The model shows to better preserve image geometry while being robust to noise.

Paper Citation

in Harvard Style

Pereira-Sánchez I., Sans E., Navarro J. and Duran J. (2024). Beyond Variational Models and Self-Similarity in Super-Resolution: Unfolding Models and Multi-Head Attention. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 190-198. DOI: 10.5220/0012395400003660

in Bibtex Style

@conference{visapp24,
author={Ivan Pereira-Sánchez and Eloi Sans and Julia Navarro and Joan Duran},
title={Beyond Variational Models and Self-Similarity in Super-Resolution: Unfolding Models and Multi-Head Attention},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP},
year={2024},
pages={190-198},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012395400003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP
TI - Beyond Variational Models and Self-Similarity in Super-Resolution: Unfolding Models and Multi-Head Attention
SN - 978-989-758-679-8
AU - Pereira-Sánchez I.
AU - Sans E.
AU - Navarro J.
AU - Duran J.
PY - 2024
SP - 190
EP - 198
DO - 10.5220/0012395400003660
PB - SciTePress