CAVC: Cosine Attention Video Colorization

Leandro Stival, Ricardo Torres, Ricardo Torres, Helio Pedrini

2024

Abstract

Video colorization is a challenging task, demanding deep learning models to employ diverse abstractions for a comprehensive grasp of the task, ultimately yielding high-quality results. Currently, in example-based colorization approaches, the use of attention processes and convolutional layers has proven to be the most effective method to produce good results. Following this way, in this paper we propose Cosine Attention Video Colorization (CAVC), which is an approach that uses a single attention head with shared weights to produce a refinement of the monochromatic frame, as well as the cosine similarity between this sample and the other channels present in the image. This entire process acts as a pre-processing of the data from our autoencoder, which performs a feature fusion with the latent space extracted from the referent frame, as well as with its histogram. This architecture was trained on the DAVIS, UVO and LDV datasets and achieved superior results compared to state-of-the-art models in terms of FID metric in all the datasets.

Paper Citation

in Harvard Style

Stival L., Torres R. and Pedrini H. (2024). CAVC: Cosine Attention Video Colorization. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 385-392. DOI: 10.5220/0012348500003660

in Bibtex Style

@conference{visapp24,
author={Leandro Stival and Ricardo Torres and Helio Pedrini},
title={CAVC: Cosine Attention Video Colorization},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP},
year={2024},
pages={385-392},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012348500003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP
TI - CAVC: Cosine Attention Video Colorization
SN - 978-989-758-679-8
AU - Stival L.
AU - Torres R.
AU - Pedrini H.
PY - 2024
SP - 385
EP - 392
DO - 10.5220/0012348500003660
PB - SciTePress