Latent Video Transformer

Ruslan Rakhimov, Denis Volkhonskiy, Alexey Artemov, Denis Zorin, Denis Zorin, Evgeny Burnaev

2021

Abstract

The video generation task can be formulated as a prediction of future video frames given some past frames. Recent generative models for videos face the problem of high computational requirements. Some models require up to 512 Tensor Processing Units for parallel training. In this work, we address this problem via modeling the dynamics in a latent space. After the transformation of frames into the latent space, our model predicts latent representation for the next frames in an autoregressive manner. We demonstrate the performance of our approach on BAIR Robot Pushing and Kinetics-600 datasets. The approach tends to reduce requirements to 8 Graphical Processing Units for training the models while maintaining comparable generation quality.

Paper Citation

in Harvard Style

Rakhimov R., Volkhonskiy D., Artemov A., Zorin D. and Burnaev E. (2021). Latent Video Transformer. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 101-112. DOI: 10.5220/0010241801010112

in Bibtex Style

@conference{visapp21,
author={Ruslan Rakhimov and Denis Volkhonskiy and Alexey Artemov and Denis Zorin and Evgeny Burnaev},
title={Latent Video Transformer},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP},
year={2021},
pages={101-112},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010241801010112},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP
TI - Latent Video Transformer
SN - 978-989-758-488-6
AU - Rakhimov R.
AU - Volkhonskiy D.
AU - Artemov A.
AU - Zorin D.
AU - Burnaev E.
PY - 2021
SP - 101
EP - 112
DO - 10.5220/0010241801010112
PB - SciTePress