Video Summarization through Total Variation, Deep Semi-supervised Autoencoder and Clustering Algorithms

Eden Pereira da Silva, Eliaquim Monteiro Ramos, Leandro Tavares da Silva, Leandro Tavares da Silva, Jaime S. Cardoso, Gilson A. Giraldi

2020

Abstract

Video summarization is an important tool considering the amount of data to analyze. Techniques in this area aim to yield synthetic and useful visual abstraction of the videos contents. Hence, in this paper we present a new summarization algorithm, based on image features, which is composed by the following steps: (i) Query video processing using cosine similarity metric and total variation smoothing to identify classes in the query sequence; (ii) With this result, build a labeled training set of frames; (iii) Generate the unlabeled training set composed by samples of the video database; (iv) Training a deep semi-supervised autoencoder; (v) Compute the K-means for each video separately, in the encoder space, with the number of clusters set as a percentage of the video size; (vi) Select key-frames in the K-means clusters to define the summaries. In this methodology, the query video is used to incorporate prior knowledge in the whole process through the obtained labeled data. The step (iii) aims to include unknown patterns useful for the summarization process. We evaluate the methodology using some videos from OPV video database. We compare the performance of our algorithm with the VSum. The results indicate that the pipeline was well succeed in the summarization presenting a F-score value superior to VSum.

Paper Citation

in Harvard Style

Pereira da Silva E., Ramos E., Tavares da Silva L., Cardoso J. and Giraldi G. (2020). Video Summarization through Total Variation, Deep Semi-supervised Autoencoder and Clustering Algorithms. In Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 4: VISAPP; ISBN 978-989-758-402-2, SciTePress, pages 315-322. DOI: 10.5220/0008969303150322

in Bibtex Style

@conference{visapp20,
author={Eden Pereira da Silva and Eliaquim Monteiro Ramos and Leandro Tavares da Silva and Jaime S. Cardoso and Gilson A. Giraldi},
title={Video Summarization through Total Variation, Deep Semi-supervised Autoencoder and Clustering Algorithms},
booktitle={Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 4: VISAPP},
year={2020},
pages={315-322},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008969303150322},
isbn={978-989-758-402-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 4: VISAPP
TI - Video Summarization through Total Variation, Deep Semi-supervised Autoencoder and Clustering Algorithms
SN - 978-989-758-402-2
AU - Pereira da Silva E.
AU - Ramos E.
AU - Tavares da Silva L.
AU - Cardoso J.
AU - Giraldi G.
PY - 2020
SP - 315
EP - 322
DO - 10.5220/0008969303150322
PB - SciTePress