Keyword based Keyframe Extraction in Online Video Collections

Edoardo Ardizzone, Marco La Cascia, Giuseppe Mazzola

2015

Abstract

Keyframe extraction methods aim to find in a video sequence the most significant frames, according to specific criteria. In this paper we propose a new method to search, in a video database, for frames that are related to a given keyword, and to extract the best ones, according to a proposed quality factor. We first exploit a speech to text algorithm to extract automatic captions from all the video in a specific domain database. Then we select only those sequences (clips), whose captions include a given keyword, thus discarding a lot of information that is useless for our purposes. Each retrieved clip is then divided into shots, using a video segmentation method, that is based on the SURF descriptors and keypoints. The sentence of the caption is projected onto the segmented clip, and we select the shot that includes the input keyword. The selected shot is further inspected to find good quality and stable parts, and the frame which maximizes a quality metric is selected as the best and the most significant frame. We compare the proposed algorithm with another keyframe extraction method based on local features, in terms of Significance and Quality.

References

  1. Bay, H., Tuytelaars, T., Van Gool, L. 2006. Surf: Speeded Up Robust Features. In Proceedings of European Conference on Computer Vision ECCV, 404-417.
  2. Chan, P. P. K., et al., 2011. A Novel Method to Reduce Redundancy in Adaptive Threshold Clustering Keyframe Extraction Systems. Proceedings of the 2011 International Conference on Machine Learning and Cybernetics, Guilin, vol. 4, pp.1637-1642.
  3. Chasanis, V., Likas, A., and Galatsanos, N., 2007. Scene Detection in Videos Using Shot Clustering and Symbolic Sequence Segmentation. IEEE 9th Workshop on Multimedia Signal Processing, pp. 187- 190.
  4. D'Avila, S. E., et al., 2011. VSUMM: a Mechanism Designed to Produce Static Video Summaries and a Novel Evaluation Method. Pattern Recognition Letters 32 (1) () 56-68.Guan, G., Wang, Z., Lu, S., Deng, J. D., and, D. D., 2013. Keypoint-Based Keyframe Selection. Circuits and Systems for Video Technology, IEEE Transactions on. 23(4), 729-734.
  5. Hu, W., et al.., 2011. A Survey on Visual Content-Based Video Indexing and Retrieval. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, vol.41, no.6, pp.797-819.
  6. Jiang, X. H., Sun, T. F., Li, J. H., and Chen, X., 2008. A Novel Shot Edge Detection Algorithm Based on ChiSquare Histogram and Macro-Block Statistics. Proc. of International Symposium on Information Science and Engineering, vol. 2, pp. 604-607.
  7. Liu, G., Wen, X., Zheng, W., and He, P., 2009. Shot Boundary Detection and Keyframe Extraction Based on Scale Invariant Feature Transform. In Proceedings of the 2009 Eigth IEEE/ACIS International Conference on Computer and Information Science (ICIS 7809), pp. 1126-1130.
  8. Sysomos, 2010. http://www.sysomos.com/reports/ YouTube.
  9. Wang, J. Y., and Luo. W., 2008. A Self-Adapting DualThreshold Method for Video Shot Transition Detection. IEEE International Conference on Networking, Sensing and Control, pp. 704-707.
  10. Wei, J., Cotton, C., and Loui, A.C., 2011. Automatic consumer video summarization by audio and visual analysis. Multimedia and Expo (ICME), 2011 IEEE International Conference on, pp (1-6).
  11. Yue, G., Wei-Bo, W., Jun-Hai, Y., 2008. A Video Summarization Tool using Two-Level Redundancy Detection for Personal Video Recorders. IEEE Transactions on Consumer Electronics, 54, 2, 521- 526.
Download


Paper Citation


in Harvard Style

Ardizzone E., La Cascia M. and Mazzola G. (2015). Keyword based Keyframe Extraction in Online Video Collections . In Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 2: ICPRAM, ISBN 978-989-758-077-2, pages 170-177. DOI: 10.5220/0005190001700177


in Bibtex Style

@conference{icpram15,
author={Edoardo Ardizzone and Marco La Cascia and Giuseppe Mazzola},
title={Keyword based Keyframe Extraction in Online Video Collections},
booktitle={Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 2: ICPRAM,},
year={2015},
pages={170-177},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005190001700177},
isbn={978-989-758-077-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 2: ICPRAM,
TI - Keyword based Keyframe Extraction in Online Video Collections
SN - 978-989-758-077-2
AU - Ardizzone E.
AU - La Cascia M.
AU - Mazzola G.
PY - 2015
SP - 170
EP - 177
DO - 10.5220/0005190001700177