A Search Engine for Retrieval and Inspection of Events with 48 Human Actions in Realistic Videos

G. J. Burghouts, L. de Penning, M. Kruithof, P. Hanckmann, J-M Ten Hove, S. Landsmeer, S. P. van den Broek, R. den Hollander, C. van Leeuwen, S. Korzec, H. Bouma, K. Schutte

2013

Abstract

The contribution of this paper is a search engine that recognizes and describes 48 human actions in realistic videos. The core algorithms have been published recently, from the early visual processing (Bouma, 2012), discriminative recognition (Burghouts, 2012) and textual description (Hanckmann, 2012) of 48 human actions. We summarize the key algorithms and specify their performance. The novelty of this paper is that we integrate these algorithms into a search engine. In this paper, we add an algorithm that finds the relevant spatio-temporal regions in the video, which is the input for the early visual processing. As a result, meta-data is produced by the recognition and description algorithms. The meta-data is filtered by a novel algorithm that selects only the most informative parts of the video. We demonstrate the power of our search engine by retrieving relevant parts of the video based on three different queries. The search results indicate where specific events occurred, and which actors and objects were involved. We show that events can be successfully retrieved and inspected by usage of the proposed search engine.

References

  1. Bouma, H., Hanckmann, P., Marck, J-W., de Penning, L., den Hollander, R., ten Hove, J-M., van den Broek, S. P., Schutte, K., Burghouts, G. J., 2012. Automatic human action recognition in a scene from visual inputs, SPIE.
  2. Breiman, L., 2001, Random forests, Machine Learning.
  3. Burghouts, G. J., Schutte, K., 2012, Correlations between 48 human actions improve their detection, ICPR.
  4. Chang, C.-C., Lin, C.-J., 2001, LIBSVM: a library for support vector machines, software available at http:// www.csie.ntu.edu.tw/cjlin/libsvm.
  5. Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D., 2010, Object detection with discriminatively trained part based models, PAMI.
  6. Hanckmann, P., Schutte, K., Burghouts, G. J., 2012, Automated Textual Descriptions for a Wide Range of Video Events with 48 Human Actions, ECCV.
  7. Laptev, I., 2005, On Space-Time Interest Points, IJCV.
  8. Moosmann, F., Triggs, B., Jurie, F., 2006, Randomized Clustering Forests for Building Fast and Discriminative Visual Vocabularies, NIPS.
  9. Ramanan, D., 2006, Learning to parse images of articulated bodies”, NIPS.
  10. van de Sande, K. E. A., Gevers, T., Snoek, C. G. M., 2010, Evaluating Color Descriptors for Object and Scene Recognition, PAMI.
  11. Stauffer, C., Grimson, W., 1999, Adaptive background mixture models for real-time tracking, CVPR.
  12. Withagen, P. J., Schutte, K., Groen, F. C. A., 2004, Probabilistic classification between foreground objects and background, ICPR
  13. Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C., 2007, Local features and kernels for classification of texture and object categories: A comprehensive study, IJCV.
Download


Paper Citation


in Harvard Style

J. Burghouts G., de Penning L., Kruithof M., Hanckmann P., Ten Hove J., Landsmeer S., P. van den Broek S., den Hollander R., van Leeuwen C., Korzec S., Bouma H. and Schutte K. (2013). A Search Engine for Retrieval and Inspection of Events with 48 Human Actions in Realistic Videos . In Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-8565-41-9, pages 413-418. DOI: 10.5220/0004196904130418


in Bibtex Style

@conference{icpram13,
author={G. J. Burghouts and L. de Penning and M. Kruithof and P. Hanckmann and J-M Ten Hove and S. Landsmeer and S. P. van den Broek and R. den Hollander and C. van Leeuwen and S. Korzec and H. Bouma and K. Schutte},
title={A Search Engine for Retrieval and Inspection of Events with 48 Human Actions in Realistic Videos},
booktitle={Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2013},
pages={413-418},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004196904130418},
isbn={978-989-8565-41-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - A Search Engine for Retrieval and Inspection of Events with 48 Human Actions in Realistic Videos
SN - 978-989-8565-41-9
AU - J. Burghouts G.
AU - de Penning L.
AU - Kruithof M.
AU - Hanckmann P.
AU - Ten Hove J.
AU - Landsmeer S.
AU - P. van den Broek S.
AU - den Hollander R.
AU - van Leeuwen C.
AU - Korzec S.
AU - Bouma H.
AU - Schutte K.
PY - 2013
SP - 413
EP - 418
DO - 10.5220/0004196904130418