CAPTURING THE HUMAN ACTION SEMANTICS USING A QUERY-BY-EXAMPLE

Anna Montesanto, Paola Baldassarri, A. F. Dragoni, G. Vallesi, P. Puliti

Abstract

The paper describes a method for extracting human action semantics in video’s using queries-by-example. Here we consider the indexing and the matching problems of content-based human motion data retrieval. The query formulation is based on trajectories that may be easily built or extracted by following relevant points on a video, by a novice user too. The so realized trajectories contain high value of action semantics. The semantic schema is built by splitting a trajectory in time ordered sub-sequences that contain the features of extracted points. This kind of semantic representation allows reducing the search space dimensionality and, being human-oriented, allows a selective recognition of actions that are very similar among them. A neural network system analyzes the video semantic similarity, using a two-layer architecture of multilayer perceptrons, which is able to learn the semantic schema of the actions and to recognize them.

References

  1. Bobick A. and Davis J., 2001 “The Recognition of Human Movement Using Temporal Templates,” IEEE Transactions on Pattern Recognition and Machine Intelligent, vol.23, pp. 257-267.
  2. Bobick A.F. and Wilson A.D., 1997 “A State-Based Technique to the Representation and Recognition of Gesture”, IEEE Transactions on Pattern Analysis and Machine Intelligent, Vol.19, pp.1325-1337.
  3. Bouget J.Y., 1999 “Pyramidal Implementation of the Lucas Kanade Feature Tracker. Description of the Algorithm”, Intel Corporation, internal report.
  4. Brand M. and Kettnaker V., 2000 “Discovery and Segmentation of Activities in Video,” IEEE Transactions on Pattern Analysis and Machine Intelligent, Vol.22, pp.844-851.
  5. Hu W., Tan T.,, Wang L., and Maybank S., 2004 “A Survey on Visual Surveillance of Object Motion and Behaviors” IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, Vol. 34, No. 3.
  6. Johansson G., 1973 “Visual Perception of Biological Motion and Model of its Analysis”, Perception and Psychophysics, Vol.14, pp.201-211.
  7. Kang H., Lee C.-W., and Jung K., 2004 “RecognitionBased Gesture Spotting in Video Games”, Pattern Recognition Letters, Vol.25, No. 15, pp. 1701-1714.
  8. Marchand-Maillet S., 2000 “Content-based Video Retrieval: An overview”, Technical Report Vision.
  9. Meier U., Stiefelhagen R., Yang J., and Waibel A., 2000 “Toward unrestricted lip reading,” International Journal on Pattern Recognition and Artificial Intelligent, Vol.14, no.5, pp.571-585.
  10. Oliver N. M., Rosario B., and Pentland A. P., 2000 “A Bayesian Computer Vision System for Modeling Human Interactions,” IEEE Transactions on Pattern Analysis and Machine Intelligent, Vol.22, pp.831-843.
  11. Owens J. and Hunter A., 2000 “Application of the SelfOrganizing Map to Trajectory Classification,” Proceedings of IEEE International Workshop on Visual Surveillance, pp.77-83.
  12. Rumelhart D., Ortony A., 1964 “The Representation of Knowledge in Memory” In R.C. Anderson, R.J. Spiro, W.E. Montague (Eds.) Schooling and the acquisition of knowledge pp.99-135. Hillsdale, NJ: Erlbaum.
  13. Rumelhart D.E., Hinton G.E., and Williams R.J., 1986 “Learning Representations by Back-propagation of Errors”, Nature, Vol.323, pp.533-536.
  14. Runeson S., 1994 “Perception of Biological Motion: the KSD-Principle and the Implications of a Distal Versus Proximal Approach”. In G. Jansson, W. Epstein & S. S. Bergström (Eds.), Perceiving events and objects, pp.383-405.
  15. Sumpter N. and Bulpitt A., 2000 “Learning SpatioTemporal Patterns for Predicting Object Behaviour,” Image and Vision Computing, Vol.18, No.9, pp.697- 704.
  16. Venkatesh Babu R. and Ramakrishnan K.R., 2004 “Recognition of Human Actions using Motion History Information Extracted from the Compressed Video”, Image and Vision Computing, Vol.22, No.8, pp.597- 607.
  17. Viviani P. and Schneider R., 1991 “A Developmental Study of the Relationship Between Geometry and Kinematics in Drawing Movements” Journal of Experimental Psychology: Human Perception and Performance, Vol.17, pp.198-298.
  18. Yoon H., Soh J., Bae Y. J., Yang H.S., 2001 “Hand Gesture Recognition Using Combined Features of Location, Angle, and Velocity”, Pattern Recognition, Vol.34, pp.1491-1501.
Download


Paper Citation


in Harvard Style

Montesanto A., Baldassarri P., F. Dragoni A., Vallesi G. and Puliti P. (2008). CAPTURING THE HUMAN ACTION SEMANTICS USING A QUERY-BY-EXAMPLE . In Proceedings of the International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2008) ISBN 978-989-8111-60-9, pages 356-363. DOI: 10.5220/0001932703560363


in Bibtex Style

@conference{sigmap08,
author={Anna Montesanto and Paola Baldassarri and A. F. Dragoni and G. Vallesi and P. Puliti},
title={CAPTURING THE HUMAN ACTION SEMANTICS USING A QUERY-BY-EXAMPLE},
booktitle={Proceedings of the International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2008)},
year={2008},
pages={356-363},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001932703560363},
isbn={978-989-8111-60-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, (ICETE 2008)
TI - CAPTURING THE HUMAN ACTION SEMANTICS USING A QUERY-BY-EXAMPLE
SN - 978-989-8111-60-9
AU - Montesanto A.
AU - Baldassarri P.
AU - F. Dragoni A.
AU - Vallesi G.
AU - Puliti P.
PY - 2008
SP - 356
EP - 363
DO - 10.5220/0001932703560363