A Summarized Semantic Structure to Represent Manipulation Actions

Tobias Strübing, Fatemeh Ziaeetabar, Florentin Wörgötter

2021

Abstract

To represent human manipulation actions in a simple and understandable way, we had proposed a framework called enriched semantic event chains (eSEC) which creates a temporal sequence of static and dynamic spatial relations between objects in a manipulation. The eSEC framework has so far only been used in manipulation actions consisting of one hand. As the eSECs descriptors are in the form of huge matrices, we need to have a concise version of them. Here, we want to extend this framework to interactions which involve more hands. Therefore, we applied statistical and semantic analyses to summarize the current eSEC while preserving its important features and introducing an enhanced eSEC (e2SEC). This summarization is done by reducing the number of rows in an eSEC matrix and merging semantic spatial relations between manipulated objects. Eventually, we presented the new e2SEC framework which has 20% fewer rows, 16.7% less static spatial and 11.1% less dynamic spatial relations while still maintaining the eSEC efficiency in recognition and differentiation of manipulation actions. This simplification paves the way for a simpler recognition and predicting complex actions and interactions in a shorter time and is beneficial in real time applications such as human-robot interactions.

Paper Citation

in Harvard Style

Strübing T., Ziaeetabar F. and Wörgötter F. (2021). A Summarized Semantic Structure to Represent Manipulation Actions. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 370-379. DOI: 10.5220/0010258803700379

in Bibtex Style

@conference{visapp21,
author={Tobias Strübing and Fatemeh Ziaeetabar and Florentin Wörgötter},
title={A Summarized Semantic Structure to Represent Manipulation Actions},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP},
year={2021},
pages={370-379},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010258803700379},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP
TI - A Summarized Semantic Structure to Represent Manipulation Actions
SN - 978-989-758-488-6
AU - Strübing T.
AU - Ziaeetabar F.
AU - Wörgötter F.
PY - 2021
SP - 370
EP - 379
DO - 10.5220/0010258803700379
PB - SciTePress