Automatic Recognition of Human Activities Combining Model-based AI and Machine Learning

Constantin Patsch, Marsil Zakour, Rahul Chaudhari

2022

Abstract

Developing intelligent assistants for activities of daily living (ADL) is an important topic in eldercare due to the aging society in industrialized countries. Recognizing activities and understanding the human’s intended goal are the major challenges associated with such a system. We propose a hybrid model for composite activity recognition in a household environment by combining Machine Learning and knowledge-based models. The Machine Learning part, based on structural Recurrent Neural Networks (S-RNN), performs low-level activity recognition based on video data. The knowledge-based part, based on our extended Activation Spreading Network architecture, models and recognizes the contextual meaning of an activity within a plan structure. This model is able to recognize activities, underlying goals and sub-goals, and is able to predict subsequent activities. Evaluating our action S-RNN on data from the 3D activity simulator HOIsim yields a macro average F1 score of 0.97 and an accuracy of 0.99. The hybrid model is evaluated with activation value graphs.

Paper Citation

in Harvard Style

Patsch C., Zakour M. and Chaudhari R. (2022). Automatic Recognition of Human Activities Combining Model-based AI and Machine Learning. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-547-0, pages 15-22. DOI: 10.5220/0010747000003116

in Bibtex Style

@conference{icaart22,
author={Constantin Patsch and Marsil Zakour and Rahul Chaudhari},
title={Automatic Recognition of Human Activities Combining Model-based AI and Machine Learning},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2022},
pages={15-22},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010747000003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - Automatic Recognition of Human Activities Combining Model-based AI and Machine Learning
SN - 978-989-758-547-0
AU - Patsch C.
AU - Zakour M.
AU - Chaudhari R.
PY - 2022
SP - 15
EP - 22
DO - 10.5220/0010747000003116