Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition

Yasser Boutaleb, Yasser Boutaleb, Catherine Soladie, Nam-Duong Duong, Jérôme Royan, Renaud Seguier

2022

Abstract

First-person hand activity recognition is a challenging task, especially when not enough data are available. In this paper, we tackle this challenge by proposing a new low-cost multi-stage learning pipeline for first-person RGB-based hand activity recognition on a limited amount of data. For a given RGB image activity sequence, in the first stage, the regions of interest are extracted using a pre-trained neural network (NN). Then, in the second stage, high-level spatial features are extracted using pre-trained deep NN. In the third stage, the temporal dependencies are learned. Finally, in the last stage, a hand activity sequence classifier is learned, using a post-fusion strategy, which is applied to the previously learned temporal dependencies. The experiments evaluated on two real-world data sets shows that our pipeline achieves the state-of-the-art. Moreover, it shows that the proposed pipeline achieves good results on limited data.

Paper Citation

in Harvard Style

Boutaleb Y., Soladie C., Duong N., Royan J. and Seguier R. (2022). Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 839-848. DOI: 10.5220/0010856200003124

in Bibtex Style

@conference{visapp22,
author={Yasser Boutaleb and Catherine Soladie and Nam-Duong Duong and Jérôme Royan and Renaud Seguier},
title={Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP},
year={2022},
pages={839-848},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010856200003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP
TI - Multi-stage RGB-based Transfer Learning Pipeline for Hand Activity Recognition
SN - 978-989-758-555-5
AU - Boutaleb Y.
AU - Soladie C.
AU - Duong N.
AU - Royan J.
AU - Seguier R.
PY - 2022
SP - 839
EP - 848
DO - 10.5220/0010856200003124
PB - SciTePress