Generation-Based Data Augmentation Pipeline for Real-Time Automatic Gesture Recognition

Mingxi Cheng, Fatima Daha, Amit Srivastava, Ji Li

2023

Abstract

With the SARS-CoV-2 pandemic outbreak, video conferencing tools experience huge spikes in usage. Gesture recognition can automatically translate non-verbal gestures into emoji reactions in these tools, making it easier for participants to express themselves. Nonetheless, certain rare gestures may trigger false alarms, and acquiring data for these negative classes in a timely manner is challenging. In this work, we develop a low-cost fast-to-market generation-based approach to effectively reduce the false alarm rate for any identified negative gesture. The proposed pipeline is comprised of data augmentation via generative adversarial networks, automatic gesture alignment, and model retraining with synthetic data. We evaluated our approach on a 3D-CNN based real-time gesture recognition system at a large software company. Experimental results demonstrate that the proposed approach can effectively reduce false alarm rate while maintaining similar accuracy on positive gestures.

Paper Citation

in Harvard Style

Cheng M., Daha F., Srivastava A. and Li J. (2023). Generation-Based Data Augmentation Pipeline for Real-Time Automatic Gesture Recognition. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-623-1, pages 437-446. DOI: 10.5220/0011886600003393

in Bibtex Style

@conference{icaart23,
author={Mingxi Cheng and Fatima Daha and Amit Srivastava and Ji Li},
title={Generation-Based Data Augmentation Pipeline for Real-Time Automatic Gesture Recognition},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2023},
pages={437-446},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011886600003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Generation-Based Data Augmentation Pipeline for Real-Time Automatic Gesture Recognition
SN - 978-989-758-623-1
AU - Cheng M.
AU - Daha F.
AU - Srivastava A.
AU - Li J.
PY - 2023
SP - 437
EP - 446
DO - 10.5220/0011886600003393