Gesture Recognition for UAV-based Rescue Operation based on Deep Learning

Chang Liu, Tamás Szirányi, Tamás Szirányi

2021

Abstract

UAVs play an important role in different application fields, especially in rescue. To achieve good communication between the onboard UAV and humans, an approach to accurately recognize various body gestures in the wild environment by using deep learning algorithms is presented in this work. The system can not only recognize human rescue gestures but also detect people, track people, and count the number of humans. A dataset of ten basic rescue gestures (i.e. Kick, Punch, Squat, Stand, Attention, Cancel, Walk, Sit, Direction, and PhoneCall) has been created by a UAV’s camera. From the perspective of UAV rescue, the feedback from the user is very important. The two most important dynamic rescue gestures are the novel dynamic Attention and Cancel which represent the set and reset functions respectively. The system shows a warning help message when the user is waving to the UAV. The user can also cancel the communication at any time by showing the drone the body rescue gesture that indicates the cancellation according to their needs. This work has laid the groundwork for the next rescue routes that the UAV will design based on user feedback. The system achieves 99.47% accuracy on training data and 99.09% accuracy on testing data by using the deep learning method.

Paper Citation

in Harvard Style

Liu C. and Szirányi T. (2021). Gesture Recognition for UAV-based Rescue Operation based on Deep Learning. In Proceedings of the International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE, ISBN 978-989-758-511-1, pages 180-187. DOI: 10.5220/0010522001800187

in Bibtex Style

@conference{improve21,
author={Chang Liu and Tamás Szirányi},
title={Gesture Recognition for UAV-based Rescue Operation based on Deep Learning},
booktitle={Proceedings of the International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE,},
year={2021},
pages={180-187},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010522001800187},
isbn={978-989-758-511-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE,
TI - Gesture Recognition for UAV-based Rescue Operation based on Deep Learning
SN - 978-989-758-511-1
AU - Liu C.
AU - Szirányi T.
PY - 2021
SP - 180
EP - 187
DO - 10.5220/0010522001800187