loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock
Pervasive Hand Gesture Recognition for Smartphones using Non-audible Sound and Deep Learning

Topics: Applications: Image Processing and Artificial Vision, Pattern Recognition, Decision Making, Industrial and Real World Applications, Financial Applications, Neural Prostheses and Medical Applications, Neural Based Data Mining and Complex Information Process; Convolutional Neural Networks; Deep Learning

Authors: Ahmed Ibrahim ; Ayman El-Refai ; Sara Ahmed ; Mariam Aboul-Ela ; Hesham Eraqi and Mohamed Moustafa

Affiliation: Department of Computer Science and Engineering, The American University in Cairo, New Cairo, Egypt

Keyword(s): Sonar, Gesture Recognition, Convolutional Neural Network, Data Augmentation, Transfer Learning, Feature Fusion, Doppler Effect.

Abstract: Due to the mass advancement in ubiquitous technologies nowadays, new pervasive methods have come into the practice to provide new innovative features and stimulate the research on new human-computer interactions. This paper presents a hand gesture recognition method that utilizes the smartphone’s built-in speakers and microphones. The proposed system emits an ultrasonic sonar-based signal (inaudible sound) from the smartphone’s stereo speakers, which is then received by the smartphone’s microphone and processed via a Convolutional Neural Network (CNN) for Hand Gesture Recognition. Data augmentation techniques are proposed to improve the detection accuracy and three dual-channel input fusion methods are compared. The first method merges the dual-channel audio as a single input spectrogram image. The second method adopts early fusion by concatenating the dual-channel spectrograms. The third method adopts late fusion by having two convectional input branches processing each of the dual- channel spectrograms and then the outputs are merged by the last layers. Our experimental results demonstrate a promising detection accuracy for the six gestures presented in our publicly available dataset with an accuracy of 93.58% as a baseline. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.149.232.87

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Ibrahim, A.; El-Refai, A.; Ahmed, S.; Aboul-Ela, M.; Eraqi, H. and Moustafa, M. (2021). Pervasive Hand Gesture Recognition for Smartphones using Non-audible Sound and Deep Learning. In Proceedings of the 13th International Joint Conference on Computational Intelligence (IJCCI 2021) - NCTA; ISBN 978-989-758-534-0; ISSN 2184-3236, SciTePress, pages 310-317. DOI: 10.5220/0010656200003063

@conference{ncta21,
author={Ahmed Ibrahim. and Ayman El{-}Refai. and Sara Ahmed. and Mariam Aboul{-}Ela. and Hesham Eraqi. and Mohamed Moustafa.},
title={Pervasive Hand Gesture Recognition for Smartphones using Non-audible Sound and Deep Learning},
booktitle={Proceedings of the 13th International Joint Conference on Computational Intelligence (IJCCI 2021) - NCTA},
year={2021},
pages={310-317},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010656200003063},
isbn={978-989-758-534-0},
issn={2184-3236},
}

TY - CONF

JO - Proceedings of the 13th International Joint Conference on Computational Intelligence (IJCCI 2021) - NCTA
TI - Pervasive Hand Gesture Recognition for Smartphones using Non-audible Sound and Deep Learning
SN - 978-989-758-534-0
IS - 2184-3236
AU - Ibrahim, A.
AU - El-Refai, A.
AU - Ahmed, S.
AU - Aboul-Ela, M.
AU - Eraqi, H.
AU - Moustafa, M.
PY - 2021
SP - 310
EP - 317
DO - 10.5220/0010656200003063
PB - SciTePress

- Science and Technology Publications, Lda.
RESOURCES

Proceedings

Papers

Authors

Ontology

CONTACTS

Science and Technology Publications, Lda
Avenida de S. Francisco Xavier, Lote 7 Cv. C,
2900-616 Setúbal, Portugal.

Phone: +351 265 520 185 (National fixed network call)
Fax: +351 265 520 186
Email: info@scitepress.org

EXTERNAL LINKS

PRIMORIS

INSTICC

SCITEVENTS

CROSSREF

PROCEEDINGS SUBMITTED FOR INDEXATION BY:

dblp

Ei Compendex

SCOPUS

Semantic Scholar

Google Scholar

Microsoft Academic