Authors:
Marco Nicolini
;
Federico Simonetta
and
Stavros Ntalampiras
Affiliation:
LIM – Music Informatics Laboratory, Computer Science Department, University of Milan, Milan, Italy
Keyword(s):
Audio Pattern Recognition, Machine Learning, Transfer Learning, Convolutional Neural Network, YAMNet, Human Activity Recognition.
Abstract:
This paper employs the acoustic modality to address the human activity recognition (HAR) problem. The cornerstone of the proposed solution is the YAMNet deep neural network, the embeddings of which comprise the input to a fully-connected linear layer trained for HAR. Importantly, the dataset is publicly available and includes the following human activities: preparing coffee, frying egg, no activity, showering, using microwave, washing dishes, washing hands, and washing teeth. The specific set of activities is representative of a standard home environment facilitating a wide range of applications. The performance offered by the proposed transfer learning-based framework surpasses the state of the art, while being able to be executed on mobile devices, such as smartphones, tablets, etc. In fact, the obtained model has been exported and thoroughly tested for real-time HAR on a smartphone device with the input being the audio captured from its microphone.