Streamlining Action Recognition in Autonomous Shared Vehicles with an Audiovisual Cascade Strategy

João Ribeiro Pinto, João Ribeiro Pinto, Pedro Carvalho, Pedro Carvalho, Carolina Pinto, Afonso Sousa, Afonso Sousa, Leonardo Capozzi, Leonardo Capozzi, Jaime S. Cardoso, Jaime S. Cardoso

2022

Abstract

With the advent of self-driving cars, and big companies such as Waymo or Bosch pushing forward into fully driverless transportation services, the in-vehicle behaviour of passengers must be monitored to ensure safety and comfort. The use of audio-visual information is attractive by its spatio-temporal richness as well as non-invasive nature, but faces the likely constraints posed by available hardware and energy consumption. Hence new strategies are required to improve the usage of these scarce resources. We propose the processing of audio and visual data in a cascade pipeline for in-vehicle action recognition. The data is processed by modality-specific sub-modules, with subsequent ones being used when a confident classification is not reached. Experiments show an interesting accuracy-acceleration trade-off when compared with a parallel pipeline with late fusion, presenting potential for industrial applications on embedded devices.

Paper Citation

in Harvard Style

Pinto J., Carvalho P., Pinto C., Sousa A., Capozzi L. and Cardoso J. (2022). Streamlining Action Recognition in Autonomous Shared Vehicles with an Audiovisual Cascade Strategy. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 467-474. DOI: 10.5220/0010838900003124

in Bibtex Style

@conference{visapp22,
author={João Ribeiro Pinto and Pedro Carvalho and Carolina Pinto and Afonso Sousa and Leonardo Capozzi and Jaime S. Cardoso},
title={Streamlining Action Recognition in Autonomous Shared Vehicles with an Audiovisual Cascade Strategy},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={467-474},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010838900003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Streamlining Action Recognition in Autonomous Shared Vehicles with an Audiovisual Cascade Strategy
SN - 978-989-758-555-5
AU - Pinto J.
AU - Carvalho P.
AU - Pinto C.
AU - Sousa A.
AU - Capozzi L.
AU - Cardoso J.
PY - 2022
SP - 467
EP - 474
DO - 10.5220/0010838900003124
PB - SciTePress