Vision Substitution with Object Detection and Vibrotactile Stimulus

Ricardo Ribani, Mauricio Marengoni

2019

Abstract

The present work proposes the creation of a system that implements sensory substitution of vision through a wearable item with vibration motors positioned on the back of the user. In addition to the developed hardware, the proposal consists in the construction of a system that uses deep learning techniques to detect and classify objects in controlled environments. The hardware comprise of a simple HD camera, a pair of Arduinos, 9 cylindrical DC motors and a Raspberry Pi (responsible for the image processing and to translate the signal to the Arduinos). In the first trial of image classification and localization, the ResNet-50 model pre-trained with the ImageNet database was tried. Then we implemented a Single Shot Detector with a MobileNetV2 to perform real-time detection on the Raspberry Pi, sending the detected object class and location as defined patterns to the motors.

Paper Citation

in Harvard Style

Ribani R. and Marengoni M. (2019). Vision Substitution with Object Detection and Vibrotactile Stimulus. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 4: VISAPP; ISBN 978-989-758-354-4, SciTePress, pages 584-590. DOI: 10.5220/0007577205840590

in Bibtex Style

@conference{visapp19,
author={Ricardo Ribani and Mauricio Marengoni},
title={Vision Substitution with Object Detection and Vibrotactile Stimulus},
booktitle={Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 4: VISAPP},
year={2019},
pages={584-590},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007577205840590},
isbn={978-989-758-354-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 4: VISAPP
TI - Vision Substitution with Object Detection and Vibrotactile Stimulus
SN - 978-989-758-354-4
AU - Ribani R.
AU - Marengoni M.
PY - 2019
SP - 584
EP - 590
DO - 10.5220/0007577205840590
PB - SciTePress