Active Object Search with a Mobile Device for People with Visual Impairments

Jacobus C. Lock, Grzegorz Cielniak, Nicola Bellotto

2019

Abstract

Modern smartphones can provide a multitude of services to assist people with visual impairments, and their cameras in particular can be useful for assisting with tasks, such as reading signs or searching for objects in unknown environments. Previous research has looked at ways to solve these problems by processing the camera’s video feed, but very little work has been done in actively guiding the user towards specific points of interest, maximising the effectiveness of the underlying visual algorithms. In this paper, we propose a control algorithm based on a Markov Decision Process that uses a smartphone’s camera to generate real-time instructions to guide a user towards a target object. The solution is part of a more general active vision application for people with visual impairments. An initial implementation of the system on a smartphone was experimentally evaluated with participants with healthy eyesight to determine the performance of the control algorithm. The results show the effectiveness of our solution and its potential application to help people with visual impairments find objects in unknown environments.

Paper Citation

in Harvard Style

Lock J., Cielniak G. and Bellotto N. (2019). Active Object Search with a Mobile Device for People with Visual Impairments. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 4: VISAPP; ISBN 978-989-758-354-4, SciTePress, pages 476-485. DOI: 10.5220/0007582304760485

in Bibtex Style

@conference{visapp19,
author={Jacobus C. Lock and Grzegorz Cielniak and Nicola Bellotto},
title={Active Object Search with a Mobile Device for People with Visual Impairments},
booktitle={Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 4: VISAPP},
year={2019},
pages={476-485},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007582304760485},
isbn={978-989-758-354-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 4: VISAPP
TI - Active Object Search with a Mobile Device for People with Visual Impairments
SN - 978-989-758-354-4
AU - Lock J.
AU - Cielniak G.
AU - Bellotto N.
PY - 2019
SP - 476
EP - 485
DO - 10.5220/0007582304760485
PB - SciTePress