Grasping Guidance for Visually Impaired Persons based on Computed Visual-auditory Feedback

Michael Hild, Fei Cheng

2014

Abstract

We propose a system for guiding a visually impaired person toward a target product on a store shelf using visual–auditory feedback. The system uses a hand–held, monopod–mounted CCD camera as its sensor and recognizes a target product in the images using sparse feature vector matching. Processing is divided into two phases: In Phase1, the system acquires an image, recognizes the target product, and computes the product location on the image. Based on the location data, it issues a voice–based command to the user in response to which the user moves the camera closer toward the target product and adjusts the direction of the camera in order to keep the target product in the camera’s field of view. When the user’s hand has reached grasping range, the system enters Phase 2 in which it guides the user’s hand to the target product. The system is able to keep the camera’s direction steady during grasping even though the user has a tendency of unintentionally rotating the camera because of the twisting of his upper body while reaching out for the product. Camera direction correction is made possible due to utilization of a digital compass attached to the camera. The system is also able to guide the user’s hand right in front of the product even though the exact product position cannot be determined directly at the last stage because the product disappears behind the user’s hand. Experiments with our prototype system show that system performance is highly reliable in Phase 1 and reasonably reliable in Phase 2.

References

  1. Chinellato, E. and del Pobil, A. P. (2005). Vision and grasping: Humans vs. robots. In IWINAC 2005, LNCS 3561 (J. Mira and J. R. Alvares (Eds.)), pages 366-375. Springer-Verlag, Berlin Heidelberg.
  2. H. Bay, A. Ess, T. T. and Gool, L. V. (2008). Speeded up robust features (surf). Computer Vision and Image Understanding, 110:346-359.
  3. Lowe, D. G. (2004). Distinctive image features from scaleinvariant keypoints. International Journal of Computer Vision, 60:91-110.
  4. Rousseeuw, P. J. and Leroy, A. M. (1987). Robust regression and outlier detection, chapt. 3. John Wiley & Sons, New York.
  5. T. Winlock, E. C. and Belongie, S. (2010). Toward real-time grocery detection for the visually impaired. Computer Vision and Pattern Recognition Workshops (CVPRW), pages 49-56.
Download


Paper Citation


in Harvard Style

Hild M. and Cheng F. (2014). Grasping Guidance for Visually Impaired Persons based on Computed Visual-auditory Feedback . In Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2014) ISBN 978-989-758-009-3, pages 75-82. DOI: 10.5220/0004653200750082


in Bibtex Style

@conference{visapp14,
author={Michael Hild and Fei Cheng},
title={Grasping Guidance for Visually Impaired Persons based on Computed Visual-auditory Feedback},
booktitle={Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2014)},
year={2014},
pages={75-82},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004653200750082},
isbn={978-989-758-009-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2014)
TI - Grasping Guidance for Visually Impaired Persons based on Computed Visual-auditory Feedback
SN - 978-989-758-009-3
AU - Hild M.
AU - Cheng F.
PY - 2014
SP - 75
EP - 82
DO - 10.5220/0004653200750082