A Framework for Electronic Travel Aids

Florian von Zabiensky and Diethelm Bienhaus

Technische Hochschule Mittelhessen, University of Applied Sciences, Institute of Technology and Computer Science,

Wiesenstr. 14, Giessen, Germany

Keywords:

Electronic Travel Aid, Framework, Distributed System, Human Computer Interface, Simulation, Visual

Impaired.

Abstract:

Electronic Travel Aids (ETA) help visually impaired people to avoid obstacles. In the ETA area new user in-

terfaces and interaction modes are investigated. Due to increasing hardware and software complexity used in

modern ETA there is an increasing demand for simulation tool to facilitate development processes. Such tools

enable testing of larger varieties of user experiences in shorter time. In the ﬁeld of acoustic virtual displays

capabilities of different sounds and psychoacoustic phenomena to display three-dimensional sound have to

be evaluated. Different simulators make it difﬁcult to compare research results. To facilitate comparability

between different approaches an ETA framework is introduced in this work. The framework includes a sim-

ulator which allows research groups to perform experiments with their speciﬁc interfaces. Due to the same

underlying simulation environment results of various groups are comparable. Using a framework would make

algorithms or devices from different research groups be directly usable for others. In this position paper the

development of such a framework is reasoned and the basic requirements on such a framework are proposed.

1 INTRODUCTION

In the context of electronic travel aids (ETAs) for vi-

sual impaired many research projects exist and prod-

ucts are developed to support user orientation, ﬁnding

a target or recognizingan obstacle. Those projects use

sensors like a compass, an accelerometer or a gyro-

scope to gain environmental information for naviga-

tion purposes. Information is then presented by a de-

vice that serves an information channel different from

vision. Typically this is the haptic or acoustic chan-

nel.

In this position paper a framework is proposed to

support research and development of ETAs for vi-

sual impaired. In ETAs devices and software fulﬁll

two tasks (1) acquisition of environmental informa-

tion and (2) displaying them. For the display process

the chosen information channel has to be tailored pre-

cisely. It should not disturb the function of the chan-

nel e.g. perception of the environment via sound in

case of an acoustical display. On the other hand gath-

ered information should be displayed to a great ex-

tent. Preferably research on information acquisition

and displaying should be performed independently.

The approach introduced here aims on an decoupling

of these two tasks: The presentation subsystem can be

developed using our environment-simulation as part

of the framework decoupled from the data acquisition

systems. Such an environment-simulation is imple-

mented in current research projects by virtual envi-

ronments, where the user can walk by mouse or key-

board input. These simulation-games provide sensor

data gathered by virtual sensors in a virtual environ-

ment. Thus the tested presentation subsystem will

display the virtually measured data. No direct depen-

dency of the presentation subsystem to the sensor sub-

system exists, allowing parallelized research. For the

ﬁnal integration-step the modular approach provided

by the framework helps replacing the simulation sub-

system by a real acquisition system. The modular ar-

chitecture facilitates overall system optimization by

selecting the best combination of a variety of differ-

ent subsystem-implementations.

Environmental simulation may also be used to

perform user studies. The simulator enables monitor-

ing of (1) movement and (2) collisions with obstacles

as quality metrics. Other advantages of simulations in

complete virtual environments are: (3) repeatable en-

vironmental inﬂuences and (4) avoidance of real user

interaction in dangerous situations. Serious disadvan-

tages are: (1) simulations cannot cover the full range

of possible situations and (2) simulating user behav-

ior may be close to reality, but will never be exactly

realistic. E.g. a head movement would optimize the

von Zabiensky F. and Bienhaus D.

A Framework for Electronic Travel Aids.

DOI: 10.5220/0006514701720177

In Proceedings of the International Conference on Computer-Human Interaction Research and Applications (CHIRA 2017), pages 172-177

ISBN: 978-989-758-267-7

perception of the position of a sound source placed in

a three dimensional space compared to a ﬁxed head

(Wu et al., 1997). But in simulations with a keyboard

as an input device, head movements are not as intu-

itive as they are in the real world. Providing a sin-

gle open source simulation system for research groups

working in that area removes duplicated work, help-

ing development being focused on the key aspects of

a project. Also an accumulated effort of a commu-

nity driven project will lead to a far more advanced

simulation compared to the scope of work a single re-

search team can gain. An example of such a widely

used framework in the domain of mobile robotics is

the Robot Operating System (ROS)

. It combines a

communication middleware as well as simulation en-

vironment and many modules implementing common

algorithms as well as hardware drivers used in mo-

bile robotics. It is an open source project supported

by a wide community. Thus the decision to develop a

framework for ETAs is justiﬁed. Section 2 outlines a

few ETAs, the GIVE-ME framework and ROS. Sec-

tion 3 lists requirements for an ETA framework, fol-

lowed by a discussion of disadvantages and advan-

tages of such a framework.

2 RELATED WORK

Dakopouls and Bourbakis divided ETAs into three

categories based on their feedback interfaces. The

categories for classiﬁcation are “Audio feedback” ,

“Tactile feedback” and “w/o interface”. (Dakopou-

los and Bourbakis, 2010) Given below three projects

are described and categorized. After that the GIVE-

ME framework and ROS are introduced as a basis for

the intended framework. The Tyﬂos project assists vi-

sual impaired in navigation tasks and the environment

recognition.

“In particular, the Tyﬂos prototype inte-

grates a wireless portable computer, cameras,

range and GPS sensors, microphones, natural

language processor, text-to-speech device, an

ear speaker, a speech synthesizer, a 2D vibra-

tion vest and a digital audio recorder.” (Bour-

bakis et al., 2008)

Tyﬂos is a multi-modal assistance system which inter-

acts via different input and output channels to gather

or present information from the user. A tactile device

to serve an output channel is the vibration vest. Like

setting a pixel on a two dimensional monitor the vest

vibrates on a two dimensional position. To gather in-

formation of the environment or get instructions from

http://www.ros.org/

the user the project uses a camera or a microphone.

This project is categorized as “Tactile feedback”.

Gomez Valencia developed a few systems to assist

visual impaired and described them in his PhD The-

sis. He developed a system to recognize street signs

and read them to the user. This assists the user in

case of self localization in a city. Additionally he de-

veloped an acoustic display to present different colors

by different instruments. He developed an obstacle

detection and a tactile input device, too. (Gomez Va-

lencia, 2014) Because of the acoustic display for in-

formation presentation this project is categorized as

“Audio feedback”.

The sound of vision project

did much research on

three dimensional sound for acoustic displays. There-

for they developed glasses as a device to capture the

environmental and present information to the user via

sound. To provide their results to a community they

plan a reusable soniﬁcation library and a reusable

training serious game as part of their “additional out-

puts”. This project comes close to the intended open

community. But it only provides the results with a

library but not a whole framework with the possibil-

ity to exchange a device to test algorithms with other

hardware. This project is also classiﬁed as “Audio

feedback” even if it has a few elements of “Tactile

feedback” by providing a haptic west.

Khoo describes a real framework. The GIVE-

ME framework was developed under the key points

of maintaining a simulation environment for devices

with the intention to use gamiﬁcation aspects in user

tests to make the tests more interesting. He describes

the framework with the simulator and the workﬂow of

keeping this framework alive. (Khoo, 2016) But the

last changes to his website were in March of 2016.

ROS is a frameworkfor distributed systems driven

by a large and vital community. The active com-

munity is a big beneﬁt in contrast to the GIVE-ME

framework. ROS provides the ability for modules to

communicate or interact in different ways. E.g. a

module encapsulates an algorithm or a device driver.

The modules could be distributed over a network.

ROS also implements different tools to analyze live

data or replay recorded data. A mass of modules

for typical problems in mobile robotics such as self

localization or navigation are implemented and pro-

vided by ROS. ROS will be used as the basis for

the intended framework because of its scope and the

similarities between problems in mobile robotics and

ETAs for visual impaired.

http://www.soundofvision.net/

3 REQUIREMENTS

ETAs are mobile systems a user have to rely on.

Therefor there are several requirements which have

to be fulﬁlled. The requirements presented in this sec-

tion are derived from Dakopoulos’s und Bourbakis’s

work in which they compared several ETAs devel-

oped in research projects (Dakopoulos and Bourbakis,

2010). This comparison is based on a list of features

described in table 1.

Fast Data Communication. Communication be-

tween different devices like a haptic belt, an

acoustic display, a 3D camera, etc. is time critical

hence the framework has to supply a fast infras-

tructure. A slow communication would result in

a delayed perception of the environment which

could result in a collision or other dangerous

scenarios.

Reliable Data Communication. Message exchange

between modules has to be reliable. A message

has to reach the destination without fail and must

be accepted only once if an repeated message oc-

curs. If the modules would communicate in an

unreliable way the detection of an object possibly

would not be reported to the user or it could be

repeated at a time where the object is out of the

way.

Platform Independence of the Framework. The

platform a device is running on has to be transpar-

ent for the framework. T It could be reasonable to

deploy one part of the system to a Linux platform

and another part to e.g. an Android platform.

Close-to-Hardware Programming Abilities.

ETAs can use user interfaces that have to be

programmed directly (setting an output pin to

start a vibration engine). But it could be an

advantage to program the system with higher

languages in case of rapid prototyping. Therefore

a framework should be programmable on a higher

level of abstraction (Python) and on a low level of

abstraction, too.

Distributed Nodes for Different Tasks. The frame-

work should follow a distributed system architec-

ture for ETAs. This way the control of acoustic

or haptic interfaces and their drivers can all be

implemented as distributed modules which com-

municate with each other. An explicit description

of module interfaces allows their exchange. The

interface consists of data and services provided

by a module. In case a navigation functionality

should be implemented one can use a module for

self localization and another for path planning and

additional modules for engine drivers. The com-

munication of modules should not be limited to

one platform therefor a network communication

should be provided.

Physical Layer Independent Communication.

Especially in the context of a distributed system

the communication should be independent of the

used communication medium (WiFi, Bluetooth,

CAN, ... ).

Open Source. The framework should be open

source. This way the development could be

driven by a community.

Security. If the system would implement a face

recognition based on a phone book then private

data would be communicated. To ensure privacy

of such data secure communication is essential.

Data manipulation must be prevented. So security

means also authenticity and integrity.

4 DISADVANTAGES OF AN OPEN

FRAMEWORK

On ETAs resources are constrained due to mobility

and small size. If a framework is used to develop

ETAs developers may lose the focus to optimize the

code for a constrained device. The complexity of

a framework can be much higher than needed for a

task. A device which is developed without a frame-

work could reduce the power consumption to provide

a higher battery life. There are also risks in the ad-

vantage of a community. If there will be no commu-

nity formed, there is no advantage in this point. The

acceptance of manufacturers would also improve the

framework but is not necessarily predictable.

5 ADVANTAGES OF AN OPEN

FRAMEWORK

The implementation of an open and distributed frame-

work for the development of ETAs offers opportuni-

ties if the risks of a community and the acceptance of

manufacturers eventuates.

Contribute and Exchange Basic Code

In the sound of vision project an audio game for sound

localization was developed. With this game different

auditive representations on navigational hints were

tested and a suggestion to reduce front / back confu-

sion in sound localization was proposed (B˘alan et al.,

Table 1: Structural and operational features. The upper half for the user’s needs, the lower half for the developer’s and

engineer’s needs. (Dakopoulos and Bourbakis, 2010).

Feature Description

Realtime The system operates fast enough such that information exchange with the user

is useful e.g. if an obstacle detection system needs 10 seconds to detect an

obstacle that is 6 feet in front of the walking-user,then the device is not working

in appropriate ”real” time.

Wearable The device is worn on the user’s body or as a piece of his clothing. Wearable

devices are useful for applications that require computational support while the

user’s hands, voice, eyes, ears or attention are actively engaged with the phys-

ical environment. The interaction between the user and the device is constant.

Another feature is the ability of multi-tasking: users can continue their current

tasks while using the device; it has to be integrable into all other actions.

Portable The device has to be lightweight and small in size with an ergonomic shape so

that the user can carry it without effort for long distances and time.

Reliable The system functions correctly in routine but also in different hostile or/and

unexpected circumstances.

Low-Cost The device is (or it will be, when it comes to the mass production) affordable

for most users.

Friendly The device is easy to learn, easy to use and encourages the user to regard the

system as a positive help in getting the job done.

Functionalities The number and importance of the system’s functionalities.

Simple The complexity of both hardware and software is small. The hardware parts are

few and simple to use (from the user’s point of view) and simple to build (from

the designer’s aspect)

Robustness The device is well constructed so it withstands difﬁcult environmental condi-

tions or hard use. Its functionality varies minimally despite of disturbing factor

inﬂuences. It can still function in the presence of partial failures.

Wireless connectivity The device is connected wireless to a computer (server/database) in order

to continuously exchange information. Additionally, part of the processing

needed for its operation can be done on the remote computer.

Performance Overall performance

Originality The idea and the methodology are original promoting scientiﬁc and technolog-

ical knowledge.

Availability The system is implemented. A device that is ready to use and real-time exper-

iments can be performed e.g. a system that is only in the software stage is not

available.

Future Future improvements or enhancements

2015b). This way they developed a simulation to ﬁnd

the best auditive representation to use them in their

device. A similar concept of a virtual labyrinth was

introduced in the RaVis3d

project which began re-

cently. If a realistic simulation environment would

exist and other projects would propose their results

and implementations with the same simulation envi-

ronment, research projects could complement each

other in a more effective way. This can be enabled

by a common framework and a platform to exchange

results and implementations.

http://ravis-3d.de/

Modular Architecture and Uniform

Messages

In a framework based on ROS the architecture of

ETAs would be modular. If the development is re-

alized in a strictly modular way the advantage of ex-

changing modules with the same interface would be

given. Even the maintainability would proﬁt of this

architecture. The interoperability between different

devices would also proﬁt by uniform message and ser-

vice interfaces.

Multimodal User Interfaces

The amount of information delivered by an device to

a user’s senses is restricted. E.g. an auditive display

could display many information. But at some point

the user would be distracted by the auditive infor-

mation and natural signals like environmental sounds

(cars, bells, ...) would be discarded. Thus there have

to be more than one interface if the amount of data is

to big. In parallel a tactile interface could be used to

arrange the information between these devices. An-

other use case of using different devices is user train-

ing with feedback. This way B˘alan et. al. trained

users to optimize three dimensional audio localization

with haptic feedback. (B˘alan et al., 2015a) A frame-

work enables changing and bind interfaces if the mes-

sage and service interfaces are deﬁned for different

interface types.

Exchange User Interfaces

In every project it is possible to reach a point where

a change of direction would be appropriate. In the

sound of vision project they changed their presenta-

tion device from a stereo headphone to a self made

headphone.

“The conclusions from the spatial audio

tests led us away from HRTF-based solutions

towards the idea of the custom multi-speaker

headphones.” (Bujacz et al., 2016)

Such an change would be much simpler if inter-

faces are standardized. This way device drivers in op-

erating systems are implemented for different classes

of devices e.g. sound cards. With the same concept

the interface of a device driver could be implemented

for a 3D audio device by deﬁning standard messages

and services. This way a 3D audio device will be ex-

changeable. Appropriate presentation strategies (e.g.

which frequencies should be used to display a posi-

tion) are an important issue, too. As an example Coo

and Cha developed a strategy to reduce front / back

confusion to optimize sound localization (Koo and

Cha, 2008). Exchangeability of strategies are worth-

while as well. In case of an existing framework such

strategies could even be distributed between different

research groups if they implement the same interfaces

(messages and services). By this means a research

group that wants to develop strategies to present data

does not have to develop the device, and vice versa.

Every group could concentrate on their main topic.

Distributed Computation

Gomez Valencia realized different projects during his

PhD like an detection of letters, colors, objects and

their presentation (Gomez Valencia, 2014). One point

of his conclusion deals with the computational power.

Parallel execution of his applications was restricted by

the used platform. If the software would be based on

a ROS like framework applications can be deployed

on different hardware nodes and run in parallel.

Augmented Reality for Real Life Test

Environments

The intended framework will implement a simulation

environment. In this environment the usage of aug-

mented reality is intended. This will allow a realistic

movement of users in a simulation environment. An-

other beneﬁt will be the usage of markers. When a

marker is recognized its position, orientation and ID

can be determined. E.g. a wall, a door or another ob-

stacle could be detected. Markers can be placed in the

real world to place virtual objects represented e.g. by

a tone in the three dimensional space. By simulating

this known environment a device could be tested in

a realistic environment with realistic movements. By

recording of camera data tests can be evaluated after-

wards to improve results.

6 FUTURE WORK AND

CONCLUSIONS

In this position paper a framework for the develop-

ment of ETAs has been reasoned. Both the devel-

opment and the research on ETAs proﬁt by such a

framework with improvements in user tests and ex-

change of results. The ﬁrst step in the development

process of such a framework is to evaluate whether

ROS matches all requirements mentioned above. If so

several sample applications and devices will be imple-

mented and integrated to show the beneﬁts of a frame-

work for ETAs. The intended architecture is shown in

Fig. 1 showing the main part of the framework as light

gray elements. The dark gray elements illustrate the

parts that should be developed and implemented by

the community such as human interface devices with

their related drivers and algorithms.

In the development of device drivers a uniform

interface of the devices is a big advantage. By

that means quality and maintainability of software in

ETAs will be enhanced. A strict modular approach

allows independent testing and deployment on inter-

Message Definitions:

- Orientation Messages

- Obstacle Messages

- ...

Communication:

- Publisher / Subscriber

- Services

Build Environment:

- Tools

- Structure

- ...

Peripheral Devices:

- Vibration Belt

- 3D Audio Device

- Obstacle Detection Device

- ...

Algorithms:

- Obstacle Detection

- Navigation

- SLAM

- ...

Simulation:

- Virtual Environment

- Augmented Reality

- Simulation of sensor data

- ...

Figure 1: Overview of the intended framework. The light gray areas describe the framework itself with simulation, message

formats, communication and build environment. The dark gray areas describe the community driven parts of the framework

such as human interface devices as peripherals and algorithms.

connected distributed nodes. Communication mod-

ules and basic devices with its drivers will be imple-

mented within the framework. So developmentefforts

would be reduced by using existing modules for dif-

ferent tasks. During user tests of human interaction

devices the main point of research projects should be

the development of such a device and the method to

present the data. This achieves optimal results in the

task of interacting with the user. Usually research

projects are restricted in time. If the development

of a simulation environment or the communication

between modules were part of an project, this task

would consume time and costs. In providing such an

environmentas part of a framework this time could be

saved and spent more goal-oriented. Additionally the

results of different research groups are more compa-

rable by using the same simulation environment. We

think ROS is a good basis for such a framework for

ETAs because of the similarity of requirements and

constraints in mobile robotics and ETAs for visual

impaired. ROS provides already many modules for

problems in the areas of navigation, communication,

self localization and image or sensor data process-

ing. Evaluating the applicability of ROS for ETAs is

worthwhile.

REFERENCES

B˘alan, O., Moldoveanu, A., and Moldoveanu, F. (2015a).

3d Audio and Haptic Interfaces for Training the Spa-

tial Acoustic Resolution in Virtual Auditory Environ-

ments. In Student Think Tank at the 21st International

Conference on Auditory Display, page 3.

B˘alan, O., Moldoveanu, A., Moldoveanu, F., and Butean,

A. (2015b). Developing a navigational 3d audio game

with hierarchical levels of difﬁculty for the visually

impaired players. In RoCHI, pages 49–54.

Bourbakis, N., Keefer, R., Dakopoulos, D., and Esposito, A.

(2008). A Multimodal Interaction Scheme between a

Blind User and the Tyﬂos Assistive Prototype. pages

487–494. IEEE.

Bujacz, M., Kropidlowski, K., Ivanica, G., Moldoveanu, A.,

Saitis, C., Csapo, A., Wersenyi, G., Spagnol, S., Jo-

hannesson, O. I., Unnthorsson, R., and others (2016).

Sound of Vision-Spatial audio output and soniﬁcation

approaches. In International Conference on Comput-

ers Helping People with Special Needs, pages 202–

209. Springer.

Dakopoulos, D. and Bourbakis, N. (2010). Wearable Ob-

stacle Avoidance Electronic Travel Aids for Blind:

A Survey. IEEE Transactions on Systems, Man,

and Cybernetics, Part C (Applications and Reviews),

40(1):25–35.

Gomez Valencia, J. D. (2014). A computer-vision based

sensory substitution device for the visually impaired

(See ColOr). PhD thesis, University of Geneva.

Khoo, W. L. (2016). GIVE-ME: Gamiﬁcation In Virtual En-

vironments for Multimodal Evaluation-A Framework.

PhD thesis, City University of New York.

Koo, K. and Cha, H. (2008). Enhancement of a 3d Sound

Using Psychoacoustics. World Academy of Science,

Engineering and Technology, International Journal of

Electrical, Computer, Energetic, Electronic and Com-

munication Engineering, 2(1):8–12.

Wu, J.-R., Duh, C.-D., Ouhyoung, M., and Wu, J.-T. (1997).

Head motion and latency compensation on localiza-

tion of 3d sound in virtual reality. In Proceedings of

the ACM symposium on Virtual reality software and

technology, pages 15–20. ACM.