ELECTROPHYSIOLOGICAL CONTROL SIGNALS FOR
PERSONS WITH NEURODEGENERATIVE CONDITIONS:
BLENDED CONTROL SIGNALS
Ana Londral
ANDITEC, Lisbon, Portugal
Luis Azevedo
Centro de Análise e Processamento de Sinal, Instituto Superior Técnico, Lisbon, Portugal
Pedro Encarnação
Faculdade de Engenharia, Universidade Católica Portuguesa, Lisbon, Portugal
Keywords: Electrophysiological control signals, Assistive Technologies, User Interface, selection methods, progressive
conditions.
Abstract: Severe neurological conditions may considerably affect one’s functional capabilities. Special computer
interfaces and access methods have been developed in attempt to provide a mean to overcome the functional
disabilities experienced by persons in these conditions. In this paper, a case study on the usage of a brain-
body interface by a young man with Amyotrophic Lateral Sclerosis is presented. From the study different
ways of interacting with the computer, beyond the traditional direct selection and scanning methods,
emerge. These resort to control signals that combine binary and continuous features, blended control
signals. Such control signals may provide more flexible and efficient ways of interacting with Assistive
Technology systems, especially for those individuals with neurodegenerative conditions.
1 INTRODUCTION
Various diseases or conditions may impose severe
limitations in one’s motor abilities and consequently
lead to communication disorders. These diseases and
conditions can be divided in progressive and static
or improving (Glennen and DeCoste, 1996).
Examples of progressive conditions are
neurodegenerative diseases, as Amyotrophic Lateral
Sclerosis (ALS), Multiple Sclerosis or Parkinson,
and some oncological conditions. Brainstem
strokes, traumatic brain injuries or spinal injuries are
included in static or improving conditions, as they
remain unchanged or improve over time.
Assistive Technologies can be defined by “any
item, piece of equipment or product system whether
acquired commercially off the shelf, modified, or
customized that is used to increase or improve
functional capabilities of individuals with
disabilities.” (
United States Congress, 1998). Although
there are many definitions for AT, the main
objective of assistive technologies (equipments and
services) is to contribute to a better quality of life of
the many persons affected by disabilities worldwide,
through the integration of technological aspects in
equipments, services and contexts (Azevedo, 2006).
This paper is focused in AT systems, based on
computer interaction, for persons with
neurodegenerative conditions, i.e. progressive
conditions caused by neurodegenerative diseases. In
this context, individuals experience progressive
decline in motor functioning, which dramatically
affects their quality of life. Neurogenic
communication disorders are a common
consequence of neurodegenerative conditions as
individuals progressively loose their ability to write
and/or speak. Through computer interaction these
persons may access to communication aids for
writing or speaking.
254
Londral A., Azevedo L. and Encar nac¸
˜
ao P. (2008).
ELECTROPHYSIOLOGICAL CONTROL SIGNALS FOR PERSONS WITH NEURODEGENERATIVE CONDITIONS: BLENDED CONTROL SIGNALS.
In Proceedings of the First International Conference on Health Informatics, pages 254-259
Copyright
c
 SciTePress

AT selection for people with neudegenerative
conditions is a big challenge since the progression of
the disease must be “previewed”, as well as other
factors related to the individual’s context. The
progression of these diseases will lead to different
needs and capabilities along the different stages of
the disease. Flexibility is thus an utmost important
characteristic for AT systems, which have to
respond to individuals needs during all stages and
conditions.
Considering AT systems based on computer
interaction, user interface is an important part of the
system, which translates users input signals into
control signals. The most common user interfaces
for severe neurodegenerative conditions are the ones
using eyetracking techniques and the ones based on
electrophysiological signals (Felzer and Nordmann,
2006). User interfaces are much dependent on the
input signals that the user can control. The problem
of the type of electrophysiological control signals
that persons with neurodegenerative conditions can
generate to access to AT devices, from early to late
stages of disease, is addressed in this paper.
The paper is organized as follows. In Section 2, a
brief description of user interfaces and typical
selection methods used in AT systems context are
exposed. The use of electrophysiological signals as
control signals for AT systems is addressed in
Section 3. Section 4 contains a description of a case
study, which aims at evaluating the use of a
brainbody interface by a young man with ALS to
access to a computer as a communication device.
This case study is discussed in Section 5 stressing
the types of control signals the user was able to
generate and proposing a new class of control
signals – blended control signals. Paper conclusions
are presented in Section 6.
2 USER INTERFACES
One of the critical elements of AT systems for
persons with neurodegenerative conditions is the
User Interface (UI). The UI receives user’s input and
translates it into control signals to access to the AT
devices. These signals can be generated by various
movements, such as hands, eyes or head movements,
or even by other body sources as, for example,
electroencephalograph signals. Control signals are
then very dependent and conditioned by user's
physical and context conditions.
A general representation of a UI for AT Systems
is proposed in (Cook and Hussey, 2002) as shown in
Figure 1.
Figure 1: User Interface of an AT System.
The Selection Method defines the way the user
will select each element of selection set. Typically,
AT devices provide Scanning or Direct Selection
methods. Direct selection is possible if the user can
generate at least as many control signals as the
selection set. Otherwise, user has to resort to an
indirect selection method (e.g. scanning) to pick an
element of the selection set.
For example, given the task of writing in a
computer, one may use a direct selection method
pressing each key on the keyboard (AT device);
however, if the person is not able to directly select
each key, she needs to use a scanning method
controlling it with one or more binary signals.
Scanning method is much slower that direct
selection method. However, there are many
strategies that try to make this selection method
more efficient according to users’ abilities (Cook
and Hussey, 2002).
This traditional strict division of selection
methods ignores the possibility of having other kinds
of interaction, based on control signals richer than
simple on/off signals though not rich enough to
control a 2-axis interaction (as showed in Figure 2
for the example of access to a virtual keyboard).
Figure 2: Example of access to a virtual keyboard.
Traditional division for selection methods consider direct
selection (continuous control signals for 2-axis control) or
scanning method (based on one or two binary control
signals).
When focusing on progressive conditions, AT
systems must consider different kinds of access,
being flexible to adapt to users' functionality. In this
paper, the search for other kind of selection
methods, based on electrophysiological control
signals is discussed.
ELECTROPHYSIOLOGICAL CONTROL SIGNALS FOR PERSONS WITH NEURODEGENERATIVE CONDITIONS:
BLENDED CONTROL SIGNALS
255

3 ELECTROPHYSIOLOGICAL
CONTROL SIGNALS
Technology development in the field of biosensors
has shown that individuals can generate and control
various kinds of output signals that can therefore be
used as control signals. In particular, todays control
signals that are generated within the individual body
can be used for man-machine interface.
When evaluating a person in a later stage of a
neurodegenerative condition, often the main
problem is to find one control signal that the user is
able to intentionally generate. Even one single
control signal supports an indirect access method,
allowing a selection within a given set. The use of
electrophysiological signals brings new perspectives
on the number and type of control signals that a user
with severe neurodegenerative conditions may
generate.
At the skin surface level, two different types of
signals can be captured: electric (e.g.
electromyography, electrocardiography) and non-
electric information (e.g. temperature, blood
pressure) (Allanson, 2002). Typically, the former are
the ones used for AT control systems, as it is the
case of the AT system presented in this paper.
In case of individuals with neurodegenerative
diseases, especially in later stages of the disease,
these signals can be an efficient way of generating
control signals. For example, an individual with very
low motor control, who can’t press a switch, can be
able to generate control signals captured by an EMG
sensor. In fact, an electrophysiological signal can
provide a motor independent control signal even for
persons in locked in state (Wolpaw et al., 2002;
Wills and MacKay, 2006). However, an important
issue to consider is that, due to its physiologic
nature, electrophysiological information depends on
the physical and environment conditions of each
individual (such as diseases, fatigue, humour,
environment temperature, familiar context, etc.).
Thus, it is important to know the physiological
mechanisms that produce the signals and how these
signals are affected by referred conditions.
Therefore, in AT systems design, each case is a
singular case, influenced by individuals’ unique
conditions and particular disease progression.
4 CASE-STUDY
A Small Number Design methodology (Iacono,
1992)(Stevens and Edwards, 1996) was used in
order to evaluate the interaction of an individual
with ALS with a brainbody interface (™Brainfingers
Cyberlink). This brainbody interface consists on a
headband with three surface electrodes placed on the
forehead. The control signals generated by this
interface are based on muscle and brain potentials,
and are called brainfingers (Junker, 1995).
The individual that voluntarily participated in the
study is in a later stage of the disease for some years.
He can control very few movements and uses a
pressing switch activated by slight head movements
as the control interface to his communication aid. He
is thus able to control a scanning process in software
The Grid© for communication purposes and Internet
access. With this system, this person wrote a
published poetry book.
Figure 3: User studied using ™Brainfingers Cyberlink
interface in a training session.
The motivation for this case study was twofold:
are there alternative ways (and more efficient) for
this individual to interact with an AT system?; is it
possible for him (using ™Brainfingers Cyberlink
interface) to generate more control signals or
“richer“ than binary control signals?
4.1 Test Design
A protocol for evaluation was developed and tested
aiming at studying the control signals that the user
was able to generate with the interface. Starting from
the binary signal that the user used before, “richer”
signals where progressively attempted. The tests
followed the four steps described below.
a) One binary control signal
To gain confidence with the system, the user was
firstly asked to use the AT system by means of his
pressure switch, as he is used to. Then, the
mechanical switch was replaced by the brainbody
interface. Different sources of muscle potencial were
essayed as a binary control. The signal generated by
opening the jaw was found to be more efficient. In
fact, this is the gesture that user does to
communicate to his close friend and physiotherapist
as a 'yes'.
HEALTHINF 2008 - International Conference on Health Informatics
256

Figure 4: Virtual keyboard used to evaluate interaction
using a binary control signal to write a sentence by a
scanning method.
b) One continuous control signal
After getting used to access to computer using a
binary muscle signal, the user was challenged to
play a game where he had to move a bar in one axis
to catch a ball. The bar could be controlled by user
regulation of the muscle signal amplitude.
c) One “continuous and discrete” control signal
After being able to generate one binary control
signal and one continuous control signal, the user
was asked to access to his communication software
using the combination of these two control signals.
For that, a special one-row keyboard was designed
(see Figure 5) and the user had to select each cell in
a specific order. In order to do that, a continuous
signal had to be controlled between two thresholds
to move the selection bar. When this bar is in the
desired position, the signal should be raised above
the second threshold, thus making the selection (see
Figure 4). In the designed application, the
continuous signal amplitude within the two
thresholds controlled the movement within the row,
and the second threshold was used to generate a
binary control signal for selection of the highlighted
cell.
Figure 5: Virtual keyboard used to evaluate interaction
with one continuous control signal to move mouse cursor
in one-axis and one binary control signal to make key
selection.
Figure 6: Representation of the technique used to combine
two different control signals. By moving the bio-signal
amplitude (the square) between the two thresholds, the
user will move one object in one direction. When
overcoming the 2nd threshold, the user makes a selection.
d) Two continuous control signals
Then user was asked to use two continuous
control signals to navigate through rows and
columns, in a keyboard as shown in Figure 7. Two
different brainfingers (control signals generated by
the studied user interface) were used. The source of
these signals were muscle potencial generated by
opening jaw and one brainfinger potencial (Junker,
1995) generated by subtle forehead movements.
Control was based on these two control signals:
the first (described in Figure 6) to control x-axis, and
the second to control y-axis.
Figure 7: Virtual keyboard used to evaluate interaction
using two continuous control signals to move mouse
cursor in two-axis and one binary control signal to make
key selection.
5 RESULTS
The user was able to control the UI using different
selection methods. Qualitative and quantitative data
were analysed, giving together a more complete
evaluation of the results (for more details, please
refer to (Londral, 2007).
The main problems related to control signals
were low SNR, involuntary generated control signals
and delay in generating the control signal. The latter
was due to the difficulty in raising and lowering the
amplitude of the control signals. After some minutes
of training, the involuntary impulses were almost
suppressed. When writing using a scanning access
2
nd
threshold
1
st
threshold
ELECTROPHYSIOLOGICAL CONTROL SIGNALS FOR PERSONS WITH NEURODEGENERATIVE CONDITIONS:
BLENDED CONTROL SIGNALS
257

method, and after five minutes training, user was
able to do 3,18 key selections per minute.
Considering that user is able to do 5,16 key
selections per minute with his usual UI (a pressing
switch) and that this result was reached in just one
short session of training, it is expected that this
performance will improve with training.
The user could also generate one continuous
control signal using it to move the mouse cursor in
one axis successfully. Resorting to a control signal
that combines continuous and binary features, as
described in Section 4.1-c, the results obtained were
4,36 key selections per minute, thus improving the
performance attained with one binary control signal.
However, it is important to note that this selection
method was tested just with a small selection set
(smaller than the one used for the previous result).
When testing the use of two continuous control
signals, in order to control the mouse cursor in two
axis (as described in Section 4.1-d), the performance
was only 1,71 key selections per minute. This
selection method was difficult for the user,
especially in managing the control of the different
thresholds. Therefore, more training is necessary to
validate this technique.
From this case study, it is clear that the user was
able to generate various types of control signals that
could provide more flexibility to a UI, thus making it
more adaptable to the user progressive conditions.
5.1 Blended Control Signals
Traditionally, control interfaces generate binary
control signals (used to control scanning methods) or
continuous control signals in 2-axis (used to control
direct selection). Based on the various types of
electrophysiological signals that the individual in
this study could generate, a new class of control
signals is proposed – blended control signals - that
combine in a single signal discrete and continuous
features. Based on these signals, different access
methods can be designed. Beyond traditional
selection methods, these signals can potentially fill
the gap between scanning and direct access
methods, as discussed in Section 2. In fact, the
interaction described in Section 4.1-c) is neither
direct nor scanning.
From this study was demonstrated that users may
have potencial to generate control signals with more
information than just for a binary control, though not
enough to direct selection.
In progressive conditions, users experiment
different needs and abilities along different stages.
The more information the user interface can collect
from users' abilities, the faster may be the access to
AT systems.
The use of blended control signals, based on
user's electrophysiological signals, allows a better
adaptation to neurodegenerative conditions,
broadening the possibilities of ways of interaction
and enabling persons with severe neurodegenerative
disorders to interact more efficiently with AT
systems.
6 CONCLUSIONS
In this paper a case study demonstrating the use of
electrophysiological control signals by a young man
with ALS was presented.
The user was able to “upgrade” the control
signals by progressive steps. Starting by a binary
control signal using a scanning method, he was
progressively able to generate continuous control
signals, as well as combinations of these – blended
control signals. The case study here presented
clearly shows that other selection methods should
be sought taking advantage of the control signals
that this kind of users may be able to generate, in a
sense richer than binary signals, although poorer
than a continuous signal.
This kind of signals may provide more flexible
and efficient ways of interaction with AT systems, if
multimodal selection methods are designed.
Moreover, resorting to blended control signals, AT
systems may become more user friendly and
adaptable, reducing the rate of AT abandonment,
especially among people with neurodegenerative
conditions.
REFERENCES
Allanson, J., 2002. Electrophysiologically interactive
computer systems. Computer, IEEE Computer Society
Press; 35(3): 60-65.
Azevedo, L., 2006. A Model Based Approach to Provide
Augmentative Mobility to Severely Disabled Children
through Assistive Technology. PhD thesis.
Universidad del Pais Basco.
Cook, A., Hussey, S., 2001. Assistive Technologies:
Principles and Practice, Mosby. 2
nd
Edition.
Felzer, T., Nordmann, R., 2006. Speeding up Hands-free
Text Entry. Online-Proceedings of 3rd Cambridge
Workshop on Universal Access and Assistive
Technology (CWUAAT), Fitzwilliam College,
University of Cambridge. From: http://rehab-
www.eng.cam.ac.uk/cwuaat/proceedings06.html. May
2007.
Glennen, S., DeCoste, D., 1996. The Handbook of
Augmentative and Alternative Communication.
Singular Press.
HEALTHINF 2008 - International Conference on Health Informatics
258

Iacono, T., 1992. A Step Back from Small ‘N’ Design to
View the Total Picture. In: Gardner-Bonneau, D.J.,
Methodological Issues in Research in Augmentative
and Alternative Communication. Proceedings of the
Second ISAAC Research Symposium in
Augmentative and Alternative Communication: 12-16.
Junker, A., 1995. United States Patent, 5,474,082.
Londral, A., 2007. Tecnologias de Apoio para pessoas
com Esclerose Lateral Amiotrófica: estudo de
interfaces baseadas em sinais electrofisiológicos.
Master thesis. Universidade Católica Portuguesa.
Stevens, R.D., Edwards, A.D.N., 1996. An approach to the
evaluation of assistive technology. Proceedings of
Assets ’96; ACM: 64–71.
United States Congress, 1998. The Technology-Related
Assistance for Individuals with Disabilities Act of
1998. PL 100-407.
Wills, S. A. e MacKay, D. J. C., 2006. DASHER—An
Efficient Writing System for Brain–Computer
Interfaces? IEEE Transactions on Neural Systems and
Rehabilitation Engineering, Vol. 14 (2).
Wolpaw, J.R., Birmauer, N., McFarland, D.J.,
Pfurtscheller, G., Vaughan, T.M., 2002. Brain-
Computer Interfaces for communication and control.
Clinical Neurophysiology; 113: 767-791.
ELECTROPHYSIOLOGICAL CONTROL SIGNALS FOR PERSONS WITH NEURODEGENERATIVE CONDITIONS:
BLENDED CONTROL SIGNALS
259