The Use of Electroencephalogram and Electrodermal Signals in

Reinforcement Learning of a Brain-Computer Interface

Werley de Oliveira Gonçalves

, Gizelle Kupac Vianna

1,2

and Luiz Maltar Castello Branco

Programa de Pós-Graduação em Modelagem Matemática e Computacional, Instituto de Ciências Exatas,

Universidade Federal Rural do Rio de Janeiro, Seropédica, Brazil

Departamento de Computação, Instituto de Ciências Exatas,

Universidade Federal Rural do Rio de Janeiro, Seropédica, Brazil

Keywords: Brain-Computer Interface, Intelligent Control Systems, Pattern Recognition, Artificial Neural Networks,

Electroencephalogram Signal, Electrodermal Signal, Computational Intelligence.

Abstract: The objective of this work is to compare the performance of two brain-computer interfaces developed by our

research group. Both interfaces collect the electrical signals produced by the human body while a person try

to move a cursor on a digital screen, using only his thought. The collected signals are classified using the

artificial neural networks paradigm, where the first interface uses electroencephalogram signals, collected

from the scalp, to classify the mental command, and the second uses the electrodermal signal, collected from

any right-hand finger. Besides analysing the performance of the two approaches, this research contributes to

reduce the training time achieved by similar systems, reported in the literature as being in an average of 45

days, to about only 40 minutes. Our motivation is to facilitate the accessibility of people with temporary or

permanent physical limitations. In addition, we have developed a low-cost signal collection platform,

providing a solution that can help a large group of people.

1 INTRODUCTION

The limbic system has an important role in

controlling the human emotions such as motivation,

stress and rage (Boucsein, 2012a). The limbic system

integrates the sensory information from the

environment with the emotional state, where an

affective value is attributed to these stimuli, such as

fear or pleasure. A positive feedback signal is sent for

each action or intention of action performed correctly,

resulting in the reinforcement of some synapses. In

the other hand, wrong actions or intentions are

discouraged by a similar mechanism (Amaral, 2016),

(Nishida, 20126). In summary, the limbic system

generates a signal of approval or rejection for every

action we take, allowing a person to distinguish

among what he likes or dislikes.

Although the limbic lobe is located in the inner

part of the brain, from where it is very difficult to

collect signals through an electroencephalogram

(EEG), this system also controls the electrodermal

(EDA) response, which is the electrical signal present

on the skin and its glands. The EDA phenomenon is

spontaneous and results from changes induced by a

complex system of elements with different

electrophysical properties. Moreover, the skin can be

modeled by a set of resistors and capacitors, in

relatively simple way, the EDA system can be a fast,

low cost and low stress training method for brain-

computer interface (BCI) applications (Boucsein,

2012a), (Blain, 2008).

On the other hand, another important brain

interface is the EEG, whose responses are stationary

in nature and vary at each recording session. The

procedure for collecting EEG signals uses external

electrodes and it is safe, inexpensive, non-invasive,

with a satisfactory time resolution for most BCI

studies and applications (Leskov, 2000), (Iacoviello,

2015). The captured signals are a composition of

many electrical signals emitted by the human body,

which means that some unwanted signals may be

captured too. Fortunately, those useless signals can be

easily eliminated through specific filters (Noteboom,

2001).

Applications based on EEG signals could allow an

interaction between the environment and people,

translating their imaginary movements into electrical

signals. The construction of a limbic signal translator

Gonçalves, W., Vianna, G. and Branco, L.

The Use of Electroencephalogram and Electrodermal Signals in Reinforcement Learning of a Brain-Computer Interface.

DOI: 10.5220/0006773305330539

In Proceedings of the 20th International Conference on Enterprise Information Systems (ICEIS 2018), pages 533-539

ISBN: 978-989-758-298-1

533

system can provide a wide range of home automation

applications, such as control systems for switching

electric household appliances, or for similar use in

hospitals.

In some researches, the BCI system has proven to

be a promising tool for applications that help people

with severe motor limitations and for the

implementation of remote medical devices (Lin,

2016) (Boucsein, 2012a). Based on these premises,

our work aims to reduce the training time needed to

customize the translation of the limbic signal to

people with limitations in their motor capacities,

whether temporary or permanent. This can be useful

in situations where a person cannot wait a long time

until he can start using such a system, as during a

hospital stay.

The paper is organized as follows. In section 2 we

present the state art of approaches for modeling of

control system based on brain signals in BCI

applications. Section 3 provides a description of our

control system and the steps of the developed

algorithm since collecting the signals until the final

movement of the cursor on the screen. Subsequently,

in section 4 we conclude with a discussion of our

results. Finally, section 5 describe future research

directions.

2 MODELING OF CONTROL

SYSTEM

2.1 Electrodermal Activity

It has already been proven that the EDA signal (also

known as galvanic skin response) can be used in BCI

applications (Blain, 2008). We can consider the skin

as a set of resistors and capacitors, where the glands

are represented by a voltage source or by charged

capacitors. In 1966, Montazu and Coles (Nishida,

2016) proposed an electric model of the skin, which

can be represented by (1):























 (1)

In (1), resistor R1 represents the equivalent

resistance located in the dermis, resistor R2 models

the resistance of the outermost layer of the skin, and

resistance Rtot models the value of all sweat glands.

It has already been found that many lesions of the

spinal cord do not prevent the EDA signal from

remaining present. In individuals with lesions below

T8, the EDA signal can be detected in both hands and

feet, but for lesions between T4 and T8, the signal is

only present in the hands (Boucsein, 2012b).

2.2 Electroencephalogram Signals

We detected the EEG signals based on the

international system 10-20, which divides the skull

into 21 points (Plonsey, 1995). The signals captured

by the EEG are composed by brain signals combined

with several other electrical signals emitted by the

human body. However, we are only interested in the

signals that reflect the intentions of the user and we

need to eliminate everything that is considered as

noise. For the procedure of signal filtering, we choose

the discrete Fourier transform (Najarian, 2006).

2.3 The Μ Wave

The 8-12 hz wave and 12-30 (beta waves) are directly

related to the motor regions of the cerebral cortex,

what give them a great potential in BCI applications

(Zhao, 2015). The main advantages of using μ waves

are their capability of training a user to control the

amplitude of these waves, and the fact that the

muscular movements cannot interfere in the

amplitude of these waves.

Wolpaw (1991) has already shown that a person

can be trained to use the brain waves of 8-12hz to

move a one-dimensional cursor. The author has

developed a system in which a user moves a cursor

vertically to reach a moving target. Five volunteers

participated in an experiment that collected the

signals with frequency of 3hz and subdivided their

amplitude into 5 bands of μV. The amplitude of the

collected signal is used to proportionally increase or

decrease the size of the cursor offset. However,

despite achieving a success rate of almost 90%, the

training time for correct use of μ waves was up to

three months.

Jun (Jun, 2015) collected μ waves from the two

cerebral hemispheres and considered them as binary

data, combined two by two, forming six different

combinations. Each combination is used to identify

commands that activate a mechanical arm (Jun,

2015).

3 MATERIAL AND METHODS

The purpose of this paper is to classify the electrical

signals emitted by the brain and use then to vertically

move a cursor on a digital screen. Two types of

electrical signals were analysed to evaluate which one

has the best performance in the purposed task. In

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

534

both, an ANN is used to classify the signal and to

decide whether the cursor should go up or down.

In the experiment using EEG signals, these are

collected according to the international 10-20

mapping system, which specifies that the electrodes

should be attached to the C3 or C4 positions of the

skull (Figure 1).

Figure 1: The international 10-20 mapping system of the

skull (Wikipedia, 2018).

During the initial testing phase, we verified the

need for signal amplification and filtering. So, we

opted for the Chebychev bandpass filter and shielded

the circuit to eliminate the noises, including those

generated by electrical installations.

The collected signal was analog but we

discretized in the frequency domain. For the

discretization, we opted for the discrete Fourier

transform, obeying the Nyquist criterion, where the

sampling frequency must be at least twice as high as

the highest frequency present in the original signal

(Oliveira, 2017).

In the experiment using EDA signals, they can be

collected through any hand finger (Plonsey, 1995),

(Spliter, 2006).

3.1 Data Treatment and Analysis

Our experiments have shown that feelings of

frustration, anxiety and nervousness decrease the

amplitude of the EDA signal, while feelings of

surprise and satisfaction increase their amplitude. In

both cases, this variation is almost instantaneous,

occurring soon after the generation of the pulse. After

this change, the signal converges slowly to its base

value.

At the first step of the EDA experiment, we

generate labelled samples of each type of signal to

obtain a set of data for the ANN's training phase. The

samples are built by classifying each collected signal

as neutral, frustration or surprise (when the cursor did

not move, move in the wrong direction or move in the

correct direction, respectively). The EEG experiment

was conducted in a similar way.

For the signal collection, an intermittent message

is positioned at the top or bottom of the screen,

alternately, and the volunteer should try to move the

cursor to the indicated direction, using only his

thought. Each collection set results in 200 samples,

divided between 100 thoughts of rise and 100 of

descent.

After performing a relevance analysis of each

frequency collected, in each type of signal, the

frequencies of 8, 10, 12, 16, 18 and 20hz were

selected for the two types of signal (EDA and |EEG),

where the 10 and 12hz are the best all, since they are

found within the spectrum of waves μ.

For the ANN, we chose a backpropagation MLP,

a model that is proven to be suitable for pattern

recognition problems. We tested different

combinations of neural network parameters and

architectures, where each configuration was tested 50

times. The model with the best performance was a 4-

2-1 network, using a quasi-Newton Broydon-

Fletcher-Goldfarb-Shanno (BFGS) function method

to calculate the minima of a multi-variable objective

function (Mathworks, 2016), learning rate of 0.6 and

momentum rate of 0.9. The network inputs were the

frequencies 8, 10, 12, 16, 18 and 20hz and the output

was set to 0 (for down moves) or 1 (for up

movements). For each individual training conducted,

samples were randomly separated at a rate of 80% for

training and 20% for testing.

For both signal type (EEG and EAD), the

necessary steps, from collecting the signals to the

final movement of the cursor on the screen, can be

summarized by the following algorithm:

1. System training: an initial collection of signals

is conducted and each of them is stored with

its classification, UP or DOWN. At this stage,

the cursor does not move, and the goal is only

to generate a personal signal pattern for the

user.

2. Training of an initial neural network RNA0:

using the labelled signal samples generated in

phase 1, and neural network with the

architecture and parameters defined as

explained in section 3.1, the goal is to train a

personalized network for the user. Fifty

complete trainings were performed in the

neural network RNA0 to select the one that

presented the best performance, using a class

separation threshold of 0.5.

3. Collection of signal: A new collect occurs and

now each of them will be classified by the

The Use of Electroencephalogram and Electrodermal Signals in Reinforcement Learning of a Brain-Computer Interface

535

RNA0 and the cursor will be moved

accordingly.

4. Training of neural network RNA1: Training of

a new neural network, RNA1, using the

signals generated in phase 3, with the same

architecture, parameters and class separation

threshold of RNA0, to provide an updated

network for the user, assuming that his ability

should have improved after the initial

experiment. Again, 50 complete trainings

were performed in RNA1 to select the best

performance one.

5. Steps 3 and 4 are repeated twice until the

RNA3 is trained, which corresponds to four

test cycles.

4 RESULTS AND DISCUSSION

The experiment lasts, on average, 35 minutes and

Table 1 shows the performance obtained at each step

of the algorithm, for the EEG signal. From its

analysis, we can verify that the best performance is

found at step 2, when RNA1 is used to classify the

second set of signals, and after that it tends to

decrease. We can explain this by the intrinsic

characteristics of the experiment, which caused a

certain mental fatigue to all volunteers. In special, for

volunteers 3 and 7 the performance of RNA3 was

lower than RNA0, maybe reflecting their state of

tension, who were visibly worried through all the

experiment. A similar experiment, using the previous

algorithm, but the EDA signal instead of EEG, was

conducted and the results are shown in Table 2.

In Figure 2, we show the distribution curves of the

EEG signals collect at each phase of the experiment,

from volunteer #3. From the analysis of these graphs,

we can see that the distributions of the two types of

mental commands are indeed distinct, especially

during the first and the second phases. After that, the

performance begins to fall, as shown by the greater

approximation between the curves, increasing the

area of confusion between the signal patterns.

In Figure 3, we show the distribution curves of the

EDA signals collected in each phase of the

experiment, from the volunteer # 5. Differently from

the distribution of EEG signals, the EAD distributions

show that, for this type of signal, the performance

continues to improve as new trainings are conducted.

Table 1: EEG recognition performance during the training phase of the neural networks.

Volunteer

%Hints of

ANN

%Hints of

ANN

%Hints of

ANN

%Hints of

ANN

56.52

78.26

69.57

60.87

73.91

78.26

82.61

73.91

60.87

78.26

65.22

73.91

60.87

65.22

86.96

78.26

65.22

82.61

73.91

69.57

73.91

60.87

78.26

73.91

82.61

78.26

73.91

69.57

78.26

Table 2: EDA recognition performance during the training phase of the neural networks.

Volunteer

%Hints of

ANN

%Hints of

ANN

%Hints of

ANN

%Hints of

ANN

78.94

47.37

78.95

73.68

63.16

84.21

63.16

89.47

84.21

78.95

65.52

82.76

65.52

62.07

75.86

82.76

100

84.21

73.68

78.95

82.36

85.29

70.59

67.65

61.76

82.35

85.29

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

536

(a)

(b)

(c)

(d)

Figure 2. Distribution of EEG signals for the thoughts of rise (green) and fall (red) of volunteer #3.

(a)

(b)

(c)

(d)

Figure 3: Distribution of EDA signals for the thoughts of rise (green) and fall (red) of volunteer #5.

The Use of Electroencephalogram and Electrodermal Signals in Reinforcement Learning of a Brain-Computer Interface

537

5 FUTURE WORK

The EDA signal is controlled by the limbic system

(Fausett, 1994), which also generates the approval

and disapproval responses, defining our choices and

actions (Boucsein, 2012b). Proper modelling of this

behaviour can generate interesting solutions for

people with such severe physical limitations that they

cannot express their needs and feelings. Since the

electronic circuit used in this work has a low cost, the

use of EDA can, more quickly than conventional BCI

using EEG, generate solutions that reach a larger part

of the population.

Other studies found in the literature related the use

of the EDA signal to correct the commands generated

by the EEG signal (Boucsein, 2012a), but we have

shown in this work that the training time of a BCI

application can be reduced by using the EDA signal

instead of the EEG. In addition, the technology

developed by our research group, which included the

design and development of a custom acquisition

circuit, can reduce the cost of this type of BCI

application, opening possibilities for its use in other

fields of research. While a wifi EEG headset plus

electrodes could cost almost U$800.00, an EDA

detector can be bought by only U$10.00.

The choice for the ANN paradigm for signal

recognition was also a good decision. As we

predefine the network architecture and training

parameters, and parameterize the training process,

potential users of our BCI system do not need any

technical knowledge to learn how to use it.

For a future work, it will be interesting to explore

the limits of the EDA signal applied to BCI, such as

collecting EDA signals from more than one region,

for example, from the right and left hand at the same

time. The combination of these signals could increase

the variety of responses and, consequently, the

number of possible BCI applications. Tsukahana

(2002) presents another approach for electrodermal

signal codification, generating more than one binary

signal to increase the choices of movements for the

user.

REFERENCES

Amaral, J. R. Homepage, http://www.cerebromente.org.br/

n05/mente/limbic.htm, last accessed in 10/21/2016,

2016.

Blain, S.; Mihailidis, Al.; Chau, T. Assessing the Potential

of EDA Activity as an Alternative Access Pathway. In

Medical Engineering Physics, pp 498-505, 2008.

Boucsein, W. Principles of Electrodermal Phenomena in:

Eletrodermal Activity. Second Edition: New York:

Springer, pp. 1-76, 2012a.

Boucsein, W. EDA Activity, Second Edition. In: Springer

Science+Business Media, 2012b.

Fausett, L. Fundamentals of Neural Networks –

Arquitetures, Algorithms and applications, 1nd Edition,

Prentice-Hall, pp. 1-37, 59-76, 1994.

Iacoviello, D.: A Real-time Classification Algorithm for

EEG - Based BCI driven by Self-induced emotions. In:

Computer Methods and Programs in Biomedicine, vol.

122, pp. 293–303, 2015.

Jun, J.; et al.: A novel Morse code inspired method for

multiclass motor imagery brain–computer interface

(BCI) design. In: Computers in Biology and Medicine,

pp. 11–19, 2015.

Leskov, I. F., Lopes, H.: Amplificador e Filtros de Baixo

Custo para Sinais de EEG (Eletroencefalografia). In:

Simpósio Internacional de Iniciação Científica da USP,

São Carlos. XVII Congresso de Iniciação Científica e

Tecnológica em Engenharia, 2000.

Lin, J. S., Hsieh, A.: A Wireless BCI-Controlled Integration

System in Smart Living Space for Patients. In: CH.

Wireless Pers Commun, vol. 88: 395. doi:10.1007/

s11277-015-3129-0, 2016.

Mathworks. Homepage: https://www.mathworks.com/

help/nnet/ug/multilayer-neural-network-

architecture.html, last accessed in 2016/08/15.

Najarian, Kayvan; Splinter, Robert. (s.d.).

Electroencephalogram in: Biomedical Signal and

Image Processing, CRC Press – Taylor & Francis

Group, pp 197-216, 2006.

Narendra, Kumpati S. Adaptative Control Using Neural

Networks. In: SUTTON, Richard S.; WERBOS, Paul

J.; MILLER III, W. Thomas,“Neural Networks for

Control”, MIT Press, vol. 211996. pp.115–142.

Nishida, S. M.: Neurobiologia das emoções - SISTEMA

LÍMBICO, Homepage, http://www.ibb.unesp.

br/Home/Departamentos/Fisiologia/Neuro/aula27.siste

ma_limbico_silvia.pdf, last accessed in 10/28/2016,

2016.

Noteboom, J. T.: Activation of the arousal response can

impair performance on a simple motor task. In: Journal

of Applied Physiology, vol. 91, pp.:821-831, 2001.

Oliveira, K. S., Homepage: http://astro.if.ufrgs.br/

med/imagens/fourier.htm, last accessed 06/28/2017,

2017.

Plonsey, J. M.: Bioelectromagnetism. In: J. Malmivuo,

Bioelectromagnetism - Principles and Applications of

Bioelectric and Biomagnetic Fields, New York: Oxford

University Press, cap 13, 1995.

Spliter, K. N. In: Biomedical Signal and Image Processing,

CRC Press - Taylor & Francis Group, pp 79-99, 2006.

Tsukahana, Reiko, Aoki, Hisashi. Skin potential response

in letter recognition task as an alternative

communication channel for individuals with severe

motor disability. Clinical Neurophysiology 2002, pp

1723–1733.

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

538

Wikipedia, 10-20 System (EEG), Homepage:

https://en.wikipedia.org/wiki/10%E2%80%9320_syste

m_(EEG). Last accessed in 01/18/2018.

Wolpaw, J. R.: An EEG-Based Brain-Computer Interface

for Cursor Control. In: Eletroencephalography and

Clinical Neuro-physiology, pp. 252-259, 1991.

Zhao, J., Li, W., & Li, M. Comparative Study of SSVEP-

and P300-based Models for the Telepresence Control of

Humanoid Robots. PLoS ONE 10(11): e0142168.

doi:10.1371/journal.pone.0142168, 2015.

The Use of Electroencephalogram and Electrodermal Signals in Reinforcement Learning of a Brain-Computer Interface

539