TOWARD DOMOTIC APPLIANCES CONTROL

THROUGH A SELF-PACED P300-BASED BCI

F. Aloise

, F. Schettini

, P. Aricò

, F. Leotta

, S. Salinari

, D. Mattia

F. Babiloni

1,3

and F. Cincotti

Neuroelectrical Imaging and BCI Lab, Fondazione Santa Lucia IRCCS, Rome, Italy

Deparment of Computer Science and Engineering, University of Rome “Sapienza”, Rome, Italy

Department of Human Physiology and Pharmacology, University of Rome “Sapienza”, Rome, Italy

Keywords: Brain Computer Interface, P300, Domotic, EEG, Asynchronous Control.

Abstract: During recent years there has been a growing interest in Brain Computer Interface (BCI) systems as an

alternative means of interaction with the external world for people with severe motor disabilities. The use of

the P300 event-related potentials as control feature allows users to choose between various options (letters

or icons) requiring a very short calibration phase. The aim of this work is to improve performances and

flexibility of P300 based BCIs. An efficient BCI system should be able to understand user's intentions from

the ongoing EEG, abstaining from doing a selection when the user is engaged in a different activity, and

changing its speed of selection depending on current user's attention level. Our self-paced system addresses

all these issues representing an important step beyond the classical synchronous P300 BCI that forces the

user in a continuous control task. Experimentation has been performed on 10 healthy volunteers acting on a

BCI-controlled domestic environment in order to demonstrate the potential usability of BCI systems in

everyday life. Results show that the self-paced BCI increases information transfer rate with respect to the

synchronous one, being very robust, at the same time, in avoiding false negatives when the user is not

engaged in a control task.

1 INTRODUCTION

A BCI system allows to control simple devices,

including communication facilities, without using

muscles and peripheral nerves (Wolpaw et al. 2002);

in particular an EEG based BCI system is able to

understand user's intentions translating his brain

activity into a control signal (Wolpaw and

McFarland, 2004). Progresses made in recent years

have brought these systems to be considered as an

alternative means of communication and control in

situations where environmental conditions can

transiently compromise the mobility of the user, and

not only for people with severe disabilities. The

P300 event-related potential is typically a large and

positive deflection in the EEG activity (ca. 10-

20µV) with a latency ranging between 250 and 400

ms (Polich et al. 1995, Fabiani et al.1987). It is

elicited when the subject recognizes a particular

stimulus (Target stimulus) presented within a train

of frequent stimuli (NoTarget stimuli). An average

of several epochs related to a specific target stimulus

is required to distinguish the P300 potential from the

spontaneous EEG activity. For this reason classic

P300 based BCI systems provide a well-defined

number of stimulation repetition at the end of which

a selection is made; this last issue is the cause of

obvious drawbacks because the user is continuously

engaged in controlling the interface and his

distractions produce wrong classifications.

Moreover, the number of stimuli repetitions needed

to make a selection (and therefore time) depends on

the user's attention level; few sequences are needed

if the user is very concentrated, but fatigue or

distractions may cause a significant decrease in

performance and in this case it is preferable that the

system refrains from making selections avoiding a

wrong classification. This work presents a

methodology for the classification of EEG signals

related to P300 potentials which allows the user to

divert attention from the stimulation interface,

suspending the control, without incurring in a wrong

239

Aloise F., Schettini F., Aricò P., Leotta F., Salinari S., Mattia D., Babiloni F. and Cincotti F..

TOWARD DOMOTIC APPLIANCES CONTROL THROUGH A SELF-PACED P300-BASED BCI.

DOI: 10.5220/0003162002390244

In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-2011), pages 239-244

ISBN: 978-989-8425-35-5

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

classification (selection). In order for the BCI

systems to move from research labs to people's

homes it is absolutely necessary that they are able to

recognize user's state without any outside input; in

particular they have to detect user’s control state or,

in other words, to distinguish when the user intends

to operate through a control interface from when he

is engaged in a different task and therefore to avoid

making any selections. This is why in recent years

many research groups have been involved in the

problem of a self-paced BCI, both for the motor

imagery (Mason and Birch, 2000; Millan and

Mourino, 2003; Townsend et al, 2004) and for the

P300-based systems (Zhang et al, 2008).

In this work we show a simple and fast self-

paced system based on an heuristic method; we

allow the system to recognize user's status,

introducing thresholds in the classifier. Moreover the

choice of Target stimulus become dynamic and this

allows system to improve its selection speed

depending on the user’s ability and attention level.

For this reason particular attention will be also put

on system performances in terms of information

transfer rate (ITR) and accuracy. Another important

thing to emphasize is that we conducted our

experiments in an environment completely operated

by the BCI system (Cincotti et al. 2008) and we

based user’s task on real life situations. This is

because we want to demonstrate the feasibility of

using these systems in everyday life.

2 MATERIALS AND METHODS

For the aim of acquisition protocol we investigated

the use of P3Speller application (Farwell and

Donchin, 1988) provided with the BCI2000

framework (Schalk et al. , 2004) to control an home

automation system. We organized 8 different

stimulus classes in a 4 by 4 matrix, each consisting

of a row or a column.

Matrix elements were 16 simple B&W icons (in

order to minimize the variability and possible VEPs

due to a high variety of colors and load information

of each icon) representing the actions that the user

could perform on the environment (e.g. light control,

DVD player, webcam for remote monitoring, mobile

phone and opening the door). Stimulation consisted

in a random intensifications of each stimulus class,

with a duration of 125ms each one. Inter Stimulus

Interval (ISI) was set to 125ms, so 250ms lag

between two stimuli.

We distinguished two different states in which

the user can be: the Control State, during which

he/she was attending to the stimulation because he

intended to exercise control over the surroundings

through the interface; and the NoControl State,

during which the user was engaged in another task

and then he wanted the system refrained to make

decisions. The EEG signal was reorganized in

overlapping epochs representing the 800 ms time

intervals immediately following the onset of each

stimulus; we can distinguish the epochs acquired

during Control trials in Target Epochs and NoTarget

Epochs. Target Epochs relate to the onset of the row

or the column stimulus containing the icon that the

user intends to select, while NoTarget are related to

no relevant stimuli. Then epochs were grouped into

sequences; a sequence denotes a single presentation

of each stimulus class on the control interface; in

this case a sequence consisted of 8 epochs, one for

each stimulus class of the interface. A single

sequence lasted 2 seconds. With the term Trial we

refer to a set of sequences at the end of which a

selection is made. Between 2 Trials we took 4

seconds during which the system presented to the

users the Target icon. All the icons on the interface

were proposed as a Target to the subject with the

same frequency and this because we wanted that

each stimulus was equally likely for subsequent

analysis. Finally, a Run consists of a series of trials

at the end of which data acquisition is stopped.

2.1 Data Acquisition Protocol

10 volunteers participated to data acquisition

protocol (4 female and 6 male) aged between 23 and

38 years; 5 subjects had already experience with the

BCI systems. Scalp EEG data were acquired from

each subject during BCI sessions using the

g.MobiLab device from g.Tec (Austria, 256Hz). The

EEG was recorded from 8 Ag-AgCl electrodes : Fz,

Cz, Pz, Oz, P3, P4, PO7 and PO8. This channel set

represents the union of the classical channels used to

extract the P300 response (Fz, Cz and Pz) with

channels in the posterior regions that have strong

correlation with desired matrix target (Krusienski et

al. 2006).

Each subject completed 2 recording sessions, we

used collected data for subsequent off-line analysis

and during recording session subject had not any

feedback about classification results. The first

recording session included a total of 8 runs, the first

2 compounded of 8 trials each during which the

subject was asked to always exercise control on the

interface; the number of stimulation sequences per

trial was fixed a priori to 10. Over the last 6 runs

Control Trials and NoControl Trials were alternated

BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

240

for 10 trials in total. During the NoControl Trials the

subject was asked to focus the eyes on a fixation

cross in the center of the stimulation interface. This

is because we wanted that the subject was not

completely immune to the stimulation even when he

was in the NoControl state. It is assumed that a BCI

user can’t move his head or eyes to ignore the

stimulation, so it is essential that the NoControl state

is robust to random stimuli which the user does not

actually paying attention to.

The second session provided 2 runs in Control

State to verify the subject's attitude to control the

interface using the P300 potential, and 6 runs in

which Control trials and NoControl trials were

alternated, however we considered different

NoControl tasks; in fact, the last 6 runs can be

divided into 2 groups of 3 runs each:

 Control and View: The subject during the

NoControl Trials was watching a video on the

other half of the screen.

 Control and Answer: During the NoControl

trials the subject had to do computation as

quickly as possible looking at the fixation cross

at the center of the interface.

Following this approach the data set contains

NoControl trials representing real situations in

which the user turns his attention elsewhere or

speaks with another person.

2.2 Features Extraction

and Classification

Signal preprocessing is necessary before perform the

classification. In particular, the EEG signal was

divided into overlapped epochs of 800 ms starting

from the onset of each stimulus, and then each epoch

was down-sampled at 85 Hz. Such down-sampling

reduces the data size and at the same time speeds up

the ensuing EEG processing significantly. Then,

EEG epochs were reorganized into a three-

dimensional array: each 2D matrix of the array

represents a single epoch related to a single stimulus,

where rows represent acquisition channels and

columns correspond to samples of each epoch.

Despite this first preprocessing the amount of

data was still significant and a further reduction in

features space was performed using the Stepwise

Linear Discriminant analysis (SWLDA). Stepwise

Linear Discriminant analysis (Draper and Smith,

1981) is an extension of Fisher’s linear discriminant

(FLD, Fisher 1936) that performs feature space

reduction by selecting suitable features to be

included in the discriminant function. Farwell and

Donchin first introduced this method for classifying

P300 features into EEG signal (Farwell and

Donchin, 1988); Krusienski et al. confirmed that this

simple technique is really efficient for online

communication (Krusienski et al., 2008). We ran the

stepwise function on a testing data set including

NoControl trials. We assigned a label equal to zero

to the NoTarget and NoControl epochs while label

was equal to 1 for Target epochs. Using SWLDA,

the final discriminant function was restricted to

contain a maximum of 60 features. Nonzero weights

were assigned to these features, w. Then the scores

values for each epoch were calculated as:





=∙





+



(1)

Where i denote all features related to single

stimulus j. It is assumed that a P300 is elicited for

one of the four row/column intensifications during

Control trials, and that the P300 response is invariant

to row/column stimuli, the resultant classification is

taken as the maximum of the scored feature vectors

for the respective rows, as well as for the columns:

 =max







 =max







The icon that appears at the intersection of the

predicted row and column in the matrix is the one

chosen.



= 



 

2.3 Threshold Values

Self-paced control is based on the introduction of

some thresholds in the classifier; the classification

was performed as explained before but the system

will refrain from making a selection until a row and

a column scores exceeds the threshold value. The

threshold values were chosen through a procedure

that relies on the use of ROC curves. In particular

we calculated the scores value on the Target,

NoTarget and NoControl epochs. A normal

distribution well fit the scores distributions and to

confirm this we ran a t-test on the 3 different score

distributions for each subject. T-test results show

that the hypothesis of normal distribution is true

with 95% confidence level. Next step was

investigating if the 3 score's distributions can be

considered different; for this reason we ran the

Kolmogorov-Smirnov test on each pair of samples.

Table 1 report the results of this test; the hypothesis

of different distribution was confirmed with the 95%

TOWARD DOMOTIC APPLIANCES CONTROL THROUGH A SELF-PACED P300-BASED BCI

241

confidence level for all subjects except one (Subject

1: No Target vs NoControl). For this reason it is

necessary to take account of NoControl trials for

thresholds extraction.

Table 1: Kolmogorov-Smirnov test on each pair of

sample.

Target vs

NoTarget

Target vs

NoControl

NoTarget vs

NoControl

SUBJ 1 8,75E-192 9,65E-196 0,44287182

SUBJ 2 1,74E-154 1,84E-136 6,99E-04

SUBJ 3 8,52E-256 6,38E-205 2,01E-21

SUBJ 4 2,74E-213 1,79E-182 2,82E-11

SUBJ 5 1,24E-198 3,95E-159 2,48E-10

SUBJ 6 1,81E-255 3,42E-207 9,35E-19

SUBJ 7 1,77E-277 2,77E-241 2,67E-23

SUBJ 8 2,34E-177 4,52E-134 8,90E-22

SUBJ 9 1,04E-147 2,30E-107 2,39E-13

SUBJ 10 6,67E-178 1,14E-126 1,31E-24

The threshold values were chosen according to the

number of stimulation sequences accumulated in the

trial. In fact the scores for the general stimulus i at

the sequence s will be defined as:







=





+





=1,2,…

= 1,2,…

(2)

Where 





is given by (1), Ns = 8 for the domotic

interface and the number of stimulation sequences

was fixed to 10.

Subsequently, for each sequence we looked for the

maximum score of the row stimuli and the

maximum score of the column stimuli, and then to

them was assigned a label equal to 1 if the maximum

scores were relative to a Target stimulus and equal

to 0 if it referred to NoTarget or NoControl stimuli.

In this way we are sure to include the maximum

score values related to NoControl trial in ROC

curves training, so threshold values taking into

account possible artifacts that may occur when the

subject was not engaged in BCI control. Now ROC

curves can be plotted for each sequence using the

corresponding scores. An example is shown in

figure 1 where it is evident that when the number of

sequences accumulated in the trial increases the

ROC curves assume an ideal trend. Finding a

tradeoff between false positive rate and false

negative rate, is necessary in order to choose the

threshold. We have chosen to set the maximum

False Positive Rate (FPR) to 0.05 and the lowest

True Positive Rate (TPR) to 0.5, so the threshold

will be chosen at the intersection of the ROC curve

to the straight line joining points (0.1) and (0.05 and

0.5) .

Figure 1: Area under the ROC curve: thresholds are

identified from a tradeoff between the True Positive Rate

(yellow area) and False Positive Rate(blue area). Figure

shows only the first five stimulation sequences, over the

trend is approximately the same.

3 RESULTS

This section describes the results obtained through

the off-line analysis performed on data acquired

during the 2 recording sessions. Then we compare

our self-paced system with a classic P300 based BCI

in terms of information transfer rate considering

only the Control trials.

3.1 Off-line Analysis

An off-line cross validation was performed on the

data collected during recording sessions. In

particular, we divided the data into a training data

set consisting of 3 runs from the first session and 4

runs from the second session. In this way we

included in the training data set NoControl trials

related to all 3 different NoControl tasks. The

remainder of the data set was used as a test data set.

Specifically, the train data set was used for features

extraction and to select the threshold values. The

figure 2 shows the results obtained in cross

validation.

There are 5 different classification outcomes,

depending on the user’s state.

During Control trials we can distinguish

between:

 Correct Classification: the target is

correctly recognized;

 Wrong Classification: there’s a target

misclassification;

 Missed Classification: the thresholds are

never exceeded, for this reason the system

abstains from take a decision.

During NoControl trials possible classification

outcome can be:

BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

242

 Abstention: the system properly refrain

from taking decisions;

 Missed Abstention: the thresholds are

exceeded and the system wrongly makes a

choice.

Figure 2: Results of offline cross validation for Control

and No-Control task.

Figure 2 also shows the averaged number of

sequences needed to make a selection when the user

was in the Control state. From the graph it can be

see how the system was proved to be robust during

NoControl trials in fact Abstentions reached an

average of 98%. On the other hand there is a high

percentage of abstentions during the Control trials

(average 8,4%), this may seem an inconvenience but

it really represents the system's ability to avoid

misclassification, because the percentage of wrong

classification did not exceed an average of less than

3%. We used the first 2 runs of the first 2 sessions,

in which the subject was always in a Control state, to

assess his ability with a classic P300-based BCI. We

found the ‘optimal’ number of stimulation sequences

just doing a cross validation offline: particularly we

used 2 runs to train SWLDA and to extract

significant features and the other 2 to test these

parameters, then we averaged the results of

classification for each possible combination of

training and testing data set. The figure 3 shows the

trend of the percentages of correct classification

based on the number of stimulation sequences

accumulated for each subject. The black line

represents an accuracy of 95%, which corresponds

to a false positive rate of 5% that is the maximum of

false positive allowed in the self-paced system

through ROC curves. We used these results to

estimate the information transfer rate.

Figure 3: Results of offline cross validation for Control

task depending on number of stimulation sequences.

3.2 Information Transfer Rate (ITR)

To assess the efficiency of the two systems in terms

of information transfer rate we used the definition of

bit rate given by Wolpaw et al.(2000) and widely

used in BCI systems, this is based on the definition

of information rate proposed by Shannon for noisy

channels with some simplifying assumptions: the

symbols have all the same a priori occurrence

probability =1/



, the classifier accuracy P is the

same for all target symbols and that the

classification error 1 is equally distributed

amongst all remaining symbols.





=log







+log



+



1



log







1







1



(3)

This express the bit rate or bit/trial for each

selection. The information transfer rate (bits/minute)

is equal to B

Wolpaw

multiplied by speed of selection S

(Selection per minute). In turn the speed selection

for P300-based system depends on the number of

sequences of stimulation used and when we

calculated it we have taken account of the 4 seconds

between a trial and the other which are used to

present the results of the classification. The Table 2

shows the values of ITR for each subject calculated

using the average number of sequences and the

percentage of accuracy obtained by off-line analysis

both for self-paced BCI and for the synchronous

one. In particular, for the synchronous system we

imposed the number of stimulation sequences that

allowed subject to achieve 95% of accuracy, if this

did not happen we have used the minimum number

of sequences that produced the highest accuracy.

TOWARD DOMOTIC APPLIANCES CONTROL THROUGH A SELF-PACED P300-BASED BCI

243

Table 2: ITR in both modalities.

SYNC SELF

SUBJ 1 12,39 4,24

SUBJ 2 5,08 11,37

SUBJ 3 16,81 16,36

SUBJ 4 12,01 11,87

SUBJ 5 11,03 18,00

SUBJ 6 16,81 15,00

SUBJ 7 14,01 16,06

SUBJ 8 5,98 15,38

SUBJ 9 4,67 10,17

SUBJ 10 5,63 11,83

Mean 10,44 13,03

4 CONCLUSIONS

The introduction of a threshold based classification

system in the P300-based BCIs allows the user to

divert his attention from control interface at any time

and without the use of external inputs, and it also

brings positive effects on the bit rate that is

incremented when the user is in the best control

conditions. A further advantage consists in

increasing the accuracy of the system by preventing

errors through abstentions; in this way the BCI

system acquires more dynamicity and flexibility by

reducing its gap with traditional input interfaces.

Future applications could consider the use of

dynamic thresholds fitting the user's current action

or the environment state: the system would be able

to automatically identify the user's most likely action

and facilitate its selection by reducing the threshold

values for that item. This work represents a step

towards the use of the BCI systems as an aid for

functional communication and environmental

control for people with severe motor disabilities or

as alternative means when the usual channels of

communication and interaction are temporarily

compromised.

ACKNOWLEDGEMENTS

This work is partly supported by the EU grant FP7-

224332 “SM4ALL” project, and FP7-224631

“TOBI” project. This paper only reflects the authors’

views and funding agencies are not liable for any use

that may be made of the information contained

herein.

REFERENCES

Cincotti, F. et al., 2008. Non-invasive brain-computer

interface system: towards its application as assistive

technology. Brain Research Bulletin, 75(6), 796-803.

Draper, N. R. & Smith, H., 1998. Applied Regression

Analysis Third., Wiley-Interscience.

Fabiani, M. et al., 1987. Definition, identification and

reliability of measurement of the P300 component of

the event-related brain potential. , 2, 1-78.

Farwell, L. A. & Donchin, E., 1988. Talking off the top of

your head: toward a mental prosthesis utilizing event-

related brain potentials. Electroencephalography and

Clinical Neurophysiology, 70(6), 510-523.

Fisher, R. A., 1936. The Use of Multiple Measurements in

Taxonomic Problems. [[Annals of Eugenics]], 7, 179–

188.

Krusienski, D. J. et al., 2008. Toward enhanced P300

speller performance. Journal of Neuroscience

Methods, 167(1), 15-21.

Krusienski, D. J. et al., 2006. A comparison of

classification techniques for the P300 Speller. Journal

of Neural Engineering, 3(4), 299-305.

Mason, S. G. & Birch, G. E., 2000. A brain-controlled

switch for asynchronous control applications. IEEE

Transactions on Bio-Medical Engineering, 47(10),

1297-1307.

Millán, J. D. R. & Mouriño, J., 2003. Asynchronous BCI

and local neural classifiers: an overview of the

Adaptive Brain Interface project. IEEE Transactions

on Neural Systems and Rehabilitation Engineering: A

Publication of the IEEE Engineering in Medicine and

Biology Society, 11(2), 159-161.

Polich, J. & Kok, A., 1995. Cognitive and biological

determinants of P300: an integrative review.

Biological Psychology, 41(2), 103-146.

Schalk, G. et al., 2004. BCI2000: a general-purpose brain-

computer interface (BCI) system. IEEE Transactions

on Bio-Medical Engineering, 51(6), 1034-1043.

Townsend, G., Graimann, B. & Pfurtscheller, G., 2004.

Continuous EEG classification during motor imagery--

simulation of an asynchronous BCI. IEEE

Transactions on Neural Systems and Rehabilitation

Engineering: A Publication of the IEEE Engineering

in Medicine and Biology Society, 12(2), 258-265.

Wolpaw, J. R. et al., 2002. Brain-computer interfaces for

communication and control. Clinical

Neurophysiology: Official Journal of the International

Federation of Clinical Neurophysiology, 113(6), 767-

91.

Wolpaw, J. R. & McFarland, D. J., 2004. Control of a

two-dimensional movement signal by a noninvasive

brain-computer interface in humans. Proceedings of

the National Academy of Sciences of the United States

of America, 101(51), 17849-17854.

Zhang, H., Guan, C. & Wang, C., 2008. Asynchronous

P300-based brain-computer interfaces: a

computational approach with statistical models. IEEE

Transactions on Bio-Medical Engineering, 55(6),

1754-1763.

BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

244