Entropic Brain-computer Interfaces

Using fNIRS and EEG to Measure Attentional States in a Bayesian Framework

Samuel W. Hincks

, Sarah Bratt

, Sujit Poudel

, Vir V. Phoha

, Robert J. K. Jacob

Daniel C. Dennett

and Leanne M. Hirshﬁeld

Tufts University, Medford, MA, 02155, U.S.A.

Syracuse University, Syracuse, NY, 13210, U.S.A.

Keywords:

BCI, Brain-computer Interface, fNIRS, EEG, Workload, Implicit Interface, Attention, Task-positive Network,

Default Mode Network, Entropy, Physiological Computing, Entropic Brain-computer Interface, Bidirectional

Brain-computer Interface, ADHD, Meditation.

Abstract:

Implicit Brain-Computer Interfaces (BCI) adapt system settings subtly based on real time measures of brain

activation without the user’s explicit awareness. For example, measures of the user’s cognitive proﬁle might

drive a system that alters the timing of notiﬁcations in order to minimize user interruption. Here, we consider

new avenues for implicit BCI based on recent discoveries in cognitive neuroscience and conduct a series

of experiments using BCI’s principal non-invasive brain sensors, fNIRS and EEG. We show how Bayesian

and systems neuroscience formulations explain the difference in performance of machine learning algorithms

trained on brain data in different conditions. These new formulations posit that the brain aims to minimize its

long-term surprisal of sensory data and organizes its calculations on two anti-correlated networks. We consider

how to use real-time input that portrays a user along these dimensions in designing Bidirectional BCIs, which

are Implicit BCIs that aim to optimize the user’s state by modulating computer output based on feedback from

a brain monitor. We introduce Entropic Brain-Computer Interfacing as a type of Bidirectional BCI which uses

physiological measurements of information theoretical dimensions of the user’s state to evaluate the digital

ﬂow of information to the user’s brain, tweaking this output in a feedback loop to the user’s beneﬁt.

1 INTRODUCTION

In contrast to direct brain-computer interfaces

(BCIs), which attempt to build a brain-based substi-

tute for mice, keyboards, and other direct input meth-

ods, implicit BCIs (Poel et al., 2012; Zander et al.,

2014; Zander et al., 2010) strive to portray dimen-

sions about the user that are otherwise invisible to a

computer, and use brain data to update system pa-

rameters whose values are better left beyond direct

user control or attention but might still be usefully

adjusted in response to the user’s cognitive proﬁle.

In this paper, we attempt to sketch a process for au-

tomatically inferring whether or not a system prop-

erty is a useful target for adaptation based on how it

alters brain activation along two dimensions. In or-

der to do this, we ﬁrst explain the concept of entropy

as it relates to brain functioning. In brief, Bayesian

cognitive science posits that information processing

in the brain increases when the brain fails to account

for system input; and this increased prediction er-

ror, randomness, and computation in the brain cor-

relates with the introspectable richness of experience

(Carhart-Harris et al., 2015). Therefore, a user in-

terface that tracked the cognitive burden that its con-

stituent elements placed on a user, and adapted these

information scores based on brain entropy measures,

could learn how to customize itself to optimally en-

rich user experience.

In this paper, we compare the performance of

machine learning algorithms calibrated on data from

fNIRS on EEG when detecting different levels of cog-

nitive workload as well as transitions between task

and rest. Based on the Bayesian formulation that

computation and energy expenditure in the brain de-

pends in part on the novelty of information, we hy-

pothesize that the performance of machine learning

algorithms will degrade in the second session of an

experiment. In the three sections of this paper, we ex-

plain how this property can be exploited as a feature

not a bug of BCI.

Three interdisciplinary research questions under-

Hincks, S., Bratt, S., Poudel, S., Phoha, V., K. Jacob, R., Dennett, D. and Hirshﬁeld, L.

Entropic Brain-computer Interfaces.

DOI: 10.5220/0006383300230034

In Proceedings of the 4th International Conference on Physiological Computing Systems (PhyCS 2017), pages 23-34

ISBN: 978-989-758-268-4

Figure 1: The three challenges of BCI and how they inter-

act.

lie the creation of implicit brain-computer interfaces.

First, an appropriate set of dimensions for the user’s

state needs to be identiﬁed. Much of previous BCI

research uses machine learning algorithms to classify

the user’s state; the dimensionalization component of

BCI amounts to deciding what states about the user

the machine learning algorithms will classify. For ex-

ample, many implicit BCIs (Afergan et al., 2014b;

Afergan et al., 2014a; Afergan et al., 2015; Yuksel

et al., 2016; Solovey et al., 2012) attempt to parse

the user’s cognitive workload. We refer to the chal-

lenge of identifying dimensions with both neurobio-

logical meaning and relevance to interface design as

the dimensionalization challenge of BCI. Second, for

HCI, these dimensions must somehow be extracted

on the basis of measurements collected from non-

invasive and low-cost sensors, which we call the por-

trayal challenge of BCI. With known dimensions and

methods of extracting them using practical sensors,

the ﬁnal challenge, what we call the interaction chal-

lenge, is to map classiﬁcation interactively onto dy-

namic system parameters that drive adaptations to the

user’s beneﬁt, and to measure such beneﬁts. We illus-

trate these challenges and how they interact in Figure

There are currently two brain sensors sufﬁciently

promising for ordinary human-computer interaction

scenarios: EEG (electroencephalography) and fNIRS

(functional near infrared spectroscopy). They each

have different advantages and disadvantages, comple-

ment each other quite well, and have been studied

separately many times. EEG and fNIRS both index

a user’s brain activation. EEG relies on a chemi-

cally induced electrical charge that occurs at a neu-

ron’s action potential (the fundamental unit of com-

putation in the brain). The EEG infers the aggregate

of many such events by measuring voltage changes

on the user’s scalp. The detected voltage at the scalp

oscillates according to speciﬁc patterns, which con-

veys information about the activity of neurons, espe-

cially near the cortex (Teplan, 2002). fNIRS relies

on a neuron’s need for oxygen to execute an action

potential. When a given region is enlisted for com-

putation, it consumes the oxygen it has available lo-

cally and requests more. By introducing near-infrared

light from sensors by a user’s forehead, which pene-

trates skin and bone but is sensitive to hemoglobin in

the blood, a nearby light detector can infer changes

to the amount of oxygen present, as oxygenated and

deoxygenated blood absorb and scatter the light dif-

ferently (Ferrari and Quaresima, 2012). Unlike EEG,

whose most studied signals oscillate at frequencies

ranges above 1 Hertz, fNIRS takes seconds to register

changes in state. This prevents fNIRS from indexing

the high frequency patterns powering the EEG signal

but opens the device to measure longer term patterns

in brain activation (Pfurtscheller et al., 2010).

Complementing each other’s weaknesses and sup-

plementing each other’s strengths, fNIRS and EEG

seem to invite integration into a singular input device

that delivers a suite of predictions about the user’s

cognitive, emotional, and intentional state. EEG has

poor spatial resolution (Berka et al., 2004), mean-

ing it is difﬁcult to resolve from where a given sig-

nal is originating; but it has good temporal resolu-

tion, meaning a given measurement is temporally very

close to the phenomenon it endeavors to portray. Con-

versely, fNIRS has good spatial resolution but poor

temporal resolution (Ferrari and Quaresima, 2012).

Because of its good temporal resolution, EEG can

capture a brief episode of mental activity and trans-

late that activity into a command. For example, a

user might imagine body movements (moving left or

right hands) which would produce a machine learn-

able EEG signal, which then could be mapped to a

deliberate command (Guger et al., 2003). fNIRS is

very difﬁcult to use for direct input because it would

necessarily take seconds for the command to register.

But EEG’s sensitivity to short term electrical ﬂuctua-

tions leaves it vulnerable to noisy inputs. For exam-

ple, an eye blink and other movement also produces

an electrochemical effect that drowns the neurologi-

cal signal underlying state classiﬁcation (Berka et al.,

2004; Hoffmann and Falkenstein, 2008). fNIRS does

not suffer as severely from noise (Maior et al., 2015;

Solovey et al., 2009).

In this paper, we make three contributions to im-

plicit brain-computer interfacing, in dimensionaliza-

tion, portrayal, and interaction, and show how these

three pieces ﬁt together using the concept of entropy

to lead to the design and implementation of bidirec-

tional brain-computer interfaces:

PhyCS 2017 - 4th International Conference on Physiological Computing Systems

Dimensionalization: In the second section, we

connect the concept of cognitive workload to sys-

tem entropy in the Bayesian framework. Cognitive

workload is the standard term for cerebral engage-

ment, and is the user dimension that existing implicit

BCIs (Afergan et al., 2014b; Solovey et al., 2012;

Yuksel et al., 2016) have generally targeted for adap-

tation. Here, we ground cognitive workload with

information-theoretical principles in Bayesian cogni-

tive science and suggest that BCI supplement the one-

dimensional workload attribute with a second user di-

mension, which resolves which spatial network is ex-

ecuting the cognitive workload.

Portrayal: In the third section, we run two sets

of identical two-session experiments, where the user

undergoes a series of computer-based task transitions.

We use the EEG and fNIRS datasets to build machine

learning algorithms that classify users along the di-

mensions proposed in section two, and evaluate their

performance internally using standard cross-fold val-

idation techniques. In the process, we relate the mea-

surement of the two brain sensors (EEG and fNIRS)

which are the obvious candidates to be the mouse

and keyboard of next generation brain-computer in-

terfaces but whose performance is seldom evaluated

in concert. We suggest that EEG and fNIRS are better

suited for detecting different dimensions of a user.

Interaction: Section four uses theory from section

two and data from section three to propose a way for

interactive systems to automatically discover and ad-

just what elements can be updated in order to pull the

user’s state in directions that improve the quality of

attention, leading to designs for future user interfaces.

2 DIMENSIONALIZATION

Good dimensionalization is essential to implicit BCI

because without a link between mental states and the

neurobiological machinery that computes it, it should

be impossible to build an algorithm that predicts the

mental state from brain sensors. If dimensionalization

is done properly, then implicit BCI has a small library

of dimensions that are true enough to the basic oper-

ation of the brain to show an effect to non-invasive

brain sensors but far enough away from its mathe-

matical calculations to transmit meaning to a user in-

terface designer. Physiological computing research

attempts to measure dimensions of the user’s men-

tal states, e.g. cognitive workload (Fairclough et al.,

2005; Venables and Fairclough, 2009), task engage-

ment (Fairclough et al., 2009), stress (Fairclough,

1993), and various emotions (Picard, 1997). These

dimensions are born from psychological research, and

provide the basis for our previous adaptive systems

(Afergan et al., 2014b; Afergan et al., 2015; Afergan

et al., 2015). In these systems, we train a machine

learning algorithm to separate a user’s cognitive pro-

ﬁle during different levels of the n-back, a psycho-

logical induction scheme that varies the strain on a

user’s attention and short term memory. Cognitive

workload classiﬁcations then drive system adapta-

tions, such as controlling interruptions (Afergan et al.,

2015) or changing difﬁculty levels (Afergan et al.,

2014a). In this section, we reinterpret these user di-

mensions from cognitive and systems neuroscience

literature. The two most practical dimensions we ﬁnd

in this investigation provide an information theoreti-

cal formulation of cognitive workload, stress, and task

engagement. In section 3, we show how these formu-

lations are useful for explaining the behavior of ma-

chine learning algorithms; and in section 4, we con-

sider new avenues for adaptive design based on these

dimensions.

There is compelling evidence in recent neu-

roimaging literature to suggest that the brain is con-

sistent in how it organizes the goals of its informa-

tion processing (Friston, 2010). According to the

Bayesian Brain Hypothesis (BBH), the basic goal of

the brain is to actively and parsimoniously predict and

suppress external sensory signals using the knowl-

edge of internal models, and to update these mod-

els so that prediction error is minimized in the fu-

ture (Friston, 2010). The brain might be summa-

rized as a hierarchical prediction and error correc-

tion machine (Clark, 2013), in which information

processing proceeds bidirectionally, so that statisti-

cally informed prediction ﬂows from the top-down

and prediction error modiﬁes internal statistics from

the bottom-up. Thus, computation at large (cogni-

tive workload) ought to increase when the informa-

tion content of the task or environment dictates in-

terest and modiﬁcation in more top-down predictive

machinery. When circumstance is novel relative to

the brain, more prediction error propagates through

the hierarchical Bayesian ﬁlter, causing data ﬂows

to branch and spread up the information processing

hierarchy (Carhart-Harris et al., 2015); and it is to

this broadly deﬁned event which we hypothesize non-

invasive brain sensors positioned at the outer edges of

the brain are approximately sensitive.

Entropy: In general, entropy indexes the average

amount of surprisal in an information processing sys-

tem, and in the brain, entropy is high when internal

models fail to account for system input. High entropy

states include infant consciousness (characterized by

the lack of concepts needed to suppress external stim-

uli), early psychosis (characterized as a bug to top-

Entropic Brain-computer Interfaces

down reality testing), near death experiences (a sit-

uation that is unusual and interesting), psychedelics

(which bind to receptors in hierarchically central top-

down information throttlers) and creativity (where ex-

isting models are penetrated and recalibrated by orig-

inal material) (Carhart-Harris et al., 2015). The as-

sociation between the reported richness of these ex-

periences and their consistent grounding in terms of

information processing that we hypothesize EEG and

fNIRS can detect opens new avenues for design; it

implies that appropriately designed implicit BCIs can

coerce rich and creative experience by intelligently

modulating information content in response to brain

activity.

Predominant Network: When the informational

exchange between human and computer is too low or

high, the mind tends to retreat into an internal mode

of cognition (Csikszentmihalyi, 1996; Raichle et al.,

2001). In this mode, the brain grants more resources

to endogenous systems, which are not yoked to the ex-

ternal environment (Raichle, 2010). Instead of com-

puting on data that originates from the senses, these

resting networks (the most prominent of which is the

default mode network) (Raichle et al., 2001) operate

on endogenous data or memories – planning, reﬂect-

ing, and fantasizing. Because these networks gener-

ally operate even when the organism is otherwise lib-

erated to take a break, the net oxygen consumption of

the brain decreases by less than 5% at rest (Raichle,

2010), implying that a BCI that attempts to maximize

user experience must supplement its cognitive work-

load or entropy index with a secondary dimension that

informs the user interface the space in which compu-

tation is occurring in the brain. Conveniently for BCI,

these endogenous resting networks are anti-correlated

with the exogenous task-positive networks, making

them especially suitable candidates for detection by

a spatially well-resolved brain monitor, since nodes

from the opposite network can be used to constrain

false positives and most regions have a known and

consistent bias towards involvement in either one of

these two networks (Glasser et al., 2015). Since time

spent mind wandering predicts reported present and

future happiness (Killingsworth and Gilbert, 2010),

implicit BCI that learned how to apply system adapta-

tions that minimized endogenous retreat could impact

user happiness beyond the immediate experience.

Together, entropy and attentional orientation ex-

plain the user dimensions already posited in psycho-

logical and physiological computing literature. For

example, task engagement and stress are both high en-

tropy, but engagement may involve the more consis-

tent application of exogenous computation, whereas

stress may imply periodic retreat into introspective

networks. Designing for cognitive neuroscience di-

mensions and not psychological dimensions is worth-

while for implicit BCI, since these systems need to

base user classiﬁcation on physiological measure-

ments from brain sensors. We predict that the greater

the distance between tasks along these two dimen-

sions, the easier it is for a machine learning algorithm

to separate instances of those tasks.

The problem for BCI is that each user’s brain is

unique, and therefore poses unique external require-

ments for coercing it into different states. Further-

more, brains change over time based on new input.

For these reasons, we think it is critical to dimension-

alize the user with the intuitions of Bayesian mathe-

matics (Perfors et al., 2011). If the energy expenditure

of the brain is proportional to the degree to which it

must update belief and action in order to integrate evi-

dence from the environment and if non-invasive brain

sensors are principally sensitive to physiological cor-

relates of that energy expenditure, then the novelty

of a task in relation to a control (e.g. resting state)

is likely low hanging fruit for a machine learning al-

gorithm operating on either fNIRS or EEG data to

classify. To test this, we have repeated our experi-

ment twice, and evaluated the change in performance

of machine learning algorithms from a ﬁrst to second

session.

Bayesian cognitive science argues that the brain

seeks to minimize its entropy but that it generates

more computation and experience (Carhart-Harris

et al., 2015) when it is in a state of high entropy.

Many humans struggle to reconcile their brain’s nat-

ural urge to rewire itself towards lower entropy with

the desire for rich experience. A well-designed user

interface can assist in this endeavor. In section 4, we

describe entropic BCIs, which attempts to judiciously

modulate user-mediated information in order to pull

the brain into exogenous high entropy states. Figure 2

simpliﬁes the user as always belonging to one of four

states, the desirability of which depends on the current

scenario. A rigid and mild endogenous brain indi-

cates boredom or tiredness; a random and intense en-

dogenous brain indicates creativity or anxiety; a rigid

and mild exogenous brain indicates relaxation; and a

random and intense exogenous brain indicates ﬂow or

stress. Each state has speciﬁc user affordances, tasks

which suits it, controlled and incidental means to cal-

ibrate machine learning algorithms that detect it, as

well as guidelines for how to transform the state into

another one.

In our experiment, we hypothesize that fNIRS,

which has good spatial resolution, is better than EEG

at classifying when user’s transition from resting to

task states, but that EEG, which has good temporal

PhyCS 2017 - 4th International Conference on Physiological Computing Systems

Figure 2: A simpliﬁcation of users into four states along two

dimensions, which we expect to be low hanging fruit for

detection via fNIRS and EEG. The Cognitive Workload or

Entropy dimension indexes the degree of computation and

surprise in the user’s brain, and the Attentional Orientation

or Predominant Network dimension indexes the degree to

which this computation is engaged in a goal-directed loop

with its sensory environment.

resolution, is better than fNIRS at indexing a user’s

entropy or workload, since information processing

occurs quickly in the brain.

3 PORTRAYAL

In typical BCI experiments at the portrayal level, the

subject undergoes a series of task transitions under si-

multaneous interrogation of brain monitors (Girouard

et al., 2009; Hirshﬁeld et al., 2009; Hirshﬁeld et al.,

2011). These experiments and many others pack-

age results as the performance and behavior of ma-

chine learning algorithms, where the brain data of tri-

als of the same condition receive the same label and

are translated into a statistical feature-space. Machine

learning algorithms, such as support vector machines,

build models for relating the feature-space to the la-

bel, and, if the brain data differs between the two con-

ditions, then machine learning algorithms should be

able to predict the labels of trials that were not a part

of its training model. The performance of a machine

learning algorithm can be evaluated using a cross-fold

validation scheme, where a model is built many times,

each time excluding from its training set a new set of

trials for its testing set. This machine learning method

will be the basis for how we distribute results in this

section. Because the novelty of the experiment is that

we have repeated it four times using two portable

Figure 3: Hitachi fNIRS equipment.

brain sensing instruments (EEG and fNIRS), we can

observe what dimensions of user information process-

ing are better left to the jurisdiction of either sensor,

in the hope of specifying how an fNIRS and EEG-

integrated could jointly classify the user’s state.

3.1 Equipment

The EEG used in this experiment was Advanced Brain

Monitoring’s b-alert X10, a 9 channel wireless EEG

system with a linked mastoid reference, sampling at

256hz. The EEG headset was placed on users us-

ing standard 10-20 measurement set-up techniques

(Homan et al., 1987). Figure 5 shows the nine regions

of the brain measured by each channel.

The fNIRS device used in this experiment was the

Hitachi ETG-4000 fNIRS device with a sampling rate

of 10Hz. The fNIRS probe (Figure 3) was a 3x11

probe with 17 light sources and 16 detectors, resulting

in 52 locations measured on the head.

3.2 Method

Twenty-three subjects (8 female) between the ages

18 and 49 participated in the experiment. Upon ar-

rival, subjects consented to the experiment and were

ﬁtted with the fNIRS or EEG sensors. The prepack-

aged b-alert and Hitachi software calibrated itself to

the detected connection with the user’s scalp. Then,

the subject alternated between 8 instances of an arith-

metic task and 8 instances of an image-matching task,

performing each task for 35 seconds, with 15-second

controlled rest periods in between. For the image-

matching task, users indicated whether sequences of

images matched each other, as in an n-back (Gevins

and Smith, 2003) with n permanently set to 1, similar

to the low cognitive workload condition used in previ-

ous implicit BCI work (Afergan et al., 2014a). For the

arithmetic task, users added two two-digit integers to

each other, entering the response into a text-box. We

included workload in two separate modalities in order

to, potentially, induce two separate states of workload

In ﬁgure 4, we show the computer output for these

tasks and in ﬁgure 2, we plot these tasks (and com-

Entropic Brain-computer Interfaces

Figure 4: The three conditions of the experiment.

parisons between them) along the spectra of cognitive

workload and attentional orientation.

We were thus interested in the transition between

task and rest, and how that would change in the ﬁrst

and second session for both EEG and fNIRS. Our in-

terest in inter-session comparison was born out of a

consideration for how machine learning algorithms

might decay over time if they did not account for up-

dates to the user’s cognition. We were especially in-

terested in corroborating a previous small long term

pilot (Hincks et al., 2016), where we tracked one of

the author’s fNIRS data over the course of several

months as he made himself an expert at the cognitive

workload tasks typically used in implicit BCI (the n-

back). In later sessions, we noticed that fNIRS failed

to register a strong effect unless the difﬁculty placed

him at the edge of his ability, and we hypothesized

that the task had over time generated too efﬁcient top-

down schemes for solving it, thereby leaking less pre-

diction error, the information theoretical construct to

which we hypothesize these cortically-sensing instru-

ments are principally sensitive. If that is the case,

then machine learning algorithms operating from ei-

ther EEG or fNIRS data should degrade in a second

session of the experiment. The experiment was thus

repeated four times in two sessions for each device

on two separate days. The second session for a given

participant was at most 27 days later and at least 23

days later.

For each of the 23 subjects, we therefore had four

datasets (two sessions for each device). Each dataset

included 8 trials for the math task, 8 for the image

task, and 16 for the resting task. We were interested

in whether or not we could build a machine learn-

ing algorithm to separate the image and math task

using both fNIRS and EEG data, and how this algo-

rithm might change in the second session. We built a

new machine learning algorithm, for each subject for

each session, and for each neuroimaging device. We

tried one approach for feature design using speciﬁca-

tions we optimized elsewhere (Treacy Solovey et al.,

2015), and for ease of communication, we let this al-

gorithm be identical for both EEG and fNIRS, leaving

the critical preprocessing components to the software

distributed by Hitachi and the b-alert EEG.

For fNIRS, the input to our data mining scheme

was a matrix of 52 channels that had been converted

from light intensity into oxygenation measurements

according to the Beer-Lambert Law, and bandpass ﬁl-

tered, leaving only the components of the signal that

ﬂuctuated between 0.01 and 0.5 hz. In our analysis,

we omitted deoxygenation measurements since these

values largely convey the same information as oxy-

genation. For EEG, the raw data was processed by

Advanced Brain Monitoring’s proprietary acquisition

software, which includes artifact decontamination al-

gorithms for eye blinks, muscle movements, and en-

vironmental/electrical interference. After decontami-

nating the raw data, the input was a matrix of 90 chan-

nels, consisting of the average power spectral density,

averaged together into one second time periods, at

each of the nine channel locations. Power spectral

density was computed for the ten frequency bands of

delta (1-3Hz), theta slow (3-5Hz), theta fast (5-7Hz),

theta total, (3-7Hz) alpha slow (8-10Hz), alpha fast

(10-13Hz), alpha total (8-13Hz), sigma (12-15Hz),

beta (13-30Hz), and gamma (25-40Hz), at each of the

channel locations.

For each instance, we computed the mean, linear

slope, and standard deviation of the entire time-series

of values for each channel. Thus, for inter-task (A1,

A2) comparisons on the fNIRS data, the 350 readings

x 52 channel windows became 2 condition x 8 tri-

als x 156 feature instances and for EEG, the 35 read-

ings x 90 channels window became 2 condition x 8

trials x 180 feature windows. For the task vs. rest

comparisons (B1, B3), the same transformation oc-

curred but the ﬁrst 150 readings of the task were ex-

tracted, and compared to the 150 readings of resting

data. We fed these feature sets into Matlab’s Statistics

and Machine Learning Toolbox implementation of the

linear kernel support vector machine (SVM) and did

not change default parameters (since the goal was to

discern to what fundamental dimensions the machine

learning algorithms were most sensitive and how that

differed between devices, and not to maximize ma-

chine learning performance). We evaluated each ma-

chine learning separation using 10-fold cross valida-

tion (Refaeilzadeh et al., 2009), training the machine

learning algorithm on all but an approximate tenth of

the data, changing what tenth was omitted from the

dataset and using that set for testing the trained clas-

siﬁer in ten separate tests. For all tests, evaluation in-

stances were drawn from the same subject and session

as the training instances that drove the machine learn-

ing algorithm. Next, we report on the averaged 10-

fold cross validation classiﬁcation accuracy for each

test of interest.

PhyCS 2017 - 4th International Conference on Physiological Computing Systems

Table 1: SVM Machine Learning Accuracies for Matching

vs. Addition Comparison (A1,A2). m denotes the mean

classiﬁcation accuracy for all 23 subjects in 10-fold cross-

validation and s refers to the standard deviation. For each

row and column, classiﬁcation accuracies have been com-

pared in a paired samples t-test, and the p-value is reported.

S EEG fNIRS p

1 m=73%, s=21% m=72%, s=18% 0.7358

2 m = 80%, s = 29% m = 71%, s =17% 0.0525

p 0.2430 0.8900

3.3 Results

3.3.1 Task Comparison: A1,A2

For each session, we made two comparisons for each

device, ﬁrst distinguishing between the two task con-

ditions and then separating the two tasks. Table 1

shows classiﬁcation accuracies in cross-fold valida-

tion for the inter-task separation. In the previous sec-

tion, Figure 2 shows how we expect these two tasks

to differ from each other with respect to the workload

and attentional orientation dimensions for the differ-

ent sessions. The addition task presumably poses a

greater burden to the user’s cognitive workload than

the matching task. We have compared the mean clas-

siﬁcation accuracies for each of the 23 subjects be-

tween both device and session, and Table 1 reports

the probability that the null hypothesis is true in a

paired sample t-test. There were no signiﬁcant ef-

fects in these comparisons, but the EEG-based ma-

chine learning algorithms trended towards better per-

formance. The results highlight that both devices can

effectively parse the user along the cognitive work-

load dimension.

3.3.2 Rest Versus Task Comparison: B1,B2

Table 2 shows an identical analysis but for the com-

parison between the two tasks and rest. Since rest

periods were shorter than task periods, we truncated

the task trials so that they only included the ﬁrst 15

seconds of data. Table 1 indicates that this compari-

son primarily manipulates whether or not the user has

engaged an endogenous or exogenous network. For

fNIRS, machine learning performance in the ﬁrst ses-

sion (m = 84%, s = 9%) is signiﬁcantly better than

machine learning performance in the second session

(m = 75%, s = 13%) (p =0.0096, N = 23). Similarly,

for EEG, machine learning performance decays sig-

niﬁcantly from the ﬁrst session (m = 79%, s = 12%)

to the second session (m = 68%, s = 17%) (p =0.0270,

N = 23).

Table 2: SVM Machine Learning Accuracies for Task vs.

Rest Comparison (B1,B2).

S EEG fNIRS p

1 m = 79%, s = 11% m = 84%, s = 9% 0.11

2 m = 68%, s = 17% m = 75%, s =13% 0.11

p 0.0270* 0.0096**

Figure 5: Channel locations for fNIRS and EEG. For

fNIRS, only highlighted Brodmann Regions are measured.

3.4 Discussion

It is interesting that EEG (m = 76%) outperformed

fNIRS (m = 71%) at separating the mathematical and

image recognition task, which manipulates the user

along the cognitive workload dimension but fNIRS

(m = 80%) outperformed EEG (m = 74%), which ma-

nipulates whether or not the user is engaging a task-

positive or task-negative network. Even though these

differences are not signiﬁcant, the results are consis-

tent with the hypothesis that these two devices com-

plement each other, covering the other’s weakness.

fNIRS is generally regarded as supporting better spa-

tial resolution whereas EEG has better temporal res-

olution. Since every region in the brain is better de-

scribed as belonging to either a task-positive or task-

negative network and these two networks are anti-

correlated, the fNIRS-based features (which are not as

confused as EEG about the tissue they measure) might

provide the information the SVM needed to discern

the notion of anti-correlated networks, and robustly

predict the user’s state.

In general, as BCIs attempt to portray ﬁner dimen-

sions about the user, fNIRS may prove the better in-

strument for detecting spatial dimensions where dif-

ferent states of the dimension imply the enlistment of

different neuroanatomic space. Beyond predominant

network, an additional spatial dimension might in-

clude parsing the modality of input (spatial vs. motor

vs. auditory) (Glasser et al., 2015). Conversely, EEG

may prove better at extracting information processing

phenomena. Beyond entropy or cognitive workload,

this may include the abstraction (explicit vs. implicit)

(Kahneman, 2011), continuity (static vs. dynamic)

(Baddeley and Hitch, 1974), and direction (top-down

Entropic Brain-computer Interfaces

vs. bottom-up) (Pinto et al., 2013) of cerebral data

ﬂows. Investigation into how to induce, portray, and

design for these four other fundamental dimensions is

an area of future research for implicit BCI.

It is also interesting that in both fNIRS and EEG

experiments for the separation rest versus task, classi-

ﬁcation accuracy in session 1 reduces signiﬁcantly in

the second session for both fNIRS (p = 0.0096) and

for EEG (p = 0.0270), but not for the separation be-

tween the two tasks, where classiﬁcation accuracy is

in fact better in the second session for EEG and ap-

proximately the same for FNIRS. But this is not sur-

prising if the brain is dimensionalized according to

the Bayesian framework. Both fNIRS and EEG mea-

sure brain activation principally at the outer-tips of

the brain, its cortex. In a Bayesian framework, the

outer-tips of the brain’s hardware presumably carry

out computations very high in the information pro-

cessing hierarchy. In the second session, subjects had

already been exposed to the tasks; thus, the second

session cognitive makeup likely included internal rep-

resentations that solved the input-output relations dic-

tated by the task at a more primitive point in the infor-

mation processing hierarchy, reducing the prediction

error and its associated corrective events to penetrate

the higher level regions of the brain under interroga-

tion by the brain sensors.

Speciﬁcally, we attribute the relatively increased

difﬁculty of the SVM to predict transitions between

task and rest in the second session to the greater dif-

ference in system entropy between the resting and

task conditions in the ﬁrst session than in the sec-

ond session. Since prediction error dictates the entire

operation of a Bayesian brain, this may be another

way of expressing that, in the second session, the user

had absorbed efﬁcient probability distributions for the

task, enabling competing endogenous resting state in-

puts (which draw from the same ﬁnite pool of oxygen

supply) to ﬂourish and thus produce a proﬁle that bet-

ter matched the resting state. In simpler terms, the

user’s brains had ﬁgured how to efﬁciently solve the

task in the second day of the experiment, but not how

to efﬁciently rest. With this interpretation, machine

learning accuracy did not change signiﬁcantly for the

inter-task comparison since the user had previously

engaged both tasks, making so that task-induced en-

tropy would decrease equally in both conditions.

The difﬁculty to control for system entropy be-

tween tasks and sessions is a feature not a bug of

brain-computer interfacing so long as it is acknowl-

edged by the designer. By fusing the user dimension

of cognitive workload with system entropy, there is an

opportunity to build a device that uses brain classiﬁ-

cation to manipulate the quality of user experience.

In the next section, we show how to use the dimension

of entropy in a bidirectional brain-computer interface.

4 INTERACTION

The interaction challenge of implicit BCI is to map

machine learning classiﬁcations onto system vari-

ables whose values are better set with the knowl-

edge of the user’s cognitive state. There are a va-

riety of such implicit BCI implementations reported

in human-computer interaction venues for fNIRS. In

previous work, we built an engine for adjusting task

difﬁculty by training machine learning algorithms to

separate fNIRS data pertaining to easy and hard ver-

sions of the n-back task (Afergan et al., 2014a). Once

trained, the support vector machine provided a traf-

ﬁc monitoring simulation predictions about the user’s

workload, which then removed planes from user ju-

risdiction when they were deemed to be overworked,

and added planes when they were underworked or

bored. Users performed better at the task when adap-

tation was driven by changes in their brain as opposed

to randomly. Other implicit BCIs have followed a

similar principle, updating system properties based

on brain activity, and comparing user performance to

constant or random properties (Afergan et al., 2014b;

Afergan et al., 2015; Solovey et al., 2012; Yuksel

et al., 2016).

We refer to a BCI which attempts to alter the

user’s state as a Bidirectional BCI (Hincks et al.,

2017), emphasizing the two-way channel in a system

which outputs physical events which affect the state

of the brain, measures this effect using non-invasive

sensors, and uses this measurement to modify future

interventions in a feedback loop. An Entropic BCI is

a Bidirectional BCI which uses information theoret-

ical models of cognition to interpret data from non-

invasive sensors such as fNIRS and EEG in order to

in real-time dimensionalize users along spectra which

describe how the brain is currently processing infor-

mation (e.g. system entropy, high vs. low cognitive

workload, or attentional orientation). An Entropic

BCI determines how to modulate the ﬂow of infor-

mation to the user based on these classiﬁcations in

order to coerce more desirable attentional states. En-

tropic BCIs design for a brain that will retreat into

endogenous processing when the informational ex-

change with the computer is too low or too high, and

they acknowledge that the brain seeks to minimize its

entropy but often relishes high entropy, learning, and

the rich experiences that occur when the brain must

adapt to new input. With current technology, Bidirec-

tional BCIs can use four categories of intervention

PhyCS 2017 - 4th International Conference on Physiological Computing Systems

Figure 6: Four methods for implicitly modulating the infor-

mational exchange between the user and computer, and the

state in which it is most suitable to apply them.

for manipulating the user’s state, but new modalities,

e.g. mixed reality and brain stimulation (Hincks et al.,

2017) will expand this arsenal. In previous work, we

have tested prototypes for three of these four cate-

gories (see ﬁgure 6), and we reinterpret these proto-

types within an Entropic BCI framework.

Interruptions (e.g. text messages, email, or social

media notiﬁcations) can be framed positively as meth-

ods for breaking malproductive rumination provided

that they occur when the user has retreated into a mild

task-negative endogenous state. In previous work, we

built a system that triggered more wearable notiﬁca-

tions when the user was deemed to be in a state of

low cognitive workload, but in the associated experi-

ment, performance measures did not improve dramat-

ically (Afergan et al., 2015). An improved interrup-

tion engine might recognize the predominant network

dimension and time notiﬁcations to break endogenous

day-dreaming instead of low cognitive workload.

Escalations increase the difﬁculty or complexity

of a task, and can make an underworked exogenous

state more engaged. For example, one of our previ-

ous experiments increased the number of unmanned

aerial vehicles under user jurisdiction when the user

was predicted to be underworked, which improved

overall performance in (Afergan et al., 2014a). Other

BCIs that might be framed as escalations include a

musical interface which adapted the challenge of mu-

sical scores (Yuksel et al., 2016) and a simulation

which controlled the degree of automation in a semi-

automated robot (Solovey et al., 2012). In general, if

an HCI task is more strictly exogenous (meaning that

it, like a computer game, aims to instantiate a swift

input-output bond with the user), the more suitable it

is to apply escalations.

Personalizations do not aim to modify the user’s

state, instead aiming to facilitate and design for the

state that has unfolded. For example, we built a brain-

based bubble cursor (Grossman and Balakrishnan,

2005), which modiﬁed the ease of selecting graphi-

cal user interface elements with a mouse dependent

on the user’s state (Afergan et al., 2014b). Like es-

calations, adaptations that cater to the user’s current

state can apply to the system’s semantic layer (its in-

ternal values and parameters) or to its syntactic layer

(its inputs and outputs) (Treacy Solovey et al., 2015).

Predictables are brain-adaptive computer output

that convey no essential information to the user, ex-

isting only by virtue of the fact that the brain is a

prediction machine that will faithfully dedicate pro-

cessing and manipulate the user’s state given unpre-

dictable stimuli. We know of no BCI implementa-

tions that would qualify as Predictables but poten-

tially, the world of unexplored ideas has many novel

opportunities, making it uncharted area of BCI re-

search. Music is noise organized by a musician to

occupy a predictive sweet-spot for the listener, where

the parameters of the sound are predictable enough to

enlist top-down circuitry but unpredictable enough to

necessitate adaptation and simulate vivacious experi-

ence. In an Entropic BCI, the sound would need to

modify in response to brain activation. One simple

alteration would be to modify the sound’s origin in 3-

dimensional space. This would function as a system

knob for controlling the user’s implicit computation

since low level regions of the brain would need to pre-

dict and adapt for the moving sound. Potentially, this

phenomenon could be replicated digitally by assign-

ing a unique motion pattern to each constituent instru-

ment or electronic track of a song. That sound could

then be delivered by 3D audio headphones, adapting

the amount of motion based on brain activity.

Beyond audio spatialization, music has many

more opportunities for adapting sound and interfacing

with the brain in an Entropic Brain-Computer Inter-

face since any parameter governing how sound is pro-

duced in a synthesizer could be subject to optimiza-

tion. In practice, such audio brain-computer inter-

faces might proliferate if headphones augmented with

brain sensors (such as the Kickstarter project Mind-

set) became commercially available and if songs were

distributed as programs which adapted based on in-

put instead of as mp3s. The proliferation of adaptive

music depends on musicians writing electronic songs

using the Web Audio API (Rogers, 2012) and open

source javascript software that extends it (Choi and

Berger, 2013; Mann, 2015) since songs written using

web synthesizers can be distributed online for play-

Entropic Brain-computer Interfaces

back on any browser, where they could change ele-

ments of a song in response to input as any other web

application. Discovering other predictables is pro-

ductive future work, especially in mixed and virtual

reality scenarios that access more of the user’s senso-

rium. With the Entropic Brain-Computer Interfacing

framework, any system which speciﬁed a set of vari-

ables governing the information presented to the user

could be tweaked by a common algorithm for auto-

matically inferring when and how to modify settings

to these variables in order to steer the user into desir-

able states. Such bidirectional BCI UI software would

formalize the notion of user relative information and

customize it over time depending on the user’s cogni-

tive state. The space of interaction between the user

and computer could be encoded in a high level object

referred to as the information space. The information

space consists of objects, chosen to represent the inde-

pendent set of elements that best describes the user’s

current exchange with the computer, as well as a set

of transitions between different versions of that ob-

ject. The novelty of this interface is that both objects

and transitions have an information score, which de-

scribes the average entropy burden the object or tran-

sition poses on the user. These values are initially set

randomly (or pre-calibrated according to expectations

or the cloud), but adjust incrementally (up or down)

based on concurrent brain-based entropy calculations.

Over the long run on many users, we expect these in-

formation scores to be accurate even if the underlying

classiﬁcation algorithm is not because of regression to

the mean. The set of transitions with low inferred en-

tropy are deﬁned as implicit. Depending somewhat on

the goals of the interaction, a Bayesian implicit BCI

could execute low entropy (implicit) transitions into

an information space with a higher sum to the overall

information score when the user had engaged a mild

resting state network, implying that the cerebration

demanded from the user’s task-positive network was

sufﬁciently low that the competing, internally rumi-

nating branch of cognition was granted access to the

energy supply and allowed to subtract the user from

external experience. The effectiveness of that inter-

vention would itself be evaluated by how the predic-

tion of the user’s state switched on the task-positive

versus task-negative dimension, internally rewarding

the transitions that succeeded and punishing the one’s

that failed.

5 CONCLUSIONS

In this paper, we have given theoretical and empirical

arguments for why fNIRS and EEG can jointly es-

timate the user’s state along the dimensions of atten-

tional orientation and cognitive workload. We syn-

onymize cognitive workload with entropy in order to

make a bridge between implicit BCI and a growing

Bayesian literature that is positing an equivalence be-

tween entropy and the introspectable temperature of

experience (Carhart-Harris et al., 2015). We sketch

four interface designs for pulling the user into high

temperature externally oriented ﬂow states based on

brain activation, and sketch algorithm for how these

interventions might improve automatically if the user

interface formalizes, adjusts and adapts user relative

information. We expect BCI to mature alongside vir-

tual, augmented, and mixed reality technology. In

due time, the computer output under the jurisdiction

of mixed BCI technology will then extend beyond

merely notiﬁcations and other system settings to the

complete sensorium molding the brain’s state. With

concurrent measures of user entropy and attentional

orientation, it may be possible to customize such a

virtual reality so that it interactively creates arrange-

ments of sensory information that maximize its user’s

experience and joy. A working implementation of this

technology would warrant a careful ethical consider-

ation of how to judiciously wield this new power over

the brain.

ACKNOWLEDGEMENTS

We thank Beste Yuksel, Daniel Afergan, Evan Peck,

Erin Solovey, Leanne Hirshﬁeld, Tomoki Shibata,

Anzu Hakone, Ronna ten Brink, Min Bu, Sonal Chat-

ter, Yan Huo, Eun Youb Lee, Bushra Alkadhi, Feiyu

Lu, Mary Skitka, Nick Sempere, Maya DeBellis, Tal

August, James Carney, Nik Liolios, Beibei Du, and

Calvin Liang who are students and alumni of the

HCI group at Tufts. We thank Remco Chang, Erika

Hussey, Tad Brunye, Sergio Fantini and Angelo Sas-

saroli from Tufts University. We thank Stuart Hirsh-

ﬁeld from Hamilton College. We thank Google Inc.

for support of this research. Vir V. Phoha was sup-

ported in part by National Science Foundation Award

SaTC Number: 1527795.

REFERENCES

Afergan, D., Hincks, S. W., Shibata, T., and Jacob, R. J.

(2015). Phylter: a system for modulating notiﬁcations

in wearables using physiological sensing. In Inter-

national Conference on Augmented Cognition, pages

167–177. Springer.

Afergan, D., Peck, E. M., Solovey, E. T., Jenkins, A.,

Hincks, S. W., Brown, E. T., Chang, R., and Jacob,

PhyCS 2017 - 4th International Conference on Physiological Computing Systems

R. J. (2014a). Dynamic difﬁculty using brain metrics

of workload. In Proceedings of the 32nd annual ACM

conference on Human factors in computing systems,

pages 3797–3806. ACM.

Afergan, D., Shibata, T., Hincks, S. W., Peck, E. M., Yuksel,

B. F., Chang, R., and Jacob, R. J. (2014b). Brain-

based target expansion. In Proceedings of the 27th

annual ACM symposium on User interface software

and technology, pages 583–593. ACM.

Baddeley, A. D. and Hitch, G. (1974). Working memory.

Psychology of learning and motivation, 8:47–89.

Berka, C., Levendowski, D. J., Cvetinovic, M. M., Petro-

vic, M. M., Davis, G., Lumicao, M. N., Zivkovic,

V. T., Popovic, M. V., and Olmstead, R. (2004). Real-

time analysis of eeg indexes of alertness, cognition,

and memory acquired with a wireless eeg headset. In-

ternational Journal of Human-Computer Interaction,

17(2):151–170.

Carhart-Harris, R. L., Leech, R., Hellyer, P. J., Shanahan,

M., Feilding, A., Tagliazucchi, E., Chialvo, D. R., and

Nutt, D. (2015). The entropic brain: a theory of con-

scious states informed by neuroimaging research with

psychedelic drugs. Psychoanalytical neuroscience:

Exploring psychoanalytic concepts with neuroscien-

tiﬁc methods, page 140.

Choi, H. and Berger, J. (2013). Waax: Web audio api ex-

tension. In NIME, page 499502.

Clark, A. (2013). Whatever next? predictive brains, situated

agents, and the future of cognitive science. Behavioral

and Brain Sciences, 36(03):181–204.

Csikszentmihalyi, M. (1996). Flow and the psychology of

discovery and invention. New Yprk: Harper Collins.

Fairclough, S. (1993). Psychophysiological measures of

workload and stress. Driving future vehicles, pages

377–390.

Fairclough, S. H., Moores, L. J., Ewing, K. C., and Roberts,

J. (2009). Measuring task engagement as an input to

physiological computing. In 2009 3rd International

Conference on Affective Computing and Intelligent In-

teraction and Workshops, pages 1–9. IEEE.

Fairclough, S. H., Venables, L., and Tattersall, A. (2005).

The inﬂuence of task demand and learning on the psy-

chophysiological response. International Journal of

Psychophysiology, 56(2):171–184.

Ferrari, M. and Quaresima, V. (2012). A brief review on

the history of human functional near-infrared spec-

troscopy (fnirs) development and ﬁelds of application.

Neuroimage, 63(2):921–935.

Friston, K. (2010). The free-energy principle: a uni-

ﬁed brain theory? Nature Reviews Neuroscience,

11(2):127–138.

Gevins, A. and Smith, M. E. (2003). Neurophysiologi-

cal measures of cognitive workload during human-

computer interaction. Theoretical Issues in Er-

gonomics Science, 4(1-2):113–131.

Girouard, A., Solovey, E. T., Hirshﬁeld, L. M., Chauncey,

K., Sassaroli, A., Fantini, S., and Jacob, R. J.

(2009). Distinguishing difﬁculty levels with non-

invasive brain activity measurements. In IFIP Con-

ference on Human-Computer Interaction, pages 440–

452. Springer.

Glasser, M., Coalson, T., Robinson, E., Hacker, C., Harwell,

J., Yacoub, E., Ugurbil, K., Anderson, J., Beckmann,

C., Jenkinson, M., et al. (2015). A multi-modal par-

cellation of human cerebral cortex. Nature.

Grossman, T. and Balakrishnan, R. (2005). The bubble cur-

sor: enhancing target acquisition by dynamic resizing

of the cursor’s activation area. In Proceedings of the

SIGCHI conference on Human factors in computing

systems, pages 281–290. ACM.

Guger, C., Edlinger, G., Harkam, W., Niedermayer, I., and

Pfurtscheller, G. (2003). How many people are able to

operate an eeg-based brain-computer interface (bci)?

IEEE transactions on neural systems and rehabilita-

tion engineering, 11(2):145–147.

Hincks, S. W., Afergan, D., and Jacob, R. J. (2016). Us-

ing fnirs for real-time cognitive workload assessment.

In International Conference on Augmented Cognition,

pages 198–208. Springer.

Hincks, S. W., Debellis, M., Lee, E. Y., ten Brink, R.,

and Jacob, R. J. (2017). Towards bidirectional brain-

computer interfaces that use fnirs and tdcs. In PhyCS.

Hirshﬁeld, L. M., Gulotta, R., Hirshﬁeld, S., Hincks, S.,

Russell, M., Ward, R., Williams, T., and Jacob, R.

(2011). This is your brain on interfaces: enhancing

usability testing with functional near-infrared spec-

troscopy. In Proceedings of the SIGCHI Conference

on Human Factors in Computing Systems, pages 373–

382. ACM.

Hirshﬁeld, L. M., Solovey, E. T., Girouard, A., Kebinger,

J., Jacob, R. J., Sassaroli, A., and Fantini, S. (2009).

Brain measurement for usability testing and adaptive

interfaces: an example of uncovering syntactic work-

load with functional near infrared spectroscopy. In

Proceedings of the SIGCHI Conference on Human

Factors in Computing Systems, pages 2185–2194.

ACM.

Hoffmann, S. and Falkenstein, M. (2008). The correction

of eye blink artefacts in the eeg: a comparison of two

prominent methods. PLoS One, 3(8):e3004.

Homan, R. W., Herman, J., and Purdy, P. (1987). Cere-

bral location of international 10–20 system electrode

placement. Electroencephalography and clinical neu-

rophysiology, 66(4):376–382.

Kahneman, D. (2011). Thinking, fast and slow. Macmillan.

Killingsworth, M. A. and Gilbert, D. T. (2010). A

wandering mind is an unhappy mind. Science,

330(6006):932–932.

Maior, H. A., Pike, M., Sharples, S., and Wilson, M. L.

(2015). Examining the reliability of using fnirs in

realistic hci settings for spatial and verbal tasks. In

Proceedings of the 33rd Annual ACM Conference on

Human Factors in Computing Systems, pages 3039–

3042. ACM.

Mann, Y. (2015). Interactive music with tone.js. In Proceed-

ings of the 1st annual Web Audio Conference. Citeseer.

Perfors, A., Tenenbaum, J. B., Grifﬁths, T. L., and Xu, F.

(2011). A tutorial introduction to bayesian models of

cognitive development. Cognition, 120(3):302–321.

Entropic Brain-computer Interfaces

Pfurtscheller, G., Allison, B. Z., Bauernfeind, G., Brunner,

C., Solis Escalante, T., Scherer, R., Zander, T. O.,

Mueller-Putz, G., Neuper, C., and Birbaumer, N.

(2010). The hybrid bci. Frontiers in neuroscience,

4:3.

Picard, R. W. (1997). Affective computing, volume 252.

MIT press Cambridge.

Pinto, Y., van der Leij, A. R., Sligte, I. G., Lamme,

V. A., and Scholte, H. S. (2013). Bottom-up and top-

down attention are independent. Journal of Vision,

13(3):16–16.

Poel, M., Nijboer, F., van den Broek, E. L., Fairclough,

S., and Nijholt, A. (2012). Brain computer interfaces

as intelligent sensors for enhancing human-computer

interaction. In Proceedings of the 14th ACM inter-

national conference on Multimodal interaction, pages

379–382. ACM.

Raichle, M. E. (2010). The brain’s dark energy. Scientiﬁc

American, 302(3):44–49.

Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers,

W. J., Gusnard, D. A., and Shulman, G. L. (2001). A

default mode of brain function. Proceedings of the

National Academy of Sciences, 98(2):676–682.

Refaeilzadeh, P., Tang, L., and Liu, H. (2009). Cross-

validation. In Encyclopedia of database systems,

pages 532–538. Springer.

Rogers, C. (2012). Web audio api. s Draft [online speciﬁ-

cation], Version, 1.

Solovey, E., Schermerhorn, P., Scheutz, M., Sassaroli, A.,

Fantini, S., and Jacob, R. (2012). Brainput: enhanc-

ing interactive systems with streaming fnirs brain in-

put. In Proceedings of the SIGCHI conference on

Human Factors in Computing Systems, pages 2193–

2202. ACM.

Solovey, E. T., Girouard, A., Chauncey, K., Hirshﬁeld,

L. M., Sassaroli, A., Zheng, F., Fantini, S., and Jacob,

R. J. (2009). Using fnirs brain sensing in realistic hci

settings: experiments and guidelines. In Proceedings

of the 22nd annual ACM symposium on User interface

software and technology, pages 157–166. ACM.

Teplan, M. (2002). Fundamentals of eeg measurement.

Measurement science review, 2(2):1–11.

Treacy Solovey, E., Afergan, D., Peck, E. M., Hincks, S. W.,

and Jacob, R. J. (2015). Designing implicit interfaces

for physiological computing: guidelines and lessons

learned using fnirs. ACM Transactions on Computer-

Human Interaction (TOCHI), 21(6):35.

Venables, L. and Fairclough, S. H. (2009). The inﬂuence

of performance feedback on goal-setting and mental

effort regulation. Motivation and Emotion, 33(1):63–

74.

Yuksel, B. F., Oleson, K. B., Harrison, L., Peck, E. M.,

Afergan, D., Chang, R., and Jacob, R. J. (2016). Learn

piano with bach: An adaptive learning interface that

adjusts task difﬁculty based on brain state. In Proceed-

ings of the 2016 CHI Conference on Human Factors

in Computing Systems, pages 5372–5384. ACM.

Zander, T. O., Br

onstrup, J., Lorenz, R., and Krol, L. R.

(2014). Towards bci-based implicit control in human–

computer interaction. In Advances in Physiological

Computing, pages 67–90. Springer.

Zander, T. O., Kothe, C., Jatzev, S., and Gaertner, M.

(2010). Enhancing human-computer interaction with

input from active and passive brain-computer inter-

faces. In Brain-computer interfaces, pages 181–199.

Springer.

PhyCS 2017 - 4th International Conference on Physiological Computing Systems