Recognition of Human Activities using the User’s Context and the

Activity Theory for Risk Prediction

Alfredo Del Fabro Neto

, Bruno Romero de Azevedo

, Rafael Bouﬂeuer

1,2

, Jo

ao Carlos D. Lima

1,2

Alencar Machado

, Iara Augustin

1,2

and Marcia Pasin

Informatics Graduation Program, Federal University of Santa Maria, Santa Maria, RS, Brazil

Department of Languages and Computer Systems, Federal University of Santa Maria, Santa Maria, RS, Brazil

Keywords:

Context-aware, Ubiquitous Computing, Human Activity Risk, Human Activity Performance, Activity Theory.

Abstract:

Some of the activities performed daily by people may harm them physically. The performance of such activi-

ties in an inadequate manner or in an adverse environment can increase the risk of accidents. The development

of context-aware systems capable of predicting these risks is important for human damage prevention. In this

sense, we are developing an approach based on the Activity Theory and the Skill, Rule and Knowledge model

for risk prediction of human activities in a context-aware middleware. To predict the risk in the activities, we

identify the probability for the next actions and compare the current physiological context with its future state.

In order to concept proving the proposed model, we developed a prototype and tested it with a public and a

private dataset. The results show that the proposed model can assign an appropriate risk factor to the tested

activities.

1 INTRODUCTION

It is often the case when an activity of daily living

(ADL) (Katz et al., 1963) results in an injure for the

person performing it. The reasons vary among exter-

nal factors, such as uneven ﬂoors and stairs, slippery

ﬂoors and low illumination, and/or personal factors,

such as a bad performance or an unusual physiolog-

ical condition (e.g., high blood pressure). It is spe-

cially true for the elderly, where they usually have a

weaker health due to problems such as sedentarism,

insomnia, osteoporosis, etc. Combining them with

the external factors, the risk of physical damage is in-

creased.

This way, we notice the need for a system capable

of predicting and acting over situations that present

some risk of injure to the user. An approach for

addressing this matter is by analysing the user’s be-

havior over time in order to identify anomalies either

in his physiological, performance and/or environment

situation while an activity is being performed.

Considering that the context-aware system must

take into account the user’s behavior, we used an

approach based on the Skill-Rule-Knowledge (SRK)

(Rasmussen, 1983) framework for determining the

performance level of the user in an activity and the

Activity Theory (AT) (Kuutti, 1996) for modeling ac-

tivities and its predictions. Both models act over con-

textual information classiﬁed according to the taxon-

omy of context (Kofod-Petersen and Cassens, 2006),

allowing for a well-deﬁned separation of the attributes

involved in the risk analysis.

This paper is structured as follows: Section 2

presents the core concepts that are essential for our

work; Section 3 shows related works regarding activ-

ity and action prediction, risk prediction and perfor-

mance measurement; Section 4 depicts our risk anal-

ysis model and how the related components are im-

plemented; Section 5 presents the model evaluation

by using two scenarios: one for risk in actions and

another for performance inference; ﬁnally, Section 6

draws our ﬁnal considerations and some future work.

2 CORE CONCEPTS

2.1 Giving Meaning to Contextual Data

In order to detect human activities, it is necessary to

ﬁrst detect the context and the changes that occur in

an ubiquitous environment. For this purpose, in this

work, we are modeling the context information using

the Hyperspace Analogue to Context (HAC) model

282

Neto, A., Azevedo, B., Bouﬂeuer, R., Lima, J., Machado, A., Augustin, I. and Pasin, M.

Recognition of Human Activities using the User’s Context and the Activity Theory for Risk Prediction.

In Proceedings of the 18th International Conference on Enterprise Information Systems (ICEIS 2016) - Volume 2, pages 282-289

ISBN: 978-989-758-187-8

(Rasch, 2013). The major advantages of using the

HAC model are the well-deﬁned syntax and the op-

erations that allow dealing with context data. It uses

multiple dimensions to characterize the contexts in

smart environments. These dimensions can be either

numeric or nominal with values that range between

deﬁned thresholds. This context model makes eas-

ier the understanding of the environment, since it can

capture all the context changes to assist in the detec-

tion of activities and to understand the users’ behav-

ior. Thus, the historical context information of the

user and the context information of the environment

can be used to determine the risk of all actions that

compose the activity that is being performed.

2.2 Activity Theory

To better comprehend what an activity is and its rela-

tion to the user and the environment we use Kuuti’s

approach to represent the relationship between the

components of the Activity Theory (AT) (Kuutti,

1996). In the AT, an activity is composed by actions,

which are atomic units meaning that they are com-

plete by themselves. For instance, the activity of tak-

ing a shower can be decomposed into several actions,

such as: taking the clothes off in the bathroom, enter-

ing the bathtub, turning on the water ﬂow, etc. We

chose the AT because it considers not only the ac-

tions composing the activity, but also the interactions

with other elements. For instance, the artifact used to

achieve the goal desired by the subject. For this work,

those interactions are important in the sense that they

collaborate to measure the subject’s performance and

his interactions with the environment.

The basic notion behind the Activity Theory is

that the subject is participating in an activity because

he wants to achieve some speciﬁc goal. His interest

is focused on an activity’s object that he wants to use

and/or modify in order to achieve an expected result.

The interaction between the subject and the object is

mediated by tools. This way, a basic triangle of sub-

ject, object and mediation by artifact.

The context taxonomy proposed has a pragmatic

view of artifacts’ construction and incorporates to the

context-aware systems the general concepts found in

the Activity Theory (Mikalsen and Kofod-Petersen,

2004), allowing for a well-deﬁned separation of the

user’s context.

2.3 Cognitive Workload Framework

The behavioral model Skill, Rule and Knowledge

(SRK) was created with the intention to represent the

performance of human activities based on the cogni-

tive workload required for the development of a par-

ticular activity (Rasmussen, 1983). This classiﬁcation

is deﬁned in three levels representing the conscious

control or planning used during the activity perfor-

mance: (i) skill based behavior (SBB) the subject

performs the activity in an automatic way, without

extensive mental and cognitive workloads, e.g., tak-

ing the clothes off in the bathroom; (ii) rule based

behavior (RBB) the subject performs a known ac-

tivity in a familiar situation, but has the need to re-

trieve a stored rule learned previously, e.g., the se-

quence for taking a shower: taking the clothes off in

the bathroom, entering the bathtub and then turning

on the water ﬂow; and (iii) knowledge based behav-

ior (KBB) the subject needs to make an internal map

of the environment to create a plan. The plan indicates

the way the activity should be carried on.

2.4 Measuring Performance

According to Craven et al. (Craven et al., 2007), the

cognitive workload spent can be estimated in a sub-

jective way or from observable characteristics. The

subjective ways of estimating the cognitive workload

are: (i) by forms and/or reports that are usually ﬁlled

by the user himself and (ii) the experts evaluation of

the user’s performance. This way, for this work, we

consider this approach as invasive because the user

has to report after he performed the activity. There-

fore, it is inappropriate for a context-aware system

based on the concepts of ubiquity.

Observable characteristics are used for the estima-

tion by the user’s performance related to the difﬁculty

of the task and by neurophysiological responses mea-

sured by sensors. The techniques for the latter sup-

pose that the cognitive workload is reﬂected by physi-

ological variables, such as heart activity, brain activity

and eye activity (Paas et al., 2003).

The cognitive workload estimated by the perfor-

mance of the person can be subdivided in two sub-

classes: (i) primary-task measurement, where there

is a direct measurement of a performance and (ii)

secondary-task measurement by adding some sec-

ondary activity while the user is performing the pri-

mary activity (Craven et al., 2007) (Paas et al.,

2003), that is, the level of success of the user in the

secondary-task dictates how hard it is for him the pri-

mary activity (Craven et al., 2007)(ODonnell, 1986).

For the primary-task measurement, we can assume

that when the cognitive workload increase, the ex-

tra needed resources and processing capacity of the

person will degrade the quality of his performance

(ODonnell, 1986). This way, it is possible to use

an approach that considers only one aspect relevant

Recognition of Human Activities using the User’s Context and the Activity Theory for Risk Prediction

283

for an activity (e.g., number of errors, duration or

speed of the performance) or an approach that con-

siders many aspects for a more accurate estimation.

Since this work is based on concepts of ubiq-

uitous computing, the secondary-task measurement

technique is also not adequate, because the need for

the user to perform a secondary-task for the estima-

tion of the performance of the primary-task is intru-

sive. That is, the system would have to always add

some secondary-task, this way the user would always

remember its existence. Therefore, we can notice that

the most adequate techniques for this work are the di-

rect measure of the primary activity and the usage of

physiological data.

3 RELATED WORKS

A system for classiﬁcation of emergency situations

for people that risk their lives in the line of duty, such

as the ﬁremen and the Civil Protection rescuers is pre-

sented in (Curone et al., 2010). The operators are

equipped with two sensors in their protection clothes,

an accelerometer and an ECG sensor. The system is

composed by a classiﬁer capable of recognizing many

user states that correspond to many ADLs in real time.

Tests were conducted in laboratory and the presented

system had about 88.8% of accuracy in the activities

classiﬁcation.

In the work (Wang et al., 2014), an activity is de-

ﬁned as the combination of the trajectory and dura-

tion and an abnormal activity is deﬁned as the ac-

tivity that deviates signiﬁcantly from the trajectories

and durations of the normal activities. In order to de-

termine the normal behavior of the user, the authors

performed a frequent pattern mining to ﬁnd the pat-

terns of normal activities considering their duration

and trajectory. This way, if the frequency of an item

set (in this case, it is considered as an activity) ex-

ceeds the minimum threshold deﬁned, it is classiﬁed

as a normal activity. In an environment simulated by

software, the accuracy was 96.2% (Wang et al., 2014).

A proposal for the prediction of household activi-

ties in a smart home is presented in (Gil-Quijano and

Sabouret, 2010). The goal of the authors is to adapt

the behavior of the house applications from the pre-

dicted human activities, in order to correct the behav-

ior of devices and prepare the rooms to receive people

in a pleasant condition to them. The proposal for the

activities prediction is based on the construction of a

directed graph for each occupant from the statistical

analysis of the activities performed by him. This way,

since each task is performed in a given environment,

it is possible to predict the next displacement in the

graph from the current task, characterizing the pre-

diction of activities.

4 RISK ANALYSIS MODEL

The proposed model in this paper represents the layer

Activity Manager of a developed middleware pre-

sented in previous works (Neto et al., 2013)(Neto

et al., 2014), and has as goal the realization of dif-

ferent tasks, which are: (i) the detection of actions

and activities; (ii) the assignment of a risk situation

for each action; (iii) and the inference of future ac-

tions and activities, so it is possible to predict risk

situations. The structure of our model is presented

in Figure 1 and it works in the following way: after

receiving the aggregated sensor data, the ﬁrst step is

to recognize the action being performed and infer the

next action to be performed from this action and based

on the history of actions and activities executed by the

user.

Next Action Inference

Activity Prediction

Action Risk Inference sub-system

Context Change

Future Contexts

Visualizer

Action Recognition

Activity Predicted

Risk Situation Predicted

Activity Manager

Aggregated Data

Figure 1: Proposed model for the Activity Manager layer.

With the next action to be detected, two distinct

process are initiated in parallel order: (i) the activity

prediction and (ii) the action risk inference.

For the activity prediction (i), the Activity Predic-

tion component receives the probable next action and

assumes it indeed happened, and requests to the Next

Action Inference component a new future action, us-

ing as the current action the one that was previously

detected. The Activity Prediction Component repeats

this process until a sequence of actions that represents

an activity is recognized.

In order to clarify this process, one can imagine

that the Action Recognition component recognized

the current action a

, and the Next Action Inference

ICEIS 2016 - 18th International Conference on Enterprise Information Systems

284

component detects that the probable next action to be

performed by the user is the action a

. The Activ-

ity Prediction component requests to the Next Action

Inference component the probable next action using

as the base current action the action a

. The pro-

cess used to make this inference is explained in sec-

tion 4.3; (ii) besides that, the Action Risk Inference

sub-system associates a risk situation to each inferred

action based on the context changes and on the fu-

ture contexts yielded by these changes. Each of these

components are explained in the next subsections.

4.1 Activity Recognition

Our model assumes that the activities are composed

of actions and that the relationship between these ac-

tions determine how an activity happens. This way, if

a certain set of actions is performed in a established

time window, it is said that an activity has happened.

In order to model the registered activities in the sys-

tem, we used an approach similar to the one in ref-

erence (Naeem et al., 2007), which permits to deﬁne

if actions are or not mandatory and if they must be

performed in a deﬁned order.

With this, two types of actions are deﬁned: (i)

core actions and (ii) secondary actions. The former

represents the actions that are essential in order to

achieve the goal of the activity, and, therefore, nec-

essary for its recognition. The latter are actions that

are related to some activity, but are not essential for its

recognition, they are useful for adding meaning to the

activity. For an activity to be recognized, each action

that composes it must be performed in a pre-deﬁned

time window. This way, an initial time window is de-

termined and in the preliminary phase it is adjusted

for the system’s calibration. Since this window de-

pends on the frequency of each activity, it varies ac-

cording to the user’s behavior.

4.2 Action Recognition

The component Action Recognition receives the ag-

gregated data from each type of sensor and, based on

classiﬁcation algorithms, compares the received data

with the already classiﬁed data for each action reg-

istered in the system. In short, the set of sensor data

will be classiﬁed according to the similarity that it has

with the registered data for each action. The authors

used the software WEKA in order to apply these al-

gorithms over the raw data, with default patterns as-

sociated with each of the classiﬁers and applied to the

data after the feature extraction process. The algo-

rithm that was able to classify the highest number of

samples correctly was the K-star and was the choosed

algorithm for our model.

4.3 Next Action Inference

The inference of the next action to be performed by

the user is based on the historical data of his already

performed actions in order to reﬂect his usual behav-

ior. For such, the algorithm 1 is based on the search

for patterns of activities in the history H of the user,

that is, the search for certain sequences of activities

with the objective of discovering which action is the

next one to probably be executed after these patterns.

Require: Max Pattern Length MPL

Require: History of performed actions H

1: N

← initZero()

2: A ← H.getLastActions(MPL);

3: P ← getPatterns(A);

4: for all a ∈ A do

5: for all p ∈ P do

6: N

← getNumOccurrences(H, a, p);

7: end for

8: end for

9: ap ← max(N)

10: return ap

Algorithm 1: Algorithm for the action prediction.

This way, the algorithm 1 searches the list A of

the last MPL performed actions, where MPL is the

window size or the quantity of actions to be analyzed.

Afterwards, a search is made for the list of patterns p

in the user’s history for each of the actions in A pre-

viously found. Thus, the number of occurrences for

each of the actions a is updated for each pattern p

found and related to at most MPL periods with each

action a. The highest occurrence found is the proba-

ble future action.

4.4 Action Risk Inference

The determination of risk situations in actions and ac-

tivities proposed assumes that each user has his own

behavior pattern, since people are considered beings

of habits (da Rocha et al., 2010). In order to deter-

mine the risk, the changes that each activity causes in

the context has to be analyzed. It is worth mentioning

that this analysis has to occur before an action is ex-

ecuted by the user. That is, it is necessary to predict

the actions and, consequently, activities (composed of

actions) that could be performed and this way identify

if the user will be in a risk situation when he performs

a certain action or activity. This approach implies in

the need of (i) capturing the current user context, (ii)

inferring which is the next action to be executed, (iii)

Recognition of Human Activities using the User’s Context and the Activity Theory for Risk Prediction

285

applying the context changes resulting from this ac-

tion in the current context and (iv) evaluating the re-

sulting context while looking for risk situations.

In order to achieve this goal, we proposed a solu-

tion to predict actions with a (i) component to dis-

cover the context changes and a (ii) component to

simulate future contexts resulting from these changes

(Action Risk Inference sub-system, Figure 1). From

the predicted action, the Context Change component

searches in the user’s historical data which context

changes were previously caused by it. Therefore,

the Future Contexts Visualizer applies the context

changes found in the current user context in order to

generate a new context that represents the future state

of the current context if the predicted action is per-

formed. Based on this future context, the Future Con-

texts Visualizer looks in the user proﬁle to see if it is

not outside the safety thresholds preset for the user. If

it is not, the user is considered to be in a risk situation.

4.5 Performance Inference and

Prediction

The proposed model presented in Figure 2 is used

for the inference and prediction of the user’s perfor-

mance. In order to accomplish it, the proposed model

uses the task context and the environmental context to

determine the user’s performance considering his his-

torical events. The user’s current activity is composed

by actions, as seen in Section 2.2, which are used by

the Performance Properties Estimation component to

estimate the values for the performance properties in-

volved while the activity is still being performed. This

analysis in real time is important because it allows

the system (or application) to give to the user rele-

vant information about the current activity that may

aid him during his performance. Application domains

can cover emergency situations, recommendation sys-

tems, decision support systems, among other domains

that rely on user’s performance and behavior.

In our model the inﬂuence of actions is repre-

sented by changes in the values of the performance

properties. This way, the Performance Properties Es-

timation component uses this approach in order to es-

timate in real time the changes that each new action

cause in the current activity. For the estimation of the

current activity’s performance, it is necessary to con-

sider actions that were not yet performed and predict

the changes they will possibly make in the activity.

These actions are inferred by the component Activ-

ity Manager of the middleware, presented in previous

works (Neto et al., 2013)(Neto et al., 2014), and used

together with the already elapsed actions (i.e. already

detected by the system) to give a better estimation.

Then, the estimated values for the performance

properties are related to the environmental context

considering the past events when the current activity

was performed. This relationship is used for measur-

ing how each property indicates how the user is sen-

soring the environment. The sensoring is analogous

to the SRK’s signals, signs and symbols, which repre-

sent how well known is the surroundings of the user

while performing a speciﬁc activity. So, sensorings

resulting, for example, in symbols are related to un-

familiar situations for the user, which means that the

estimated values for the performance properties or the

values from the environmental context are not in the

patterns found in the past events of the activity (i.e.,

they are outliers).

However, for the cognitive workload inference, it

is important for each performance property to have its

value adjusted according to their relevance. The rele-

vance level of the properties is taken into account due

to the fact that they may or may not be relevant de-

pending on the user and the activity being performed.

For example, a performance property like anxiety may

not be important for the activity brushing teeth. This

can be noted by the fact that the user’s performance is

not dependent on the performance property.

The relevance for each property is measured ac-

cording to the strength of the correlation of their

values with their past sensorial inferences, the Pear-

son’s correlation between the performance property

duration and the sensorial inference, measured us-

ing the Local Outlier Probabilities (LoOP) technique

(Kriegel et al., 2009). The strength of such correlation

shows how important the performance property is for

the sensoring inference, this way we can use it as the

relevance level of the property.

LoOP is an outlier detection method that provides

an outlier score in the range of [0, 1] that is directly

interpretable as a probability of a data object for be-

ing an outlier. We use a detection outlier technique

based on outlier factor due to the fact that it enables

us to gather evidence of how is the user’s behavior.

By using a technique that tells us the probability that

some data object is an outlier, we can obtain a prob-

ability of how inadequate is the user’s behavior. In

fact, it is possible to make an analogy with the SRK

model, where LoOP values close to 0 indicate SBBs

and values close to 1 indicate KBBs, whereas values

in between indicate RBBs.

5 MODEL EVALUATION

The detection of risk situations proposed in this work

is based on the prediction of actions performed by an

ICEIS 2016 - 18th International Conference on Enterprise Information Systems

286

Current

Activity

Environmental

Context

Performance Properties

Estimation

actions

Elapsed

Inferred

Sensorial Inference

of the Properties

Historical

Events

Relevance Level

of the Properties

Cognitive Workload

Inference

SRK

Figure 2: Model for performance inference.

user’s previous behavior, as well as in context changes

yielded by such action. In this sense, it is necessary

the system to be (i) able to correctly predict the proba-

ble next actions to be executed, as well as (ii) estimate

if the context changes yielded by an action will imply

in a risky context for the user. In order to validate

the proposal of this work, we conducted two distinct

experiments. The ﬁrst one intends to analyze the ac-

curacy of the presented prediction model, while the

second one intends to validate the model for the de-

tection of risk situations based on the context changes

yielded by actions in the user’s context.

5.1 Evaluation of Actions Prediction

In order to evaluate the proposed model in this work,

we opted for the usage of public dataset, called Aruba

Dataset (Cook, 2011), because it allows the results of

the model of actions predictions to be compared to

other correlated proposals, since it is a widely used

dataset in researches of activity recognition in the

ubiquitous computing area. The dataset has 11 dif-

ferent activities registered using 42 sensors. Thus, in

this work, the obtained accuracy from the used dataset

was 78.69%.

5.2 Evaluation of the Risk Situation in

Actions

The evaluation of risk in actions was made in a dataset

of our own, since we did not ﬁnd public datasets with

relevant information, that is, with annotated actions

and some user’s physiological data. In this sense, the

dataset is composed by the actions walking, sitting,

running, lying and standing, which where captured

from accelerometer and gyroscope data coupled in a

smartphone. The physiological data gathered was the

heart rate obtained from a sensor connected to an Ar-

duino.

This way, from the 2455 entries, the model de-

tected 49 risk situations and had an accuracy of

98.94%. This accuracy was measured based on

the values true-positives (36), true-negatives (2393),

false-positives (13) and false-negatives (13). These

values were obtained from the analysis of the com-

parison between the values for the current heart rate,

the predicted thresholds for the current action and the

real thresholds for the current action. The predicted

thresholds are determined based on the preceding ac-

tion, such that it is used the median of the context

changes performed by it and the value of the heart

rate while it was being developed.

5.3 Performance Evaluation

The experiments were also performed on Aruba’s

dataset. For our experiment, the duration of the ac-

tivities was considered as the performance property

and the temperature as the environmental property.

We performed our tests for all the activities. How-

ever, here we present the results only for the activ-

ity Housekeeping due to issues regarding space in this

paper. In Table 1 is shown the values used for the

cognitive workload (C. W.) inference and if the re-

sult is considered a SBB, RBB or KBB. In order to

choose between them, we deﬁned manually thresh-

olds for the C.W. (based on the SRK analogy with the

LoOP, presented in Subsection 4.5): 0 −0.24 as SBB,

0.25−0.74 as RBB and 0.75−1.0 as KBB. We under-

stand that this values are arbitrary and that they must

be adjusted by some learning algorithm, however this

is not the current focus of our work.

Table 1: Example of obtained values for some entries for

the activity Housekeeping.

LoOP r

C. W. SRK

0.00 0.41 0.0 SBB

0.67 0.41 0.27 RBB

0.10 0.40 0.04 SBB

0.06 0.41 0.03 SBB

0.00 0.41 0.0 SBB

As can be observed in Table 1, one of the val-

ues was inferred as RBB, which indicates an unusual

behavior for the performance. In this case, the du-

ration is unusual for the activity Housekeeping . It

Recognition of Human Activities using the User’s Context and the Activity Theory for Risk Prediction

287

Table 2: Comparison of the related works with our approach.

Work Accuracy Dataset

Category

Algorithm Attributes

Act. Pred. Risk Det.

(Curone et al., 2010) 88.8% Own (in lab) X Rule Based

Accelerometer, ECG

(Wang et al., 2014) 96.2% Own (by software) X Distributed

Trajectory, Duration

[Gil-Quijano, 2010] 61.28% Aruba X Directed Graph Action Sequence

Our Work

78.69% (act. pred.)

98.94% (risk det.)

Aruba (act. pred.)

Own (risk det.)

X X

Patterns (act. pred.)

Thresholds (risk det.)

Action Sequence,

Physiological Context

is interesting to notice that the value for the C. W.

inferred was 0.27, which is almost a SBB behavior,

even though the LoOP is much higher (0.67). This

happens because the performance property duration

has a relevance of only 0.41, which shows us that it is

not a very relevant performance property.

Therefore, with our model for performance infer-

ence using the LoOP technique we can associate the

user’s behavior with the theoretical SRK framework

for the understanding of the user’s behavior. Also,

by using the LoOP technique and detecting automati-

cally the relevance for the performance properties, our

model adjusts itself according to the behavior of each

different user.

5.4 Comparison

In order to compare our proposal with the related

works presented in subsection 3, we considered some

aspects, such as: accuracy of the approach, dataset

used, category (for risk detection or action predic-

tion), algorithm used and attributes used for the risk

detection. Table 2 summarizes this comparison.

In reference (Curone et al., 2010) the detection

of risk situations is made by the usage of a pre-

determined set of combinations between known ac-

tivities, similar to a rule system. For this system to

be able to identify new kind of risk situations, it is

necessary the addition of new possible combinations

between activities. In our approach, we consider the

variation in the user’s physiological data while he is

performing some activity, which allows the deﬁnition

of adjustable thresholds (for the risk situation detec-

tion) based on the user’s history. That is, the system is

able to adapt itself to changes in the user’s execution

of his activities, making the system more ﬂexible.

The work (Wang et al., 2014) uses an approach

similar to ours, which considers deviations in the

user’s normal behavior as a risk situation. However,

in such work, the authors only consider the trajectory

and the duration of the activity’s execution. This way,

they do not account for the physiological aspects of

the users that are performing activities, thus, risk situ-

ations related to changes in such physiological aspects

are not detected.

The approach based on the usage of a directed

graph for action prediction that considers as param-

eter for the measurement of the probability the ratio

between the number of times that a sequence (two ac-

tions) was performed by the person and the number of

times that he performed the initial action in the same

edge (Gil-Quijano and Sabouret, 2010), was worse

than our approach because it only considers the last

action performed for the inference of the next action.

In this sense, we obtained better results by using an

approach that allows the discovery of an appropriate

pattern length for each case (based on the user’s his-

torical data), which can be used to consider not only

the last action, but also a higher number of previously

performed actions.

This way, considering the algorithm 1, the

best value for the MPL is 2 with an accuracy of

78.69%, higher than the result obtained from the di-

rected graph algorithm proposed in (Gil-Quijano and

Sabouret, 2010), which was 61.28% for the same

dataset. This represents a gain of 28.41% in the fu-

ture actions inference.

6 FINAL CONSIDERATIONS

The prediction of risk situations is important in order

to allow context-aware system to act in a preventive

manner, aided in the user’s decision making. Thus,

this work presented a model for the action prediction

and detection of future risk situations based on the

Activity Theory and on the Hyperspace Analogue to

Context. The used techniques were superior to other

related works, since in the actions prediction we ob-

tained an accuracy of 78.69% and in the evaluation of

risk situations we obtained a accuracy of 98.94%.

In some cases the physiological state may not be

abnormal, however the user can be in a risk situation

due to the fact that his performance might be inﬂu-

enced by some external factor (e.g., if the user is in

a strange environmental condition) or some internal

factor (i.e., his mental context). Since the SRK frame-

work considers both factors, it was used in our model

for the cognitive workload inference and performance

ICEIS 2016 - 18th International Conference on Enterprise Information Systems

288

estimation. In order to evaluate the model we used a

public dataset and veriﬁed that the approach used is

capable of classifying activities according to the SRK

model, which is useful for the risk analysis.

In future works, we intend to create an adaptive al-

gorithm that learns with the user’s behavior and adjust

the values for the thresholds used for the classiﬁcation

of a behavior as skill, rule or knowledge. We also in-

tend to improve the approach for actions prediction,

considering the evaluation of the performance of the

algorithms. Besides that, we intend to perform tests in

public datasets with a higher number of user’s physio-

logical information in order to allow a more complete

evaluation of the calculated risks.

REFERENCES

Cook, D. (2011). Learning setting-generalized activity

models for smart spaces. IEEE Intelligent Systems.

Craven, P. L., Belov, N., Tremoulet, P., Thomas, M., Berka,

C., Levendowski, D., and Davis, G. (2007). Cog-

nitive workload gauge development: comparison of

real-time classiﬁcation methods. Foundations of Aug-

mented Cognition, pages 75–84.

Curone, D., Tognetti, A., Secco, E. L., Anania, G., Car-

bonaro, N., De Rossi, D., and Magenes, G. (2010).

Heart rate and accelerometer data fusion for activ-

ity assessment of rescuers during emergency interven-

tions. Information Technology in Biomedicine, IEEE

Transactions on, 14(3):702–710.

da Rocha, C. C., Lima, J. C. D., Viera, M., Capretz, M. A.,

Bauer, M. A., Augustin, I., and Dantas, M. A. (2010).

A context-aware authentication approach based on be-

havioral deﬁnitions. In IKE, pages 178–184.

Gil-Quijano, J. and Sabouret, N. (2010). Prediction of hu-

mans’ activity for learning the behaviors of electri-

cal appliances in an intelligent ambient environment.

In Web Intelligence and Intelligent Agent Technology

(WI-IAT), 2010 IEEE/WIC/ACM International Con-

ference on, volume 2, pages 283–286. IEEE.

Katz, S., Ford, A. B., Moskowitz, R. W., Jackson, B. A., and

Jaffe, M. W. (1963). Studies of illness in the aged: the

index of adl: a standardized measure of biological and

psychosocial function. Jama, 185(12):914–919.

Kofod-Petersen, A. and Cassens, J. (2006). Using activity

theory to model context awareness. In Modeling and

Retrieval of Context, pages 1–17. Springer.

Kriegel, H.-P., Kr

oger, P., Schubert, E., and Zimek, A.

(2009). Loop: local outlier probabilities. pages 1649–

1652.

Kuutti, K. (1996). Activity theory as a potential framework

for human-computer interaction research. pages 17–

44.

Mikalsen, M. and Kofod-Petersen, A. (2004). Representing

and reasoning about context in a mobile environment.

pages 25–35.

Naeem, U., Bigham, J., and Wang, J. (2007). Recognis-

ing activities of daily life using hierarchical plans. In

Smart Sensing and Context, pages 175–189. Springer.

Neto, A. D. F., Bouﬂeuer, R., Romero de Azevedo, B.,

Augustin, I., Carlos D. Lima, J., and C. Rocha, C.

(2013). Towards a middleware to infer the risk level

of an activity in context-aware environments using the

srk model. In UBICOMM 2013, The Seventh Interna-

tional Conference on Mobile Ubiquitous Computing,

Systems, Services and Technologies, pages 38–42.

Neto, A. D. F., de Azevedo, B. R., Bouﬂeuer, R., Lima,

J. C. D., Augustin, I., and Pasin, M. (2014). An ap-

proach based on activity theory and the srk model for

risk and performance evaluation of human activities

in a context-aware middleware. In Proceedings of the

13th International Conference on Mobile and Ubiqui-

tous Multimedia, pages 40–47. ACM.

ODonnell, R.D. & Eggemeier, F. (1986). Workload assess-

ment methodology. In Boff, K., Kaufman, L., and

Thomas, J., editors, Handbook of perception and hu-

man performance., volume 2, pages 42/1–42/49. John

Wiley & Sons.

Paas, F., Tuovinen, J. E., Tabbers, H., and Van Gerven, P. W.

(2003). Cognitive load measurement as a means to

advance cognitive load theory. Educational psycholo-

gist, 38(1):63–71.

Rasch, K. (2013). Smart assistants for smart homes.

Rasmussen, J. (1983). Skills, rules, and knowledge; sig-

nals, signs, and symbols, and other distinctions in hu-

man performance models. Number 3, pages 257–266.

IEEE.

Wang, C., Zheng, Q., Peng, Y., De, D., and Song, W.-

Z. (2014). Distributed abnormal activity detection

in smart environments. International Journal of Dis-

tributed Sensor Networks, 2014.

Recognition of Human Activities using the User’s Context and the Activity Theory for Risk Prediction

289