A Layered Architecture based on Previsional Mechanisms

Francesco Fiamberti, Daniela Micucci, Marco Mobilio and Francesco Tisato

Department of Informatics, Systems and Communication

University of Milano-Bicocca, Viale Sarca 336, 20126, Milano, Italy

Keywords:

Data Fusion, Software Architecture, Architectural Patterns.

Abstract:

The paper presents a layered architecture that improves software modularity and reduces computational and

communication overhead for systems requiring data from sensors in order to perform domain-related elabora-

tions (e.g., tracking and surveillance systems). Each layer manages hypotheses that are abductions related to

objects modeling the ”real world” at a speciﬁc abstraction level, from raw data up to domain concepts. Each

layer, by analyzing hypotheses coming from the lower layer, abduces new hypotheses regarding objects at a

higher level of abstraction (e.g., from image blobs to identiﬁed people) and formulates timed previsions about

objects. The failure of a prevision causes a hypothesis to ﬂow up-stream. In turn, previsions can ﬂow down-

stream, so that their veriﬁcation is delegated to the lower layers. The proposed architectural patterns have been

reiﬁed in a Java framework, which is being exploited in an experimental multi-camera tracking system.

1 INTRODUCTION

Nowadays, sensors are used in several application

domains including surveillance, machines, medicine,

and robotics. Usually such application domains rely

on solutions that strongly depend on both the sensing

devices and the application domain of the problem.

Moreover, they usually adopt data-driven policies: a

bottom-up process abstracts domain entities from raw

data and the whole processing chain is traversed every

time new information is available.

This results in huge and monolithic software sys-

tems presenting several drawbacks: lack of modu-

larization and stratiﬁcation (reuse is difﬁcult or even

impossible); complex conﬁguration of the ﬁeld in-

stallations (mostly implying some intervention on

the software); difﬁcult coexistence of different algo-

rithms; computational overhead (e.g. in a surveillance

system, image processing algorithms are performed

from scratch for every acquired image); communi-

cation overhead (the whole chain is traversed each

time); lack of timeliness (“reasoning” is performed

only after something happened) and missing reac-

tions (sometimes anomalous situations should be rec-

ognized when nothing is perceived).

To overcome such issues, this work addresses the

problem from an architectural perspective, proposing

ALARM (A Layered Architecture based on pRevi-

sional Mechanisms), a layered architecture that is in-

dependent of the speciﬁc application domain and of

the sensors’ characteristics. This makes the architec-

ture modular, scalable and open.

ALARM is based on the following basic concepts:

• The system is structured in layers. Each layer in-

cludes objects that model the “real world” at a

speciﬁc abstraction level, from raw data up to do-

main concepts; objects are located in spatial and

temporal reference systems. Each layer performs

subsequent abstractions through abductive infer-

ence; the number and kind of layers is a domain-

related issue.

• Hypotheses are timestamped abductive inferences

based on objects;

• The “intelligence” of each layer relies on the

formulation of previsions about objects and on

their veriﬁcation against hypotheses coming from

lower layers;

• Previsions have a time-dependent validity, there-

fore their veriﬁcation is time-driven;

• Failure of the veriﬁcation process generates an

exception that is notiﬁed to the upper layer

(upstream ﬂow). This is the only mechanism

allowing information to ﬂow to higher levels

(exception-driven approach);

• A line of thought is a series of semantically linked

previsions about the evolution of an object. Many

354

Fiamberti F., Micucci D., Mobilio M. and Tisato F..

A Layered Architecture based on Previsional Mechanisms.

DOI: 10.5220/0004592503540359

In Proceedings of the 8th International Joint Conference on Software Technologies (ICSOFT-EA-2013), pages 354-359

ISBN: 978-989-8565-68-6

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

lines of thought related to the same object may be

formulated;

• The veriﬁcation of a prevision may be delegated

to a lower abstraction level (downstream ﬂow).

Since ALARM focuses on information transfor-

mation, it falls in the scope of data fusion, especially

if applied in a heterogeneously instrumented environ-

ment.

Data fusion made its ﬁrst appearance in the 1960s

in the form of mathematical models aimed at data ma-

nipulation. In the subsequent years, it has been ap-

plied in various ﬁelds. Many generic models are de-

scribed in the literature accommodating the speciﬁc

applications at hand, but in each attempt the main mo-

tivation seems to to be the partitioning of the process

into sub-tasks.

The model that obtained the highest accolade in

the literature is the JDL Data Fusion Model (White,

1991) in its revised form made in (Steinberg et al.,

1999), which has been chosen as the starting point for

many data fusion applications in a wide range of do-

mains (see for example (Bruckner et al., 2012; Jot-

shi et al., 2009; Hall et al., 2008)). Other widely

used models are the Boyd Loop (Boyd, 1987), the

Intelligence Cycle (Shulsky and Schmitt, 2002), the

Behavioral Knowledge based Model (Pau, 1988) and

the Waterfall Model (Harris et al., 1998), which is

an approach close to the layering used here. More

precisely, ALARM’s layers, in a conﬁguration as pre-

sented in the proposed case study, are addressable by

the low levels of data fusion in the waterfall model.

The JDL revised model, considering the further sub-

division of the ﬁrst level proposed in (Hall and Lli-

nas, 1997) also ﬁnds a ﬁne correspondence with the

ALARM layering, which can then be freely used

while maintaining consistence with the literature.

The paper is organized as follows. Section 2 de-

scribes the patterns at the basis of the conceptual

model underlying the ALARM architecture. Sec-

tion 3 discusses the results of preliminary tests of the

proposed approach performed on a prototype imple-

mentation. Finally, in Section 4 some comments and

possible future developments are presented.

2 CONCEPTUAL MODEL

This section introduces the conceptual patterns

ALARM relies on.

2.1 The Abduction Pattern

Charles Sanders Peirce (Peirce, 1935) ﬁrstly intro-

duced the term abduction as follows:

“Abduction is the process of forming an explana-

tor hypothesis. It is the only logical operation which

introduces any new idea.”

Using Peirce words “while deduction proves that

something must be and induction shows that some-

thing actually is operative, abduction merely suggest

that something may be”. This can be read as: deduc-

tion links premises with conclusions, induction con-

structs general propositions derived from speciﬁc ex-

amples, and abduction goes from data description to

a hypothesis that tries to explain relevant evidence.

According to Pierce’s deﬁnition, in ALARM hy-

potheses represent predicates about “real objects” at

a speciﬁc level of abstraction. The abduction process

translates hypotheses from a lower abstraction level

to a higher one.

Such a behavior is at the basis of a bottom-up or

data-driven model. It is exploited by most systems,

but has several drawbacks:

• The high-level analysis of hypotheses may be

computationally intensive and is performed at the

last step of the abductive chain. This delays high-

level decisions like the recognition of abnormal

situations;

• An abnormal situation can derive from the ab-

sence of base-level hypotheses and data-driven

models usually lack the capability of recognizing

such an absence;

• The entire processing pipe is traversed each time

a base-level hypothesis is perceived. This leads to

both computational and communication overhead.

2.2 The Veriﬁcation Pattern

Given the layered structure that performs abductions,

the core idea in ALARM is to exploit hypotheses

to produce previsions. A prevision is a predicate

about one or more hypotheses that are expected to

be valid in the future; in other words, previsions rep-

resent the expected evolution of an observed situa-

tion. While hypotheses are related to the past (they

are produced when something happens), previsions

are future-related facts.

The veriﬁcation pattern is composed of two main

activities: forecasting and veriﬁcation. The former

observes hypotheses and formulates previsions as an

estimate about future behaviors; the latter is aimed at

verifying the correctness of such previsions by com-

paring them against hypotheses.

The output of a veriﬁcation process can be one of

the following:

• Match: there exist a hypothesis and a prevision

representing the same information;

ALayeredArchitecturebasedonPrevisionalMechanisms

355

• Hypothesis without Prevision: a hypothesis ex-

ists without a matching prevision;

• Prevision without Hypothesis: there is a previ-

sion without a corresponding hypothesis;

• Hypothesis / Prevision Incongruence: there are

both a prevision and a hypothesis, but they are in

conﬂict with each other.

The key concept in handling these cases is that

the hypothesis course is driven by exceptions, which

means that hypotheses at a certain level l ﬂow up to

the level l + 1 only if the veriﬁcation at l produces a

mismatch.

Mismatches are notiﬁed to the upper level as new

hypotheses, thus maintaining coherence with the as-

sumption that only hypotheses ﬂow upward. A match

does not produce any output; the case of a hypothesis

without a prevision lets the same hypothesis ﬂow up-

ward and ﬁnally a prevision without a hypothesis pro-

duces a new hypothesis stating that the correspond-

ing expected real object was not observed. As for the

fourth possibility, it is not necessary to deal with it as

an independent possibility, since it is actually a com-

bination of the other two mismatch cases.

2.3 The Delegation Pattern

The veriﬁcation pattern reduces the computational

and communication overhead but still suffers from

an architectural issue. In fact, in order to be able to

formulate meaningful previsions, the low-level activ-

ities should embed domain-related knowledge. This

would strongly affect system modularity and software

re-use: in a well-structured architecture, low-level ac-

tivities should not be aware of high-level issues, mak-

ing it hard, or even impossible, to formulate helpful

previsions at lower levels; rather, previsions at lower

levels should come from higher levels, according to

domain issues.

If the generation of previsions is restricted to

higher levels only, the information ﬂow between al-

most all the levels will increase, thus reducing one of

the advantages offered by the approach. In order to

solve this issue, one additional pattern is introduced:

the delegation pattern.

The delegation pattern consists of delegating the

veriﬁcation of previsions to a lower level than the

one at which they have been generated. It is com-

posed of two different activities: delegation and de-

duction. The former selects the previsions to be del-

egated, while the latter is in charge of actually trans-

lating them to the corresponding lower-level versions;

thus, this activity represents the inverse of the transla-

tion performed by abduction (which in logic is known

as deduction). If the obtained lower-level previsions

match lower-level hypotheses, the latter will undergo

the ﬁltering performed by veriﬁcation and will not

ﬂow up. To avoid a false mismatch at the higher level

(due to the lack of matching hypotheses if delegated

previsions have been matched at a lower level), dele-

gated previsions must be considered as matched until

one has proof of the contrary. In other terms, a level

can delegate to the lower level the falsiﬁcation of pre-

visions. The lower level can in turn delegate such pre-

visions according to the same pattern.

On one side, the approach dramatically reduces

the computation and communication overhead at the

topmost levels, thus supporting the development of

cost-effective distributed systems. On the other side,

it preserves the proper layering, thus encouraging

software re-use.

Figure 1 synthesizes the ALARM concepts in a

three-layer conﬁguration. Black lines describe hy-

pothesis ﬂows, whereas the grey ones prevision ﬂows.

Veriﬁcation, forecasting and delegation are intra-layer

activities. On the contrary, abduction and deduction

are inter-layer activities.

3 PRELIMINARY VALIDATION

To study the correctness of the ALARM approach,

some basic tests have been performed. Since the ver-

iﬁcation activity is time-driven and both hypotheses

and previsions have a relation with time, we exploited

TAM (Time Aware Machine), a Java framework that

simpliﬁes the development of time aware systems (Fi-

amberti et al., 2012).

In order to guarantee the repeatability of tests, ac-

tual sensors have been simulated by generating pre-

deﬁned observations at predeﬁned time instants and

using deterministic implementations of abducers and

forecasters.

The simulated environment is the corridor of a

building instrumented with 12 video cameras. The

test application is able to identify and track people

walking in the corridor.

Three abstraction layers have been identiﬁed:

• Ground Layer, that reasons on objects represented

by boxes with speciﬁc height and width, along

with a coordinate location and a mean color of the

box;

• Entity Layer, where boxes observed at different

times are correlated (basing on position and color)

in order to compute speed and direction of move-

ment, thereby identifying persistent entities;

• Domain Layer, where the identiﬁed entities are

ICSOFT2013-8thInternationalJointConferenceonSoftwareTechnologies

356

Strategy

Abduction

Forecasting

Abduction

Forecasting

N + 1

Forecasting

Delegation

Deduction

Delegation

Deduction

N - 1

Figure 1: A three-layer ALARM conﬁguration.

recognized as people characterized by a name and

an abstract location.

On top of the layer stack, a strategy is found, which is

notiﬁed when exceptions occur.

The goal of the tests is to measure the number of

notiﬁcations received by the strategy and the number

of data elaborations. The following scenarios have

been considered:

a a person is identiﬁed; a sequence of previsions

about the person’s future movements is generated,

one for each camera; all such previsions are con-

ﬁrmed by hypotheses.

b at ﬁrst, the behavior is the same as in the previous

scenario. However, after the sixth incoming hy-

pothesis, no more hypotheses are received by the

Ground Layer abducer, as if the person stopped in

a blind spot.

The two described scenarios will be analyzed without

delegation at ﬁrst. Afterwards, for both of them del-

egation will be introduced, in order to highlight the

corresponding beneﬁts. Figure 2 sketches the system

layering in both the situations: with and without dele-

gation. It is possible to notice that not all the activities

are present in each layer. Indeed, ALARM does not

constraint the presence of each activity in each layer.

3.1 Scenario a without Delegation

Every time a hypothesis is received by the ground

layer, it ﬂows up to the domain layer, since in the

absence of delegation no previsions are available for

veriﬁcation at the two lower layers. Thus, every in-

coming hypothesis undergoes three abductions. At

the domain layer, the ﬁrst hypothesis generates an ex-

ception, since no previsions have been formulated yet.

Using the information of the ﬁrst hypothesis, the fore-

caster is able to formulate several previsions about

what hypotheses will be received in the future. As

the time passes, the hypotheses are all matched by the

previsions, therefore no more exceptions are gener-

ated.

3.2 Scenario b without Delegation

Everything goes on as in the previous case until the

time when the seventh hypothesis is expected. The

mismatch caused by the missing hypothesis generates

a second exception at the domain layer. The fore-

caster then retracts the remaining, already generated

previsions in order to avoid future uninteresting ex-

ceptions.

3.3 Scenario a with Delegation

In this case, delegators are present at the entity and

domain layers, together with the associated deducers.

Therefore, the veriﬁcation of previsions generated by

the domain-layer forecaster is delegated to the ground

layer. Again, a single exception is generated, due to

the ﬁrst incoming hypothesis. The difference with re-

spect to the case without delegation is that now the

hypotheses after the ﬁrst one do not traverse the com-

plete abduction chain, since they are matched against

previsions already at the ground layer.

ALayeredArchitecturebasedonPrevisionalMechanisms

357

Strategy

Abduction

Entity

Abduction

Domain

Forecasting

Ground

Strategy

Abduction

Entity

Abduction

Domain

Forecasting

Delegation

Deduction

Delegation

Deduction

Ground

Figure 2: On the left, the conﬁguration without delegation pattern, on the right the one with delegation pattern.

3.4 Scenario b with Delegation

The veriﬁcation of previsions generated by the do-

main layer is delegated to the ground layer in this

case too. When the seventh expected hypothesis is

not received, a mismatch is produced and sent up-

wards. This is the only additional hypothesis that tra-

verses the complete abduction chain. Once the mis-

match is received at the domain layer, an exception is

generated, exactly as in the case without delegation.

Moreover, the forecaster retracts the remaining pre-

visions and the delegator propagates the retractions

downwards to the ground layer.

3.5 Discussion

Table 1 summarizes the results. In the table, for ev-

ery test case the total number of exceptions, abduc-

tions and previsions is shown. The corresponding re-

sults for a standard (i.e., non ALARM-based) appli-

cation structured according to the same layering (so

that incoming data traverse the same sequence of ab-

ductions) are also shown for comparison.

As the table shows, the number of generated ex-

ceptions for a given scenario is typically sensibly re-

duced with respect to non ALARM-based applica-

tions.

As for the beneﬁts of delegation, the number of

exceptions is not modiﬁed when delegation is intro-

duced. However, delegation does introduce an im-

provement in the form of reduction of computational

overhead. In fact, the complexity of the abduction

process can be expected to increase going upwards in

the layer stack, so that stopping the upward hypoth-

esis ﬂow at the lowest possible layer should greatly

reduce the amount of computations required for every

incoming hypothesis.

The table does not show the number of deductions

for the cases where delegation is present. It is easy to

see that the sum of the numbers of delegations and ab-

ductions is always equal to the number of abductions

in the absence of delegation. This can be considered

as an additional advantage of the delegation pattern,

as it is reasonable to assume that, from a computa-

tional point of view, a deduction is likely to be far

easier than the corresponding abduction.

The performed tests are, of course, in no way a

complete experimentation: they were simply aimed

at producing a preliminary quantitative analysis of the

beneﬁts of the ALARM approach and at allowing for

a ﬁrst estimate of how much the actual ﬁltering is and

where it occurs. However, these simple experimen-

tations show that using delegation leads to a sensible

improvement as far as reduction of inter-layer com-

munication and of computational overhead are con-

cerned, as expected.

4 CONCLUSIONS

In this work, the ALARM architecture has been pro-

posed as a means to overcome the typical drawbacks

of the traditional, monolithic approaches to data-

driven applications, namely computational and com-

munication overhead, and the lack of timeliness and

of the capability to recognize situations characterized

by missing data.

The proposed solution is based on a layered

ICSOFT2013-8thInternationalJointConferenceonSoftwareTechnologies

358

Table 1: Results of the preliminary tests.

Test case Exceptions Abductions Previsions

Scenario a, without delegation 1 36 11

Scenario b, without delegation 2 18 11

Scenario a, with delegation 1 14 11

Scenario b, with delegation 2 10 11

Scenario a, standard application 12 36 -

Scenario b, standard application 6 18 -

structure, where the number of layers is domain-

dependant. The internal structure of each layer is the

same and reiﬁes the patterns of abduction, veriﬁca-

tion and possibly delegation. This allows for the re-

duction of information ﬂows and computational over-

head thanks to an exception-based mechanisms which

allows information to ﬂow upwards in the layer stack

only when something unexpected happens.

In order to validate the proposed approach, a sim-

ple implementation of the general structure has been

created and used to build a three-layer test application

modeling a virtual instrumented environment. The

behavior of the test application has been observed

in several simple situations in order to obtain a ﬁrst,

quantitative estimate of the beneﬁts of this approach.

The results show how the ALARM approach actu-

ally helps in reducing computational overhead with

respect to a traditional data-driven implementation.

Its stratiﬁcation and modularization make it easy to

reuse components and adapt the architecture behavior

to ﬁt different speciﬁc needs. The hypothesis veriﬁca-

tion pattern improves responsiveness, while the dele-

gation pattern improves the decentralization of heavy

computations.

In order to give a more thorough evaluation of the

computational and informational gain introduced by

ALARM, a series of more accurate and extended tests

are needed. In particular, implementing real life cases

will be the only way to establish the actual level of

improvement.

Several open issues will be the subject of future

developments. First of all, the association of a con-

ﬁdence level to both hypotheses and previsions must

be thoroughly examined, given the unavoidable un-

certainties in both the abductive and the forecasting

processes. Moreover, the concept of delegation can

be raised at a meta-level: in many cases, the down-

stream ﬂow might consist of rules for deﬁning pre-

visions, rather than of already formulated previsions.

Finally, matching hypotheses against previsions is a

complicated issue: both are timed predicates about

objects. Predicates include operators like “similar to”,

“close to”, “simultaneous”, which can be hardly de-

ﬁned once for all. Such operators are expected to be

overloaded by speciﬁc classes of objects exploiting

domain-speciﬁc algorithms.

REFERENCES

Boyd, J. (1987). A discourse of winning and losing. Un-

published collection of lecture slides available via In-

terlibrary Loan from such sources as the Marine Corps

University Library.

Bruckner, D., Zeilinger, H., and Dietrich, D. (2012). Cog-

nitive automation - survey of novel artiﬁcial general

intelligence methods for the automation of human

technical environments. Industrial Informatics, IEEE

Transactions on, 8(2):206–215.

Fiamberti, F., Micucci, D., and Tisato, F. (2012). An object-

oriented application framework for the development

of real-time systems. In Furia, C. and Nanz, S., edi-

tors, Objects, Models, Components, Patterns, volume

7304 of Lecture Notes in Computer Science. Springer

Berlin Heidelberg.

Hall, C., McMullen, S., Hall, D., McMullen, M., and

Pursel, B. (2008). Perspectives on visualization and

virtual world technologies for multi-sensor data fu-

sion. In Information Fusion, 2008 11th International

Conference on.

Hall, D. and Llinas, J. (1997). An introduction to multisen-

sor data fusion. Proceedings of the IEEE, 85(1).

Harris, C., Bailey, A., and Dodd, T. (1998). Multi-sensor

data fusion in defence and aerospace. The Aeronauti-

cal Journal, 102(1015).

Jotshi, A., Gong, Q., and Batta, R. (2009). Dispatching and

routing of emergency vehicles in disaster mitigation

using data fusion. Socio-Economic Planning Sciences,

43(1).

Pau, L. F. (1988). Sensor data fusion. Journal of Intelligent

& Robotic Systems, 1(2).

Peirce, C. S. (1935). Collected papers of Charles Sanders

Peirce. Belknap Press.

Shulsky, A. and Schmitt, G. (2002). Silent warfare: un-

derstanding the world of intelligence. Potomac Books

Inc.

Steinberg, A., Bowman, C., and White, F. (1999). Revisions

to the JDL Data Fusion Model. In Society of Photo-

Optical instrumentation Engineers (SPIE) Conference

Series, volume 3719.

White, F. (1991). Data fusion lexicon. http://oai.

dtic.mil/oai/oai?verb=getRecord&metadataPreﬁx=html

&identiﬁer=ADA529661.

ALayeredArchitecturebasedonPrevisionalMechanisms

359