A Domotic Ecosystem Driven by a Networked Intelligence

Luca Ferrari, Matteo Gioia, Gian Luca Galliani and Bruno Apolloni

Department of Computer Science, University of Milano, Milan, Italy

Keywords:

Advanced Domotic, Internet of Things, MQTT Protocols, Learning Algorithms.

Abstract:

We describe a diffuse control system for household appliances rooted in an Internet of Thing network em-

plenty of devices with common features, yet called to satisfy in an almost repetitive way needs that may be

very diversiﬁed, depending on the user preferences. This calls for a network putting them in connection and a

cognitive system that is capable to interpret the user requests and translate them into instructions to be trans-

mitted to the appliances. This in turn requires a proper architecture and efﬁcient protocols for connecting the

appliances to the network, as well as robust algorithms that concretely challenge cognitive and connectionist

theories to produce the instructions ruling the appliances. We discuss both aspects from a design perspective

and exhibit a mockup where connections and algorithms are implemented.

1 INTRODUCTION

Control theory, as an offspring of cybernetics, is

deeply rooted in the concept of feedback (Wiener,

1948) since the forties of the previous century. In

the same period of time a different notion of control

in terms of self adapting systems emerged with the

ﬁrst modern hypotheses of the brain computing facil-

ities within the connectionist framework (Rosenblatt,

1958). These two ways of controlling a dynamic sys-

tem ran in parallel up until the nineties, when hybrid

control systems began exploiting synergies by con-

ventional controls and by either recurrent neural net-

works (Ku and Lee, 1995) or reinforcement learn-

ing algorithms (Sutton and Barto, 1998). The Inter-

net of Things paradigm suggests a renewed synergy

between the two approaches. Namely, the network

connecting things on the Web enjoys many properties

of a neural network in terms of both connectivity and

elementariness of the messages normally exchanged

between the devices and their huge number. How-

ever, rather than distributed as in the connectionist

paradigm, computations are expected to be more ef-

ﬁcient if they are performed in a centralized way, yet

exploiting the capillarity of the information the net-

work may bring them – hence we call it networked

intelligence. Per se, the machine where computations

are done is immaterial, so that we may assume them

to be carried out (and possibly distributed too) every-

where in the cloud. However, as with the multilayer

perceptron (Haykin, 1994), the information manage-

ment is dealt with better through a hierarchical archi-

tecture than through a distributed one.

This is the idea we pursue in our ecosystem.

Namely, we are devising a system constituted by an

ensemble of household appliances ruled by a social

network which is from time to time committed by a

user to operate them optimally with respect to a given

task. For instance, the user asks the network to have

trousers perfectly washed by his washing machine. In

reply, the network sends directly to the machine a se-

quence of instructions, call it recipe, such as “charge

water, heat water to 35 degrees”, etc., which drive the

machine to carry out a perfect washing. On the one

hand this a typical procedure we are used to expect

from our personal devices. On the other one, the way

of implementing the procedure is rather hard. In a

extreme synthesis we need:

1. electronics allowing the network to communicate

with the machine, possibly overriding its micro-

controller logic;

2. a logic able to produce recipes that are optimal

in respect to many criteria, from user preferences

all the way to ecological goals such as water or

electricity saving. The logic must learn how to

reach these objectives from the feedback coming

both from devices and from users. Hence it is a

cognitive system.

3. an architecture and protocols vehiculating signals

between devices and the networked intelligence in

a safe and efﬁcient way.

359

Ferrari L., Gioia M., Galliani G. and Apolloni B..

A Domotic Ecosystem Driven by a Networked Intelligence.

DOI: 10.5220/0004977603590366

In Proceedings of the 10th International Conference on Web Information Systems and Technologies (WEBIST-2014), pages 359-366

ISBN: 978-989-758-024-6

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

The good news is that the proposed ecosystem

has some appeal, so that it got funded by the Eu-

ropean Community with a horizon of 30 months

(project Social&Smart (SandS)-http://www.sands-

project.eu/). In this paper we report some of the

project’s progress as to both the architecture-and-

protocols and logics. In addition, we describe an early

mockup where instruction and signal dispatchings are

enabled by proper circuits. Speciﬁcally, in Section

2 we introduce the architecture as the backbone of

the project. In Section 3 we brieﬂy discuss proto-

cols, while in Section 4 we show the mockup in detail.

In the last section we conclude the paper by saying

where we are now and what we expect at the end of

the project.

2 SandS ARCHITECTURE

Diffuse control may be viewed as an evolution of

WEB 2.0 in terms of a social network whose goal

is the dispatching of (optimally controlled) activi-

ties rather than the providing of information services.

This ﬁts well both with the paradigm of Internet of

Things, as for architecture, and with the modern re-

word expectation from the interaction between mem-

bers within an evolved society. In this new framework

a member is spared from doing boring (because repet-

itive) activities and is enabled to enjoy better ways

of life thanks to the automatic contribution of other

members. The supporting infrastructure is a commu-

nity of personal appliances realized through their con-

nection in Internet. Let us explore these aspects in

depth.

2.1 Social Network

Social networks can be seen as a repository of in-

formation and knowledge that can be queried when

needed to solve problems or to learn procedures. The

following deﬁnition has been proposed by Vannoy

and Palvia [1] in the study of social models for tech-

nology adoption of social computing: “an ensem-

ble of intra-group social and business actions prac-

ticed through group consensus, group cooperation,

and group authority, where such actions are made pos-

sible through the mediation of information technolo-

gies, and where group interaction causes members

to conform and inﬂuences others to join the group”.

We intend to update this deﬁnition through the novel

idea of building social computing systems where part

of the computations is hidden to the social network

player, thus representing a form of subconscious com-

puting.

Namely, in our social network we will distinguish

between conscious and subconscious computing. The

former is deﬁned by the decisions and actions per-

formed by the players (i.e. the users) on the basis of

the information provided by the social service. The

latter is realized by the data processing performed au-

tomatically and autonomously by the web service in

order to search for or produce the information offered

to the social players, or to fulﬁll other purposes, such

as data mining for the advertising industry. It is an in-

telligent subconscious computing when new solutions

to new or old problems are generated on demand.

We instantiate this new social network in the

SandS project in the realm of household appliances

and domestic services. The term eahouker – meaning

easy household worker – is introduced in this context

to denote the household appliance user empowered

by the social network and social intelligence. In an

extreme synthesis the project deals with a social net-

work aimed at producing recipes with tools of com-

putational intelligence, to be dispatched to household

appliances grouped in the homes through a domes-

tic wiﬁ network. A recipe is a set of scheduled, pos-

sibly conditional, instructions (hence a sequence of

parameters such as water temperature or soak dura-

tion) which completely deﬁne the running of an ap-

pliance. They are managed by a home middleware

– called domestic infrastructure (DI) – in order to be

properly transmitted to the appliance through suitable

protocols. The entire contrivance is devised to opti-

mally carry out ordinary housekeeping tasks through

a proper function of house appliances with a minimal

intervention on the part of the user. Feedbacks are

sent by users and appliances themselves to the net-

work intelligence to close the permanent recipe op-

timization loop, with ofﬂine tips and advices on the

part of the appliance manufacturers. An electronic

board will interface each single appliance to the DI

(see Fig. 1 (Apolloni et al., 2013))

2.2 The Architecture

This architecture has head in the cloud and feet on the

appliances. The general scheme is the following (See

Fig. 2):

• user and appliances located in a house;

• both of them are interfaced to Internet through a

home router: the latter as machines endowed with

some transmission device, the former comfortably

sitting on a recliner, sending short orders to DI

from time to time;

• many houses refer to the same web-services

provider. The connection of the router to the

WEBIST2014-InternationalConferenceonWebInformationSystemsandTechnologies

360

Figure 1: A SandS project synopsis. eahouker is a combi-

nation of the words easierly and houseworker.

Figure 2: SandS domotic system.

provider is per usual, the internal networking of

the communication is committed to a special pro-

tocol and a concentrator board if necessary;

• services are delivered in the frame of a social net-

work.

With respect to this consolidated scenario, the pe-

culiarities of our system are:

1. We look for a system as uninvasive as possible.

This entails that extra-hardware must be reduced

to the minimum; hence: the appliances to be as

usual supplied by the stores plus the interfacing

chip (Arduino board in the provisional solution

we propose); no need for having a computer on,

no set-top box.

2. We look for a system as undemanding as possi-

ble. This calls for plug&play procedures when

an appliance is installed in or removed from the

home. Analogously, the need for user input of

data must be kept to a minimum; likewise the ba-

sic commands for activating the appliances. The

way of inputting data is immaterial; for instance

by smartphone, tablet, pc or other speciﬁc gadget.

3. we look for extremely adaptive services/recipes.

This passes through two functional blocks: a so-

cial network which issues recipes on demand to

the single user and a DI which administers the

recipe according to rules on a home-by-home ba-

sis.

4. We look for a system that is transparent to the

user. Hence, by deﬁnition, the DI is on the cloud,

as is the social network. Transparency binds the

entire chain from user to social network and back.

5. We look for an intelligent system. Here the in-

telligence of social network members is involved

ﬁrst when a service is initialized and then regu-

larly when the user sends fuzzy feedbacks. While

this is insufﬁcient for creating a true intelligent

system, we can ﬁll up this task with a series of

algorithms on a proper eahouker database (EDB)

that constitute the Networked Intelligence (NI) in

the core of the social network that is assigned to

issuing recipes.

2.3 A Functional Layout

Fig. 3 gives an intuitive representation of the inter-

actions between the system elements (Grana et al.,

2013). The SandS Social Network mediates the in-

teraction between a population of users (the eahouk-

ers – hence ESN the name of the social network),

each one with his/her own set of appliances. ESN

has a repository of tasks that have been posed by the

eahoukers and a repository of recipes for the use of

appliances. These two repositories are related by a

map between (to and from) tasks and recipes. This

map does not need to be one-to-one. Blue dashed ar-

rows correspond to the path followed by the eahouker

queries, which are used to interrogate the database

of known/solved tasks. If the task is already known,

then the corresponding recipe can be returned to the

eahouker appliance (solid black arrows). The ea-

houker can express his/her satisfaction with the re-

sults (dashed blue arrows). When the queried task is

unknown and unsolved then the social network will

request a solution from the SandS Networked Intelli-

gence that will consists in a new recipe deduced from

past knowledge stored in the recipe repository. This

new solution will be generated by intelligent system

reasoning. The repository of recipes solving speciﬁc

tasks can be loaded by the eahoukers while comment-

ing among themselves on speciﬁc cases, or by the ap-

pliance manufacturing companies as an example of

use to foster sales by offering customer additional ap-

pealing services. These situations correspond to the

ADomoticEcosystemDrivenbyaNetworkedIntelligence

361

Figure 3: Social and Smart system functional layout.

conscious computing done on the social web service

by human agents. The role of the Networked Intelli-

gence is to provide the subconscious computing that

generates innovation without eahouker involvement.

2.4 A Conceptual Map of a SandS

Session

Figure 4 contains the conceptual map graph (Gra

et al., 2013). The main elements are highlighted

in red: the eahouker and the appliance, in fact all

SandS, is designed to mediate between them. Green

boxes contain explicitly active computational mod-

ules such as the natural language processing module,

the task and recipe managers, the networked intelli-

gence and the domestic middleware. The blue circle

highlights the instrumental key of the system: the ap-

pliance recipe. The magenta box denotes a hidden

reinforcement learning module which the eahouker is

not aware of. The SandS session is started by the ea-

houker stating a task in natural language. The natu-

ral language processing module analyzes this expres-

sion obtaining a task description which is suitable for

a formal search in databases. The task manager ex-

plores a task database looking for the best match to

the proposed task. If there is an exact match life will

be easy for the recipe manager which needs only to

retrieve the corresponding recipe. In general, the task

manager will select a collection of best matching task

descriptions to be presented (or not) to the eahouker

to assess the accuracy of the interpretation of his/her

intentions by the system. The eahouker may agree to

the best matching task descriptions. The recipe man-

ager reads them and continues to explore the recipe

database looking for best matches, or proceeds to ask

the networked intelligence for the enrichment of the

recipe database with new solutions that may better

fulﬁll the task posed by the user. The recipe manager

produces a selection of recipes and a best matching

Figure 4: Description of a SandS session by a conceptual

map.

recipe. For the engaged user, the selection of recipes

may allow him/her to either ponder them and inﬂu-

ence the recipe choice or simply trust on the ﬁrst man-

ager selection by default. This may even be an ad-

ditional source of feedback to the networked intelli-

gence.

When the recipe is selected, it is downloaded to

the appliance via the domestic middleware, which

controls its execution. This includes any communica-

tion with the user to operate the appliance (i.e. open-

ing the appliance door). The domestic middleware

produces a monitoring followup of appliance func-

tion that may be shown to the user to keep him/her

informed of progress, expected time to completion,

etc. The appliance produces a ﬁnal result, which is re-

turned to the eahouker. Then the eahouker expresses

his/her satisfaction, which is the main feedback for all

processes.

3 THE TELECOMMUNICATION

INFRASTRUCTURE

Appliances and DI are permanently conneced in or-

der to send each other status information and com-

mands. A persistent connection is the best solution

for an event oriented project like this. Events can be

triggered by either DI side or by the appliance side

and information can be sent immediately using the al-

ready established connection.

Information is secured by cryptographic functions

and encapsulated in MQTT (http://mqtt.org/) frames

at application layer. The connections, ciphering and

MQTT encapsulation are managed by a connection

manager module on the DI as shown on the ﬁgure 5.

Communication between DI and appliances

should be encrypted to prevent that an attacker can

get information or control the appliance. The wireless

WEBIST2014-InternationalConferenceonWebInformationSystemsandTechnologies

362

Figure 5: Information exchange between DI and appliance.

network of the user would be secured with a WPA2

system, and so an attack from someone not authen-

ticated on the network will be prevented. However,

a possible attack could come from someone who has

illegitimately accessed and authenticated on the net-

work. To prevent this and also to prevent possible

attacks on the channel between the users router and

the DI an additional security layer is inttorduced (see

Fig. 6) by adding a Transport Layer Security (TLS–

http://www.ietf.org/rfc/rfc2246.txt) to the communi-

cation loop.

Figure 6: TLS and Wiﬁ encryption.

TLS also allows the use of a certiﬁcate to con-

ﬁrm that DI is really DI (and not an attacker supplant-

ing the DI). Even encrypting the communication an

attacker could just resend some previously recorded

frames in order to send orders to the appliance.

4 AN ENABLING MOCKUP

We have set up a ﬁrst mockup in the Computer

Science Department labs of University of Milano.

From a purely operational perspective, it consists

of two - three white goods wiﬁ connected to Inter-

net, each with its own Arduino board, in order to

execute recipes and send back status signals. For

the moment only a washing machine (wm) is be-

ing tested (see Fig. 7). We use an Arduino MEGA

ADK board, an Arduino WIFI SHIELD (see Fig. 8)

and a Sitecom wiﬁ router connected to the univer-

sity network. On this hardware we implement a com-

munication protocol solving, albeit in a preliminary

way, problems both of security and of exact message

addressing from middleware to appliance and back,

within a MQTT-like service (MQTTstandard, ) that

is sketched in a next section. The protocol is event

Figure 7: A early SandS mockup.

Figure 8: The Arduino interface.

driven, so messages are sent only if state variables

change value. The appliance detection is realized in

plug&play mode. The management of the appliance,

in terms of speciﬁcation and recipe requests, is done

via a browser on any mobile or ﬁxed device. The

appliance is located on the ground ﬂoor of the Com-

puter Science Department, while the consolle is on the

third ﬂoor of the same building. A distance of around

300 m is covered by ethernet wiring, while the last 10

m (including two concrete walls) are gapped via wiﬁ

connection. A sketch of the parameters managed by

the browser is reported in ﬁg. 9. In the following we

propose early considerations on the various parts of

the mockup.

4.1 Computational Requirements to the

Interfacing Board

Quite simply, we ask to the Arduino board to act as

a transducer: in input an instruction to the wm actu-

ators, in output the corresponding signal to be trans-

ADomoticEcosystemDrivenbyaNetworkedIntelligence

363

Figure 9: A screenshot of the washing machine monitor.

mitted to the wm microcontroller. From this perspec-

tive, the signals overwrite the native microcontroller

ﬁrmware with a set of commands that change over

time. Thus the microcontroller receives one command

at a time consisting of the actuator ID and the value

of the parameter to be ﬁxed, where the time is de-

cided in principle by the DI. On the one hand this sort

of unitary code (one parameter per actuator) requires

common tricks to manage multiparameter actuators.

On the other hand, we distinguish two time scales,

i.e. a time granularity triggering the shift between

scales. The general philosophy is: a sequence of in-

structions, each lasting more than the time granularity,

is dispatched at the proper time by the DI. Vice-versa,

sequences of instructions lasting less than the time

granularity are encoded into a single instruction that

is interpreted and sequenced directly by the Arduino

board. This requires a ﬁnite automaton to reside in

the board, both to carry out the above sequencing and

to manage a series of security checks to avoid plant

damages and logical inconsistencies. As a matter of

fact, even one shot instructions such as “heat the wa-

ter” requires a set of controls – e.g. the water level

to prevent the system from burning out in case of an

empty drum – which are managed locally by Arduino.

However, as to granularity, this instruction falls in the

ﬁrst category. One example of the second category

is the alternating running of the wm motor during the

pre-wash phase.

This is the basic situation with regard to normal

running. In addition, the board is called upon to su-

pervise an anomalous running at two levels – warning

and alarm – to which different standby or shutdown

sequences may follow as a consequence of the recog-

nized drawback.

Thus the interfacing board is required for compu-

tational power to store the instructions to be decoded

locally and to manage local time during the imple-

mentation of these instructions. For these purposes,

the Arduino capacities (RAM 256 KB, processor AT-

MEGA2570) prove to be sufﬁcient.

4.2 Communication Requirements to

the Interfacing Board

The communication bandwith necessary for the

mockup is normally very low. It increases relatively

when the appliance is woken-up by a recipe execution

request. The communication is stroked by the keep

alive message sent by Arduino to the DI. Thus, every

5 seconds, the board waits for instructions by the DI.

In the current debugging phase the board sends the ap-

pliance status (water temperature and level, spin rpm)

as a more informative payload. In a greater detail, the

message from board to DI is structured as follows, as

a further simpliﬁcation of MQTT scheme:

[CMD,EVN,VAN,VALUE,CHK]

[ = start of message

CMD = command.

EVN = (progressive) Event number.

VAN = Var number (indexing the boling up variables).

VALUE = Var value (type:Byte, Int, Long o String).

CHK = Checksum23

] = end of message

Conversely, the DI sends instructions to the board.

We index each instruction with a sequential number

within a recipe ID. This indexing my prove suitable

for both debugging and appliance quality control pur-

poses. The message format is analogous, with internal

event number mating with the one of the sent event for

a correct reckoning of the queues.

[CMD,EVN,VAN,VALUE,CHK]

[ = start of message

CMD = command.

EVN = Event number (the same of the message which

is the answer to).

VAN = Var number.

VALUE = Var value.

CHK = Checksum

] = end of message

Once the connection is stated between DI and

board, it proceeds in full duplex mode. The connec-

tion is open and maintained by the appliance through

the keep alive messages. In Fig. 10 we can see a seg-

ment of the conversation between appliance and DI,

as collected by the Arduino debugger.

The time granularity of the recipe transmission

takes into account various idle times. Besides normal

messages related to the recipe execution, both com-

munication addressees may send overriding messages

concerning alarm status and various kinds of shut-

down/standby commands.

WEBIST2014-InternationalConferenceonWebInformationSystemsandTechnologies

364

Figure 10: A screenshot of the Arduino debugger.

4.3 Degrees of Freedom in Overwriting

the Microcontroller Software

In principle the Arduino Interface is an open-

hardware/open software device, so that any person

may install it on his/her white good without requir-

ing any license. However, its implementation requires

a set of weld junctions and software for overwriting

the appliance microcontroller that are speciﬁc to the

appliance in hand. A further step in the mockup im-

plementation will be to extend the software standard-

ization to a maximum. But overwriting the micro-

controller is not an easy task. Rather, we may expect

the emergence of small business third parties – for in-

stance, free lance professionals or white good stores

with modern selling strategies – which are specialized

in this task. For instance, Fig. 11 reports a typical cal-

ibration curve relating the water temperature level in

the drum and the electrical signal transmitted to the

board. Nevertheless, the obvious expectation is that

appliances’ manufacturers will pursue their own busi-

ness interests and support this new paradigm of ap-

pliance usage by embedding their products with the

necessary electronics.

In this early implementation, we played the role of

networked intelligence substitute by featuring a few

recipes by ourselves. This gave us the opportunity

to appreciate the degrees of freedom of this operation

and the optimality criteria we may pursue as a coun-

terpart. Namely we jointly considered the following

goals: 1) washing efﬁcacy, 2) energy consumption, 3)

water consumption and 4) environmental pollution.

5 CONCLUSIONS

While the project is still at an initial stage, we have

come to see the high value it offers in terms both

of user convenience and of environmental advantage.

For instance, questions re the beneﬁt and ecological

Figure 11: A calibration curve.

proﬁtability of using a bio soap product or reducing

the amount of wash water become theoretical opin-

ions no longer. They may be experimented by the

single user and many similar users as well within the

eahouker social network, so that these questions may

ﬁnd a concretely scientiﬁc answer.

Though the concrete operations necessary for re-

alizing the mockup delineate a scenario where each

single user may implement her/his SandS terminal on

home appliances with some technical help, the main

way to implement our paradigm passes through a both

moral and economical suasion to convince the manu-

facturers that social appliances are more efﬁcient and

rewarding. To achieve this goal SandS project will re-

alize the above discussed architecture in order to set

up an initial social network of eahoukers where the

beneﬁts of the paradigm will be tossed in concrete by

a thousand members. These beneﬁts and the members

enjoying them will be the authentic promoters of the

SandS ecosystem, again all on the spirit of exploiting

actual facts besides sentences.

REFERENCES

Apolloni, B., Fiasch, M., Galliani, G., Zizzo, C., Caridakis,

G., Siolas, G., Kollias, S. D., Romay, M. G., Barri-

ento, F., and Jose, S. S. (2013). Social things - the

sands instantiation. In WOWMOM, pages 1–6. IEEE.

Grana, M., Apolloni, B., Fiasche, M., Galliani, G., Zizzo,

C., Caridakis, G., and et al. (2013). Social and

smart: towards an instance of subconscious social in-

telligence. In 1st Workshop on Innovative European

Policies and Applied Measures for Developing Smart

Cities, 14th EANN.

Gra

na, M., Nu

nez-Gonzalez, J. D., and Apolloni, B. (2013).

A discussion on trust requirements for a social net-

work of eahoukers. In HAIS, pages 540–547.

Haykin, S. (1994). Neural Networks: A Comprehensive

ADomoticEcosystemDrivenbyaNetworkedIntelligence

365

Foundation. Prentice Hall PTR, Upper Saddle River,

NJ, USA, 1st edition.

Ku, C.-C. and Lee, K. Y. (1995). Diagonal recurrent neural

networks for dynamic systems control. IEEE Trans.

Neural Netw. Learning Syst., 6(1):144–156.

MQTTstandard. http://mqtt.org/.

Rosenblatt, F. (1958). The perceptron: A probabilistic

model for information storage and organization in the

brain. Psychological Review, 65(6):386–408.

Sutton, R. S. and Barto, A. G. (1998). Introduction to Re-

inforcement Learning. MIT Press, Cambridge, MA,

USA, 1st edition.

Wiener, N. (1948). Cybernetics, Or Control and Commu-

nication in the Animal and the Machine. Wiley, New

York.

WEBIST2014-InternationalConferenceonWebInformationSystemsandTechnologies

366