NORMATIVE APPROACH FOR SOCIO-PHYSICAL COMPUTING

An Application to Distributed Tangible Interaction

Fabien Badeig

, Catherine Garbay

, Valentin Valls

and Jean Caelen

LIG/AMA, Universit

e de Grenoble, UFR IM2AG, BP 53, F-38041 Grenoble Cedex 9, France

LIG/MultiCom, Universit

e de Grenoble, UFR IM2AG, BP 53, F-38041 Grenoble Cedex 9, France

Keywords:

Ambient intelligence, Tangible distributed interfaces, Activity theory, Norms, Agent-based systems.

Abstract:

We present a normative multi-agent design for computer-supported collaboration in the framework of socio-

physical computing. An example application (RISK game) in the context of the TangiSense platform supports

the proposed approach. Our work is driven under four complementary views: a systemic view, according to

which various designing levels, from the physical infrastructure to the social level of human coordination are

integrated in a single modelling, a normative view, in which consistency and coordination of action is ensured

with respect to individual as well as collective systems of norms, a trace-based view, in which traces reﬂecting

human activity and its compliance to the norms are registered and an agent-oriented view, according to which

agents are meant to process, interpret and communicate information across distant tables.

1 INTRODUCTION

This research is conducted in the framework of a

project (Lepreux et al., 2011), whose objective is the

management of distant interactive surfaces support-

ing tangible and virtual objects. The TangiSense table

(Figure 1) may be seen as a magnetic retina, which

is able to detect and locate tangible objects equipped

with RFID tags. RFID tag events are transmitted to

the host PC and processed by the infrastructure layer.

The ﬁrst role of this layer is to ﬁlter potentially unsta-

ble tags IDs and positions. Its second role is to pro-

ceed to the aggregation of tag events and to maintain

consistent representations of tangible objects. Each

RFID antenna is further equipped with 4 multicolor

light emitting diodes (LEDs). When lit, they may

be considered as virtual objects displayed on the ta-

ble. The role of these diodes is to “react” to tangible

objects positioning and moves, assessing for the user

their effective detection by the table. Human activ-

ity involves the handling of tangible objects. Com-

munication between distant tables is managed via vir-

tual objects displaying the status of the original tangi-

ble objects. Our goal is that human collaboration be

mediated rather than assisted by computerized tools.

Our guiding principle is to preserve the spontaneity of

human action by designing ecological working envi-

ronments (Thomas and Kellogg, 1989). Our work is

driven under: (i) a systemic view, integrating in a sin-

gle homogeneous modelling the physical infrastruc-

ture level as well as the higher level of human coor-

dination; (ii) a normative view, in which various sys-

tems of norms are introduced to mediate human activ-

ity; (iii) a trace-based view, to record and transcribe

human activity together with its compliance to this

set of norms; and (iv) an agent-oriented view, accord-

ing to which agents are meant to process, interpret

and communicate information. Activity traces, which

are generated by the handling of tangible objects, are

made to evolve according to agent-based processing,

but also under the systems of norms at hand. In ac-

cordance with the principles of activity theory, norms

do not act as a prerequisite, or as a way to apply a

priori constraints on action. Rather, they are meant

to “situate” action, by modifying trace properties that

will in turn regulate agent activity, and for example in-

ﬂuence information or communication policies. The

Risk game is used as an example application.

2 STATE OF THE ART

When designing collaborative support systems, a ma-

jor issue is to preserve the spontaneity and ﬂuid-

ity of human activity while ensuring the consistency

and proper coordination of action (Pape and Graham,

2010). The COIN (Coordination, Organization, In-

stitutions and Norms in Agent Systems) community

309

Badeig F., Garbay C., Valls V. and Caelen J..

NORMATIVE APPROACH FOR SOCIO-PHYSICAL COMPUTING - An Application to Distributed Tangible Interaction.

DOI: 10.5220/0003717303090312

In Proceedings of the 4th International Conference on Agents and Artiﬁcial Intelligence (ICAART-2012), pages 309-312

ISBN: 978-989-8425-96-6

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

(http://www.pcs.usp.br/∼coin/) introduces the notion

of norm in a complex agent organization as a way to

cope with the conceptual antagonism between auton-

omy and control; it further allows approaching coor-

dination as a social paradigm. Behaviour in such or-

ganizations is not only guided by the agents mere ob-

jectives but also regulated by norms specifying which

actions are considered as “legal” or not by the group.

Norms are speciﬁed in a declarative way, they may be

adopted or not by the agents, and adapted to cope with

the evolution of context (Boella and Torre, 2006). In

multi-agent systems, a multitude of agents interact,

with some intended individual or collective purposes.

Such a view usually assumes structures that articulate

or restrain interactions in order to make them more

effective in reaching those goals, trustworthy for par-

ticipants or more predictable. One major issue is then

to cope with the various interaction modes of agents

belonging to the same organization, according to its

structural and functional speciﬁcations: the organi-

zational model of MOISE has been extended to this

end (Boissier et al., 2011). The goal of such speci-

ﬁcation is to allow the organization to check that in-

teraction modes are used appropriately and to allow

the agents to reason on these interaction modes like

they do with norms. A second major issue is how

to maintain consistency, especially in contexts where

human actors do not know each other, are communi-

cating from distant places, and may display opposite

or conﬂicting goals. Activity theory articulates within

Figure 1: The TangiSense table: in this simulation, each of

the four spaces is controlled by a separate PC.

a single dynamics the organizational and functional

dimensions of human activity, with a further distinc-

tion between the notions of subject, object and tool.

According to this theory, the tool supports and lim-

its activity, it mediates its structure and objective, and

carries the history of the relationship between the sub-

ject and the object (Bourguin et al., 2001). The ob-

ject is seen and manipulated not “as such” but within

the limitations set by the instrument. In turn, the tool

is transformed and built along the activity and there-

fore keeps track of the user experience. Individual

and group activity co-evolve in a context-dependent

way, they are driven according to certain rules, norms

and conventions; they depend on the actors roles and

resources, as well as their organization. Dynamic-

ity is core to activity theory, and any component, be

it a tool, a goal, or a norm, are constantly changed,

constructed, and transformed in relation to the activ-

ity outcome (Greenberg, 2001). As advocated by the

proponents of situated action (Nardi, 1996), the ob-

ject and motive of action reveal themselves only in

the process of doing, since the involvement in action

creates circumstances that could not be anticipated in

advance.

A normative view is proposed to account for the

speciﬁcities of socio-physical computing, that is to

preserve the spontaneity and ﬂuidity of human activ-

ity while ensuring the consistency and proper coor-

dination of action. Human activity in the context of

the proposed design involves the handling of physi-

cal, tangible objects that is performed under speciﬁc

application-dependent rules. The distributed frame-

work in which interaction takes place furthermore im-

plies that human activity be registered and displayed

for the distant human actor, in a way that is “situated”

with respect to the interactive surfaces at hand. Vir-

tual representatives of tangible objects are displayed

to this end on distant tables. Tangible objects ded-

icated to coordination may further be introduced to

ensure robust inter-table coordination. Activity traces

have been proposed by several researchers as a way

to represent, share and visualize human experience

in the course of its interaction with numerical plat-

forms (Djouad et al., 2010). Interaction traces have

further been explored to enhance synchronous collab-

oration, and sharing traces at a group level has been

advocated to support group awareness (Clauzel et al.,

2011). In the same line, we propose to mediate hu-

man activity by tangible objects and to track this ac-

tivity thanks to traces in the numerical environment.

Beyond human activity, traces are further meant to re-

ﬂect the activity of any computational entity in our

system, be it event from the table, pattern of move

from the infrastruture layer, semantic interpretation

or compliance to a norm from the software layer.

Any trace property may be subject to norm-dependent

analysis. In line with our trace-oriented design, this

analysis will result in signs deposited in the trace, en-

riching its content, and “situating” the activity with

respect to the system of norms at hand. Compliance

to a norm is processed in a way that is deeply depen-

dent on the application domain and social organiza-

tion at hand. The case of socio-physical computing

suggests that norms be “carried” by tangible objects

and express regulations concerning e.g. their patterns

of move. Compliance to such norm would result in

a feedback to the given tangible object (e.g. enlight-

enment of LEDs beneath its position), and therefore

ICAART 2012 - International Conference on Agents and Artificial Intelligence

310

Infrastructure+

layer+

So0ware+

layer+

JADE++

Norms+

Agents+Traces+

JADE++

Norms+

Agents+Traces+

Individual+

level+

Group+level+

Virtual+

Tangible+

Figure 2: Functional view of the distributed architecture.

to the human actor responsible for the move. Distant

collaboration across distributed interactive tables fur-

ther suggests that any tangible object move be trans-

mitted (provided it is norm compliant), for example

by means of virtual displays on the distant surfaces.

Privacy rules would on the contrary prohibit such dis-

play, for some objects, at some places, or at certain

critical times of the collaborative work. Compliance

to these norms therefore results in feedback transmit-

ted or not to distant actors. Both local and distant

feedbacks are performed by dedicated agents exploit-

ing the tangible object traces together with their norm

compliance properties.

3 PROPOSED ARCHITECTURE

We propose a layered architecture comprising two

main separate layers, from the infrastructure layer

to the software layer (Figure 2). The agents are

designed under JADE (http://jade.tilab.com/). The

software layer can be seen as integrating entities

(ﬁlters, traces and agents) transparently operating

over and communicating with any local or dis-

tant agent. Agents are activated via ﬁlters they

have deposited in the normative space as norm

instances. They operate upon traces under the

mediation of these ﬁlters whose dynamics depend

upon the agent social position, its compliance to

the norms and its individual activity. Any trace

is considered as a set of tuples (property, value).

We propose that each property be typed, to register

its compliance to the norms. A trace is therefore

expressed as trace = h(p,v)i with p = name :

consistency ∈ {new,modi f ied,valid,invalid} :

privacy ∈ {private, public}. new and modi f ied

mean that the corresponding property has been newly

created or modiﬁed, which implies that compliance

checking has to take place; valid and invalid express

compliance to some application-dependent norm.

private means that the property is not accessible

to other distant agents. Otherwise, the property is

readable. Filters are in the form (conditions,actions).

The condition part speciﬁes contextual patterns over

trace properties. The action part may involve agent

activation as well as trace modiﬁcation. Three sys-

tems of norms are considered ruling privacy, consis-

tency and coordination. Whereas the privacy pol-

icy deﬁnes the information shared between each dis-

tant player, the consistency policy checks information

with respect to application rules. The coordination

policy is a high-level policy regulating the activation

of agents at a local or global scale. To reach this level

of coordination, the trace properties of interest must

be consistent and either private (local coordination) or

public (global coordination). Privacy (resp. consis-

tency) ﬁlters modify the privacy (resp. consistency)

attribute of trace properties, specifying whether infor-

mation is to be kept local or allowed to reach global

level (resp. is valid or not). The action of ﬁlters is

restricted to trace property modiﬁcation. Contrarily,

coordination ﬁlters do not operate any trace property

modiﬁcation. Rather, their role is to activate agents

on a local or global scale. For this purpose, their con-

dition parts involve the evaluation of both the consis-

tency and privacy attributes of trace properties.

The Risk game is a turn-based strategy game for

two to six players. The standard version is played on

a board depicting a political map of the Earth, divided

into forty-two territories. The player objective is to

occupy a growing number of territories on the board,

progressively eliminating other players, with results

determined by dice rolls. A limited-scope scenario

(identiﬁcation of a new player) is used to illustrate the

three proposed normative policies (Figure 3). We dis-

tinguish between 3 agents types : Gameplay Agent

(GA) whose role is to manage the gameplay in its var-

ious phases, Interface Agent (IA) whose role is to dis-

play visual information on the interactive surfaces and

Player Agent (PA) whose role is to maintain informa-

tion about the player. When a player drops his iden-

tiﬁcation card on the table, a low-level event is trig-

gered which generates a tangible object trace t

player

in the local information space. At this time, all trace

properties are initialized with extension types new and

private. Since a new tangible player card has been de-

tected, a veriﬁcation process is triggered to check its

compliance with the rules of the game. Two valida-

tion ﬁlters ( f

valid

, f

invalid

) manage this process, under

the example rule that 6 players at most may join the

game ; the trace t

game

keeps information about the cur-

rent game. If successful, there is an update of t

player

properties from new to valid. Thus, the ﬁlters f

valid

and f

invalid

ensure that the local sub-system is consis-

tent, by changing the value of the attribute consistency

for the trace properties. In our scenario, all tangible

objects are equipped with tangible agents. For the

player card, a player agent (PA) has to be initialized

NORMATIVE APPROACH FOR SOCIO-PHYSICAL COMPUTING - An Application to Distributed Tangible Interaction

311

(name : valid : private, gamer1),

(position : valid : private, (2, 3)),

(table : valid : private, {1})

(tag : valid : private, 1),

(type : valid : private, (tot, player))}

{(id : valid : public, 1),

game

player

valid

invalid

succeed

fail

player

playerShare

(name : new : private, gamer1),

(position : new : private, (2, 3)),

(table : new : private, {1})

(tag : new : private, 1),

(type : new : private, (tot, player))}

(state : valid : private, init)}

{(surname : valid : private, Dupont),

{(surname : new : private, Dupont), {(surname : valid : private, Dupont), (name : valid : private, gamer1),

(position : valid : private, (2, 3)),

(table : valid : private, {1})

(tag : valid : private, 1),

(type : valid : private, (tot, player))

{(surname : invalid : private, Dupont),

(name : invalid : private, gamer1),

(position : invalid : private, (2, 3)),

(table : invalid : private, {1})

(tag : invalid : private, 1),

(type : invalid : private, (tot, player))}

game

{(id : valid : public, 1),

(number

player

: valid : public, 4)

(position : valid : private, (2, 3)),

(table : valid : public, {1})

(tag : valid : private, 1),

(type : valid : public, (tot, player))

(state : valid : public, init)}

{(surname : valid : private, Dupont),

(name : valid : public, gamer1),

(number

player

: valid : public, 3)

(type : valid : public, (cct, game))}

Figure 3: Evolution example of traces resulting from the ﬂexible interleaving between agent processing and ﬁlter triggering.

if the trace is validated. After this validation process

and the modiﬁcation of the trace t

player

, two ﬁlters are

launched, f

succeed

and f

f ail

, whose role is to ensure lo-

cal coordination, including display information. GA

and IA agents are launched in case of success, with

adequate parameters. IA provides a visual feedback to

the player via the LED underneath the tangible object.

The role of GA is to create a player agent PA associ-

ated to the player identiﬁer card and to enrich the trace

player

by adding the property state with the value init.

The game trace t

game

is further updated to keep track

of the new number of players. In parallel to the con-

sistency process, an information sharing process oc-

curs. The ﬁlter f

player share

deﬁnes the privacy policy.

Some parts of t

player

have now to be shared, due to the

basic rule that any new player must remain visible to

other players. The condition to share information is

that the trace is valid and the agent state is init. When

the ﬁlter is triggered, the associated action is the mod-

iﬁcation of t

player

privacy properties. Following the

speciﬁc rules of that game, distant IA agents are then

launched by the ﬁlter f

player visual

in order that some

virtual representative of the new player be displayed

on distant interactive surfaces. New traces are gen-

erated each time a new tangible object is placed on

the interactive surface. These traces evolve through

the action of agents and ﬁlters, which result in new

contextual pattern modifying the course of process-

ing. Regulation of local as well as distant activities is

transparently and smoothly integrated in this process.

4 CONCLUSIONS

We have proposed an original approach for normative

systems in the framework of socio-physical comput-

ing. In this design, norms are meant to mediate human

activity by providing signs of its compliance to the

edicted rules. Norm compliance information may be

processed and interpreted, possibly in different con-

texts, by different actors, providing a gain in ﬂexibil-

ity and modularity. Human action in such design is

mediated rather than constrained by norms which act

in a productive, constructive way, by providing signs

of their relationship to action, in agreement with ac-

tivity theory.

Work performed under grant IMAGIT - ANR

2010 CORD 01701.

REFERENCES

Boella, G. and Torre, L. (2006). Introduction to normative

multiagent systems. In Computational and Mathemat-

ical Organization Theory, volume 12, pages 71–79.

Boissier, O., Balbo, F., and Badeig, F. (2011). Controlling

multi-party interaction within normative multi-agent

organisations. In COIN in Agent Systems VI (LNAI

6541). Springer-Verlag.

Bourguin, G., Derycke, A., and Tarby, J. (2001). Beyond

the interface: co-evolution inside interactive systems

a proposal founded on activity theory. In IHM-HCI

2001 conference, pages 297–310. Springer Verlag.

Clauzel, D., Sehaba, K., and Pri

e, Y. (2011). Enhancing

synchronous collaboration by using interactive visu-

alisation of modelled traces. In Simulation Modelling

Practice and Theory, volume 19, pages 84–97.

Djouad, T., Mille, A., Reffay, C., and Benmohammed, M.

(2010). A new approach based on modelled traces

to compute collaborative and individual indicators hu-

man interaction. In ICALT’2010, pages 53–54.

Greenberg, S. (2001). Context as a dynamic construct.

Human-Computer Interaction, 16:257–268.

Lepreux, S., Kubicki, S., Kolski, C., and Caelen, J. (2011).

Distributed interactive surfaces using tangible and vir-

tual objects. In Workshop DUI at CHI, pages 65–68.

Nardi, B. (1996). Activity theory and human-computer in-

teraction. In Context and consciousness: activity the-

ory and human-computer interaction, pages 69–103.

MIT Press.

Pape, J. and Graham, T. (2010). Coordination policies for

tabletop gaming. In Graphics Interface, pages 24–25.

Thomas, J. and Kellogg, W. (1989). Minimizing ecological

gaps in user interface design. In IEEE Software, pages

78–86.

ICAART 2012 - International Conference on Agents and Artificial Intelligence

312