E-RETAIL: INTERACTION OF INTELLIGENT SELLING SPACE

WITH PERSONAL SELLING ASSISTANT

Alain Derycke, Thomas Vantroys, Benjamin Barby and Philippe Laporte

Laboratory LIFL, University of Sciences and Technologies of Lille, Villeneuve d’Ascq, France

Keywords: Pervasive-commerce, Context-Aware environments, Ubiquitous Computing, augmented reality, personal

assistant.

Abstract: With the availability of nomadic computing, and its new interaction user devices connected through wireless

networks, it is obvious that the traditional way of delivering commerce will evolve towards “pervasive-

commerce”. This paper presents our approach based on Intelligent Selling Space to augment interaction in

store department, and with the seller equipped with a personal assistant. For that purpose, we defined

interaction patterns and a generic infrastructure based on OSGi and UPnP. Our approach is currently

evaluated in a hypermarket.

1 INTRODUCTION

With the rapid dissemination and uses of mobile

wireless devices, there is an opportunity to enlarge,

or to modify, the scope and the nature of the

traditional retail industries, whatever the format of

the selling places. This means that mobile personal

devices can be useful in both situations: physical

place in the retail locations, and virtual place for the

e-commerce. We focus our attention on the possible

augmentation of the interactivity and intelligence --

for example by recognizing the client and

personalizing the contents and elements of the user

interface -- of the traditional apparatus used for the

retail place: shelves, carts, store windows, apparels,

etc.

It appears, both in the research field, and in the

practical uses in the retail industries, that new

systems (setting of different electronics components

embedded in furniture) have been designed and

experimented, using the potential of interaction with

different format of displays, and different modes of

interaction such as tactile one or user movement

detection by video camera… This will be

summarized into the section 2 with some views of

the future of the commerce as emergence of the

“pervasive-commerce” in the same way of evolution

of the Information and Communication

Technologies toward pervasive computing or

communication (Satyanaryanan, 2001).

Our attempt is to develop a more generic

approach of these systems that will be called

“Intelligent Selling Spaces”, or ISS, defined and

illustrated in the section 3.

We put attention of the extensibility of these

systems and their openness to interactions with

mobile devices carried by both the client and the

seller. Persons who are situated into the immediate

vicinity of the ISS are considered. The user’s mobile

devices, PDA or Smartphone, are loaded with some

software applications or widgets that specialize them

as Personal Selling (or Shopping) Assistant, a PSA,

and will be presented into the section 4.

We have selected two mains uses of the potential

interaction of these PSA with the proximate ISS:

First the PSA is used by the seller, who is in her/his

retail department, as a remote control device to

manage, update, and configure the ISS; Second the

ISS is derived to augment the capacity of the PSA in

order to provided, to the sellers and customers, more

interaction capacities such as large displaying of the

information… For that purpose we have identified

several patterns of interactions, developed several

typical scenarios of used in conjunction with some

partners of the retail-industries. This has led us to

design and prototype the PSA as a dynamic

extensible personal user device (see section 5). This

is the main focus of our paper.

189

Derycke A., Vantroys T., Barby B. and Laporte P. (2008).

E-RETAIL: INTERACTION OF INTELLIGENT SELLING SPACE WITH PERSONAL SELLING ASSISTANT.

In Proceedings of the Tenth International Conference on Enterprise Information Systems - HCI, pages 189-194

DOI: 10.5220/0001687801890194

 SciTePress

2 WHAT IS E-RETAIL?

2.1 Evolution of the Information and

Communication Technologies for

the Commerce

At the first look it seems that the traditional retail

industries are not heavily impacted by the ICTs in

their relationships with the customers. But in fact the

attentive observation of the different retail places

shown us that it is not the whole picture, and there is

more and more presence of the ICTs in direct

interaction with the customer, with a large diffusion

of electronics displays of different formats. In fact

the diffusion of the ICTs into the future retail places

is going more and more pervasive…

Our analysis is that, first the development of all

these systems is done mostly in ad hoc approaches,

and second that this evolution must be coordinated

with the e-commerce mode, which the same large

companies often rapidly develop at the same time.

The figure 1 gives an overview of the potential

intersection between three major trends: e-retail,

mobile communications and e-commerce. Our focus

is on the part we have called Intelligent Selling

Spaces, ISS, which will be located into a shop or a

hypermarket and provided a dedicated functions in

closed relation with the type of goods or products it

helps to commercialise.

Delimitation of the scope of an ISS

Ubiquitous Computing

Personal, Portable, Pedestrian

Heterogeneous, Identity,

Intimacy

Commerce

Web

based

At the cross of

three worlds:

The Pervasive

commerce ???

Our object of

study

ISS

: located place,

setting, immobile,

shared

The distinction is done in relation with

Places

where the purchase

are done, and on the

control

de direct environment

Figure 1: intersection of three domains and our object of

studies.

2.2 Examples of Intelligent Selling

Places

In the research field there are already several

projects which have explored the design and uses of

new computerised systems for e-retail. Very

frequently they investigated also the potential of

Radio Frequency IDentification tags attached to

individual product (Konomi, Roussos, 2007) in

order to read information rapidly and to identify

exactly the product, in a more easy interaction that

with the traditional barcode. From the scientific

literature we can see proposals about intelligent

shelves (Wasinger, Wahlster, 2006) smart carts

(Kouroithanassis, Roussos, 2006), intelligent

advertising displays, etc.

In some case a retail enterprise want to provide a

more integrated show-case of these new

technologies, both to learn how it is used on a larger

basis, and also to give a good image of them. From

our personal investigations in some of these new

places, it seems that it is mostly technology push,

and that, at this stage, there is no real or rich

interaction with the mobile devices carried by the

customers.

3 SOFTWARE ARCHITECTURE

OF AN ISS: OVERVIEW

From the previous exploratory prototypes and study

of the similar systems, we have derived a generic

software architecture organized around a dedicated

middleware to support Human-Computer Interaction

in different modalities, including speech recognition

and speech synthesis. It reuses our previous work

about multi-channel and multimodal intermediations

between a mobile personal user device and a

collection of e-services (Chevrin et al, 2006).

However the case of an ISS is simpler here because

the number of services provided is small (specialized

local functionalities), and the numbers of specific

devices used in the interaction with the customer is,

for a particular ISS, relatively small. We can

consider that the ISS is relatively autonomous,

required few connections with the information

system of the shop, and can be easily described in a

small ontology as a micro-world. So dynamic

discovering of its services will be easier than in most

of the Ubiquitous Computing projects, e.g. Smart

Homes.

Our generic software architecture must take into

account the modularity of the ISS (adjunction or

suppression of some interactive elements, needs for

adaptation to a particular retail company) and its

openness to the PSA described into the next section.

The figure 2 gives an overview of the software

architecture.

An ISS is built by assembling of services UPnP

(Universal Plug and Play) into an OSGi (Open

Services Gateway initiatives) (OSGi, 2007)

gateway:

UPnP is a wired IP protocol allowing the

creation of spontaneous networks of devices (TV

sets, HVAC, light control …) and control points

ICEIS 2008 - International Conference on Enterprise Information Systems

190

(PDA, Smartphone, touch panels …). UPnP enables

the live detection of devices and the use of their

services by the control points.

OSGi framework allows to deploy and redeploy

Java-based plug-in applications (bundles) offering

services (Donsez, 2007).

Figure 2: an overview of an ISS software architecture.

4 PERSONNAL SELLING (OR

SHOPPING) ASSISTANT: PSA

4.1 The Functions of a PSA

Our focus is mainly on the support of the sellers in

the context of an e-retail environment during all the

phase of a buying activity by the customer: before-

sale (research and selection of products or services

by the customer), helps during the experience of the

selected products (for example fitting a trouser), and

helps during the customer decision, payment phase,

after-sale…

It means that there are good parallels between

the Personal Selling Assistant owned by the sellers,

and a Personal Shopping Assistant which is carried

by the customer inside the e-retail places. The

software for the second one can reuse several

functions provided to the first one. But there are also

differences between the two kinds of PSA because:

- The customer PSA needs more context-aware

adaptation of the interaction;

- The seller PSA hardware and basic OS

platform are well known at the design time,

stable and standardized for a known network of

shops;

- The customer PSA is more heterogeneous and

known only at run-time. This joins all the work

and proposals about context-aware computing

and dynamic adaptation of the user interface to

the specificity of the particular mobile devices:

plasticity of the interface (Chevrin et al, 2006).

The Personal Selling Assistant is often extended

with some hardware and software elements in order

to read optically barcode, or RFID Tags. Several

dedicated functions are added for the management of

the store (or of the seller department or section) as

example requiring stocks for a particular product,

prices and margins…

4.2 Advanced Personal Selling

Assistant as a Coach or Instructor

In order to improve the efficiency of the sellers and

to help them in their jobs, the PSA can provide more

advanced functions. There are specific needs for

knowledge accesses and exchanges, about the

products or services sold or about the better selling

process, and also needs for contribution to the

knowledge base enrichment in the framework of

community of practices. The use of a synthetic vocal

output for the PSA, through a headphone, could

even give the possibility to provide personalised

coaching to the seller, in real time in front of the

customer. It is also the same for the potential of a

multimodal interface using voice in input, for

example with the VoiceXML standard, which we

have developed in the framework of e-commerce or

design of digital coaching.

5 INTERACTION OF ISS AND

PSA

5.1 Patterns of Interactions and

Scenarios of Use

From our previous case-studies and from the state of

the art in the e-retail domain, we have identified

several patterns of interaction between an ISS and

several PSA, which correspond to a large variety of

use-scenarios. These patterns are enumerated here

from the ISS view:

- Direct interaction with a customer without a

personal mobile device;

- Collaborative interaction with the customer

equipped with the personal mobile device able

to support Bluetooth of Wifi local connections;

- Direct interaction with the seller without

mobile device (setting request for information);

- Collaborative interaction of the seller,

equipped with his/her PSA, potentially the ISS

is seen as an extension of the PSA or is

E-RETAIL: INTERACTION OF INTELLIGENT SELLING SPACE WITH PERSONAL SELLING ASSISTANT

191

supervised by PSA, used as a remote control

device;

- Collaboration of the seller and customer

collocated near an ISS. The different

presentation devices of the ISS can support a

shared focus, giving more than 5 different

patterns depending of who owns a PSA and of

the situation about data sharing and

collaboration.

The problem about the nature and security of the

wireless networks must be also addressed seriously,

especially if the ISS are located into a large

hypermarket. The main idea is to keep at the

minimum the needs of connection and bandwidth

with the global information system, used just for

accessing some back-office information, and that

ISS are relatively autonomous. This is

straightforward with the core principles of pervasive

computing. For the links with the global information

systems, secured Wifi networks are used. Same, the

seller’s PSA can access directly the global

information system, in any places of the store, by

Wifi, and his/her PSA is protected, for example, by a

biometric access control.

The PSA is personal and carried some

information about the identity of the seller and

profiles information. This means that we can

consider that, for the seller, his/her PSA can

simultaneously communicate with the global

information system and the ISS. The PSA is then

seen as a potential gateway between the two

systems, supervised by the seller.

5.2 PSA as an Extensible User Device

5.2.1 The Extensions for Improved

Interaction with the PSA

The figure 3 gives an overview of the software and

networks architecture for the use of some ISS

capabilities in order to augment the potential of a

PSA for interaction. It must be noted that we include

in this schema the extensions which are provided by

the Personal Area Network such as the support of a

Bluetooth auricle or others devices.

The figure 3 gives the flow of information and

controls only for output extension of the PSA, both

visual and sound. One criterion to select a

destination of particular information, for example

elements of an XML document, depends of its

privacy or not. If it is only for the seller the

information (e.g. product margins) is displayed only

on seller PSA, or render by voice on his/her auricle.

At the opposite if the information is public (e.g.

characteristics of some products) and must be shared

by the seller and the customer(s), it is displayed on

the large display captured from the ISS, or render on

its the acoustical channel, and possibly duplicated on

the PSA depending of the nature of the information

(for example size an resolution of an image). The

next section will give more precision on this aspect

in the framework of a particular prototype.

Schema of

augmented PSA

(output only)

Infrastructure

for

Interaction

streams HTTP,

documents

XML on Wifi

secured

Ag Fi s si on

Pr oxy

local

Personal Mobile Device =

regular PSA

Supervision

Discover.

Ag Pr ep

standard Web

browser

Di s pl ay

pr iv ate

Chan. sound

pri vate

Local Environnement =

Smartspace

Output resources

Display HR

public

TV + OS + Bluetooth

Sound

channel

public

auri cal headphone

Bluet ooth

Client Smartphone

Bluet ooth

Other device

in output

Bluetooth Connections

Web services

IS of Hypermarket

hoc

ISS

Figure 3: an overview of the flows of information for the

collaboration of the PSA with the ISS and other personal

mobile devices.

5.2.2 The PSA as a Context-Aware Mediator

Two principal mechanisms are used for the routing

or mediation, done mostly inside the PSA, of

information elements coded into XML messages: the

fission mechanism directed toward the different

outputs, and the fusion mechanism which combine

elements of information going from different input

devices. This later is required in order to support

true multimodal interactions with the users. Of

course the idea, in the context of multi-devices

interactions, to use fission (Han et al, 2000) and

fusion (Flippo et al, 2003), is not new. However we

improve the propositions in four ways:

First:

These two transformations can be context-aware,

taking into account profiles and choices of the PSA

owners. This will be possible through use of a

dedicated scripting language for small mobile device

such as proposed by (Korpikää et al, 2006), and at

the run-time some decisions can be still under the

control of the PSA user, for example about

adjunction of a new extension;

Second:

These routing functions and their supervision are

distributed between the mobile device (PDA or

Smartphone) which plays the major role, and the ISS

which offers a well known service interface. This

ICEIS 2008 - International Conference on Enterprise Information Systems

192

schema leaves more flexibility in the design of the

global information system accesses, because the

relationship is at a more abstract level, for example

with the tagging of XML elements as public or

private, and it is more scalable.

Third:

The discovering of new capabilities is done

dynamically; either in input or in output enables

extension of the PSA. This means that it allows the

change of the forms or even the mode

(multimodality) of the user interface and interaction,

during a transaction with the global information

system. It is done if it has sense for the present

interaction, and if the user accepts this extension.

For Fission is not too difficult if the “agent” in

charge of preparing the information document in the

global information is sufficiently intelligent to react

to this new composition, i.e to change at a fly the

document transformation policies done with XSL-T.

For the fusion, it is more difficult because the shift

from a mono-modal interaction to a multimodal one,

in the context of a continuing transaction, must keep

the coherence of the user dialogue. To do that, we

are working on the design of the interface of the

PSA and the underlying software layers in a Model-

Driven Engineering approach, where the models of

the human contextual tasks are used as run time for

verifying the opportunity of changes for extensions

at right moment.

Fourth:

In some cases it is possible to use, by the global

information system, the geolocalisation of the PSA

(of sellers or customers) because they interact

locally with a precise ISS, where its location, inside

the hypermarket or the commercial mall is well

known and stable. This allows the global

information system to push information to the

sellers, or even customers, that are in accordance

with this place.

5.2.3 Distribution of the Document

Processing

In the figure 3, there are three tiers which are

involved into the processing of information

documents: the global information system, the PSA

and the ISS. Our architecture is based on a clear

separation about the role of each tier:

- The global information system is composed

first of all the services which can be of interest

for the support of the sellers and shop

managers or for the operation of the ISSs. For

the mediation with the inner e-services which

provide abstract forms of documents in XML,

we reuse and adapt a specific middleware

developed for multichannel and multimodal e-

commerce (Chevrin, 2006) based on a Multi-

Agent System.

- The PSA act as a mediator between the global

information system and the ISS. It doesn’t

exclude some direct links from this one to the

global system. The choice of using the mobile

computer embedded in the PSA is justified first

from security reasons, because the PSA is

controlled by an employee of the shop, in

presence of the ISS.

- The ISS is the third tier. It is well known since

the design time: kind of ISS, its services

interfaces, and its composition. The dynamic

discovering of possible functions for the

extension of the PSA is less difficult than the

cases of a lot of projects about ubiquitous

computing because it a relatively small world

and stable during the commercial transactions.

5.3 A Case-study: used of an ISS as a

Single Display Groupware

5.3.1 Scenario for First Deployment

The scenario is as follow: At anytime, from

anywhere in the hypermarket, the seller is able to

reach with his/her PSA, the different learning

resources to enhance his/her technical and business

tools available in the personal management system.

The seller access the corporate information system

to manage the different products of his store

department (e.g. stock). During the selling activity,

the seller has a direct access to all the products

information. By this way, she/he can quickly and

accurately answer to the customer.

To enhance the understanding of the customer,

the pattern of collaboration can be used. The seller

will augment his/her PSA with the ISS. He/She will

gain access to a large LCD screen to expose some of

the information, like technical features. By this way

is it possible for the customer to compare more

easily different products or to understand more

deeply some technical aspects of the product.

5.3.2 Implementation

For our first prototype, the PSA is based on an Ultra

Mobile PC (UMPC). In our gateway OSGi, we plug

an UPnP proxy to control the LCD screen. In this

proxy are implemented services supervising the PSA

augmentation. A control point allows the seller to

access these services in the PSA. Figure 4 illustrates

that with the augmentation of the PSA.

E-RETAIL: INTERACTION OF INTELLIGENT SELLING SPACE WITH PERSONAL SELLING ASSISTANT

193

Another OSGi service has the responsibility to

control the data fission between the PSA and the

LCD screen. This service is based on a XML

description file which defines rights for each type of

user.

Figure 4: A PSA extended with a LCD screen.

5.3.3 Evaluation

The prototype is currently under test for five months

into a real hypermarket (digital photography

department). From a technical point of view, we

want to test the communication infrastructure and

among others the latency and the information flow.

The prototype is also evaluated for its usability

and social acceptance with respect to the work

situations.But the main point of the evaluation is to

study the collaboration patterns in order to improve

their efficiency. How the seller and the customer

interact through the system? Is the ISS really good to

enhance the service offered to the customer? Is the

selling activity more efficient?

To quantify the evaluation, we are looking at

non-technical aspect, like the number of information

or knowledge resources used (is the use of mobile

device directly in the working place allows to pass

more time on knowledge acquisition?) and the

different characteristics of the selling (more selling?

more or less time with the customer?)

6 CONCLUSIONS

In this article we present the concept of Intelligent

Selling Space to enhance collaboration between

seller and customer. For that, we repose on mobile

devices and more especially on a Personal Selling

Assistant which is used as a remote control of the

micro-world constituted by the ISS.

In order to validate our approach, we are conducting

an evaluation in a real hypermarket. In the

meantime, we are designing and developing new

services to enhance the ISS and to test more deeply

the different patterns of interactions.

ACKNOWLEDGEMENTS

This project is supported partially by the p-LearNet

project funded by the Agence Nationale de la

Recherche in France. It receives the support of the

PICOM “Pôle de Compétitivité des Industries du

Commerce” and from AUCHAN, an important

worldwide retail enterprise. A special thank to V.

Chevrin, J. Rouillard and J.C. Tarby, members of

our group for their valuable contributions in the field

of HCI.

REFERENCES

Chevrin, V., Sockeel, S., Derycke, A., 2006, An

Intermediation Middleware for supporting Ubiquitous

Interaction in Mobile Commerce. ICPS’06, IEEE

International Conference on Pervasive Services 2006.

June 26-29, 2006. Lyon, pp 321-324.

Donsez, D., 2007, On-Demand Component Deployment in

the UPnP Device Architecture, Proc of the IEEE

CCNC 2007

Flippo, F. Krebs, A. Marsic, I. 2003, A Framework for

Rapid Development of Multimodal Interfaces. Proc. of

the ICMI’03 conference, IEEE Computer, November

5-7, Vancouver, Canada, pp 109-116

Han, R., Perret, V., Naghshineh, M., 2000. WebSplitter: a

unified XML framework for multi-device

collaborative Web browsing, Proc. of the ACM

conference on CSCW, December 2000, Philadelphia,

Pennsylvania, United States, p.221-230.

Konomi, S. Roussos, G. 2007, Ubiqitous computing in the

real world: Lessons learnt form large scale RFID

deployments, Personal Ubiquitous Computing,

Springer Verlag, Heidelberg, Germany, vol 11, pp 50-

521.

Korpipää, P. Malm, E.J. Rantakkoko, T. Kyllönen, V.

Kela, J. Mantyjärvi, J. Häkkilä, J. Känsälä, I. 2006,

IEEE Pervasive Computing, July-September 2006, 82-

90.

Kourouthanassis, P. Roussos, G. 2006, Developping

Consumer-friendly Pervasive Retail Systems. In

Pervasive Computer, IEEE press, April-June 2003, pp

32-39.

OSGi Alliance, 2007, http://www.osgi.org

Satyanaryanan, M., 2001, Pervasive Computing: Vision

and Challenges. In IEEE Personal Communications,

IEEE press, August 2001, pp 10-17.

Wasinger, R. Wahlster, W. 2006, The Anthropomorphised

Product Shelf: Symmetric Multimodal Human-

Environment Interaction. In True Visions: the

Emergence of Ambient Intelligence, Aarts, E.

Encarnçao, E. (eds) Springer Verlag, Heidelberg,

Germany.

ICEIS 2008 - International Conference on Enterprise Information Systems

194