USING INTERACTION TRACES FOR EVOLUTIONARY

DESIGN SUPPORT

Application on the Virtual Campus VCIel

Karim Sehaba

Université de Lyon, CNRS, Université Lyon 2, LIRIS, UMR5205, F-69676, Lyon, France

Stéphanie Mailles-Viard Metz

Université de Montpellier, CNRS, Université Montpellier 3, PRAXILING, UMR5267, F-34000 Montpellier, France

Keywords: e-Learning, Interaction traces, User-centered design, Evolutionary design.

Abstract: This article addresses evolutionary design in the context of e-learning devices. It proposes a design

approach that takes into account the changing behaviours and needs of different actors (tutors, authors and

learners) in order to evolve and adapt a training device to real practices. For this, our approach is to consider

traces of interaction as knowledge sources that the designer can exploit in the design process. The principle

technique of our proposal is to observe the quantitative and qualitative actions of actors on the learning

platform and to represent them in modelled traces, to transform these traces in order to extract high level

information on the actors activities, and finally, to propose visualisation tools of this information. We have

applied our work in virtual campus VCIEL, where data for this study were obtained via participation of 2

designers, 13 tutors and 68 students from four classes that have been trained since 2006.

1 INTRODUCTION

In most methods of device design of online training,

tasks and resources of actors are usually predefined

by the designer according to a number of scenarios

"presupposed", and do not take into account

changing needs and behaviours of users. These

methods suffer from many disadvantages because

they cannot adapt themselves to different situations

encountered in practice. Moreover, designing a

system with a complete representation of user

requirements with which he/she interacts is not

always easy for the designer.

To address these problems, we propose an

approach to control the design by analysis and

visualization of interaction traces. The traces

mentioned here are defined as a history of user

actions collected in real time from their interaction

with the system. Formally, a trace is a set of

observed elements temporally located (Clauzel,

Sehaba & Prié, 2009). Each observed element

represents the user action on platform tools such as

opening a file, clicking on a hyperlink, posting a

message on the forum, etc. These traces, called

quantitative traces, allow us to represent the user

activity (designers, authors, tutors and learners) as

computer data elements that we can transform,

visualize, or share; in order to extract knowledge

about the user activities. To analyze better and

interpret these activities, we also define qualitative

traces. These can represent the user’s point of view

of his/her own activities. Our approach is to consider

these two types of traces as knowledge sources that

the designer can use in order to evolve and adapt the

training device. Traces can also be beneficial to

learners in their learning process through reflexive

learning.

The principle method of our approach is, during

a ﬁrst phase, the collection phase, to observe and to

store the user’s actions in the form of modelled

traces. At a second phase, the transformation phase,

traces of meaningful high-level representations to

the user are calculated. At the third phase, the

visualisation phase, we visualize, in an interactive

way, the trace for the different actors of learning

environment.

237

Sehaba K. and Mailles-Viard Metz S..

USING INTERACTION TRACES FOR EVOLUTIONARY DESIGN SUPPORT - Application on the Virtual Campus VCIel.

DOI: 10.5220/0003299602370242

In Proceedings of the 3rd International Conference on Computer Supported Education (CSEDU-2011), pages 237-242

ISBN: 978-989-8425-50-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

Our work relates to the training device VCIel

(vciel.univ-lyon2.fr). This is a Master (Bac + 5)

fully-online learning environment that aims to train

professionals in the field of multimedia production,

2D-3D computer graphics, and project management.

In this innovative framework, designers have few

examples to which they can refer. We assume that

the method might be improved by a scalable design

approach based on traces. Thus, in the context of this

work, we have traced and analyzed the activities of

different actors involved in VCIel training since

2006. The traces that we considered in this study are

both quantitative and qualitative. The data from this

study are related to the participation of 2 designers,

13 tutors and 68 students from four classes that have

been trained since 2006.

The article is organized as follows: Section 2

presents and discusses some methods of ergonomic

design and introduces the concept of trace theory

that form the theoretical framework of our research.

Section 3 presents the principle of our design

approach based on the interaction traces. Section 4

shows the change of design choices by observing the

practice in the context of VCIel. The last section

presents the conclusion and perspectives.

2 THEORETICAL FRAMEWORK

This section presents the theoretical basis of our

approach. These are user-centered design methods in

the ergonomics field and the concepts of traces on

which our work is based.

2.1 The Design Process in Ergonomics

Design can be defined as an individual and

collective activity, finalized by a project to develop a

physical and symbolic artefact (Visser, 1991). Its

peculiarity is that it always starts with not–well-

defined problems. Recommendations are proposed

to guide the actors in the process.

According to the French and international

standardization body (AFNOR & ISO), for

recommendations of ergonomic design, quality is

defined as "the ability of a product or service to

satisfy the needs of a user". This concept is used and

applied particularly in the industrial design process

to put the user at the center of the process of

building products that are intended to be used by the

user. It is the user-centered design considered by

Norman (Norman, 1999) and others (Eason, 1987).

The main idea is the participation of the end user of

the product in the design process: the user is

somehow incorporated into the design team.

Research has led to the introduction of ISO

standards (ISO, 1999) that define the stages of the

process: the planning process, understanding and

specifying context of use, the user and

organizational requirements, producing design

solutions, and finally, assessing solutions in terms of

pre-defined requirements.

For each step, methods are

recommended to define better the characteristics of

users. This brief description of work around the

design shows the complexity of both the work of

designers as to the effectiveness of the approach.

2.2 The Modelled Traces Concept

In order to identify and better understand the needs

and behaviours of different actors, we use the theory

of interaction traces developed by the SILEX team

(http://liris.cnrs.fr/silex). Indeed, the SILEX

(Supporting Interaction and Learning by Experience)

research team has been working on traces for several

years, building applications and studying various

usages (Clauzel, Sehaba & Prié, 2009) (Cram,

Jouvin & Mille, 2007) (Settouti, Prié, Marty & Mille

2009). By definition, a trace is composed of

observed elements representing the interaction

between the user and the system. We call an

observed element any structured information

generated from the observation of this interaction.

Formally, each observed element has a subject, the

user that was observed during the collect, and set of

attributes/values that is related to the temporal

extension of the trace (e.g. it can be related to an

instant or a temporal interval). Each trace is

associated with a trace model that defines the types

of observed elements (i.e the attributes that

characterize them) and the types of relationships

they can maintain them. The trace model also

specifies how it is possible to understand and use the

trace.

The SILEX team also has defined a Trace-Based

Management System (TBMS) (Settouti, Prié, Marty

& Mille 2009) (Clauzel, Sehaba & Prié, 2010) that

manages the modelled traces, in paticullar

transformation and visualisation of traces. Based on

the design process in ergonomics and the trace

theory, the idea of our approach is to use the concept

of modelled traces to generate knowledge about the

activities of different actors in order to improve the

process of designing. This approach is presented in

the next section.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

238

3 DESIGN APPROACH

BASED ON TRACES

In this section, we present our evolutionary design

approach based on both quantitative and qualitative

traces. The quantitative traces represent the user

activity by low-level of observed elements which

they must be transformed in order to extract high-

level knowledge. The observed elements of a

quantitative trace can be for example: the learner

has opened such a file at t

, has clicked on such link

at t

, etc. The qualitative traces can give the point of

view of a user on his own activities. The collection

of such traces is usually based on questionnaires.

The nature of the questionnaires depends on the

application domain and the objectives of the

analysis. Section 4 presents an example of a

questionnaire intended for tutors of VCIel training.

In this section, we present our methodology for

collecting and using the quantitative traces.

3.1 The Quantitative Traces

We focus on a dynamic approach to study changes

in the activities of different actors in the device to

include in a process of evolutionary design. We rely

on the use of interaction traces as sources of

knowledge that actors can operate in different ways

to improve the quality of the device. Our approach

considers the following three phases: collection,

transformation and visualization.

3.1.1 Collection Phase

All interactions between the user and the tools of the

platform are stored in a first record. It is a raw trace

representing all user actions in observed elements.

The collection phase ensures the observation of the

use of platform tools from sources tracing. It

elaborates, in an automatic or semi-automatic way,

information generated by the interaction between

users and the tool into a raw trace. We call a source

tracing any structured information flow from which

it is possible to establish a process for traces

collection.

The raw trace is not always directly usable, and

one or more transformation(s) are needed to reach a

trace with a coherent level of activity (i.e. significant

in the context for the user).

3.1.2 Transformation Phase

The raw trace contains basic information that is

necessary to analyze and extract information from

the highest level. The extraction is complex for

several reasons:

 The raw trace contains a wealth of data produced

by events generated by user actions;

 Analysis of the raw trace must often be

supplemented by information from other sources

(surveys, tutor annotations, etc.) to be relevant;

 Analysis of the raw trace requires expertise that

the actors do not know necessarily.

All these difficulties led to a formalization (Settouti,

Prié, Marty & Mille 2009) that facilitates the

treatment and processing of the raw trace to achieve

most relevant traces. A traces transformation is a

process that can transform one or more trace to

another trace high level. A transformation model is a

set of rules expressing selection conditions or

rewrite patterns. They include, for example,

production of indicators and patterns of use and

design. (Settouti Prié, Marty & Mille 2009) gives

more details on the transformations model.

3.1.3 Visualisation Phase

Broadly speaking, the visualization of traces is

considered the best way to provide feedback to re-

engineering environments (Cram, Jouvin & Mille,

2007). We apply this principle for a scalable through

a distance learning.

For the designer, the trace visualization allows to

detect the emergence of new uses made by the tutor

and learner and to assess the suitability of a resource

in a learning situation, e.g. the number of aid

applications and assistance. For the tutor, the trace

visualization allows monitoring of the learner, e.g.

the number and duration of consultations of the

course. It also allows the assessment of the level of

collaboration among learners, e.g. the number of

messages posted on a forum.

The information contained in the trace of

interaction is part of the use of the learner. By

visualising its own traces, the learner should be

better able to take its environment and therefore

adapt their work as demonstrated by the reﬂexive

learning works (Soller, Martinez & Jermann, 2005)

(Schön, 1983). Indeed, the display traces will allow

learners to wonder about the experiences they lead,

the results they produce, and the knowledge they

conclude. Thus, by visualisation of traces, the

learner can ensure the relevance of his/her approach

or to readjust his/her actions (Cram, Jouvin & Mille,

2007).

USING INTERACTION TRACES FOR EVOLUTIONARY DESIGN SUPPORT - Application on the Virtual Campus

VCIel

239

4 USING TRACES TO MODIFY

DESIGN CHOICES

Recall that our goal is to show that the activities of

actor change and allow the specification of new

design choices that may improve the quality of the

training device. The design process is dynamic: the

evolving needs of actors to put in place new tools for

supporting their activities, and these new tools lead

to the elaboration of new requirements.

The analysis focuses on the evolution of the use

of platform tools SPIRAL (Renaut et al., 2006) by

each actor over the past four years (4 classes) of

VCIel training. For each actor, we specify the task

that he/she must accomplish (prescription), the tools

at his/her disposal; and the way he/she actually uses

them. This analysis would show that these practices

are changing the requirement of the beginning task,

which re-starts the design process. Data were

collected from:

 Direct observation on the use of the platform by

students (quantitative traces);

 Indirect observation based on questionnaires to

authors and tutors (qualitative traces). The

questionnaire below is an example of a questionnaire

that has been destined to tutors of VCIel training.

4.1 Designers

The designer must ensure the coherence of the

pedagogical and technical device. His/her

responsibility is to develop a learning strategy taking

account of evolutionary changes in the use of tools

on the platform.

Thus, the role of the designer is not limited to

prescribing scenarios; he/she also anticipates the

changing needs by monitoring and analysis

purposes. For this, the designer is responsible for:

 Identifying relevant observables. These are

elements that will be traced during the interaction.

 Building indicators of use to change the device.

Working groups have made documents during the

early phase of design. They allow one to specify the

activities of actors: authors, tutors and learners. They

are regarded as contractual, and describe the

required task of each. The analysis of the device use

may allow one to adapt better these documents to the

context. This analysis is conducted through surveys

of various stakeholders and at meetings.

4.2 Authors

The author is responsible for designing and

preparing the course materials until they are posted

online. His/her role can be summarized as:

 Designing of content/structure of supports;

 Screenwriting teaching activities;

 Updating content.

Thus, the evolution of each medium requires a

consideration of the use made by the tutor of the

content and required scenario. Design tools and

multimedia support scripts are available to authors

on the platform. Trace of activities conducted with

these tools is not registered. Therefore, it does not

allow a reflexive analysis by the author. However, it

is possible to extract traces of usage of the content of

tutors to perform a gap analysis between the made

scenario and the prescribed scenario.

We note that at the start of training, the majority

of authors also play the role of tutor. The author has

a return on the work he/she has advocated and can

adapt to the context. However, since the third class,

and for some courses, some authors are no more

tutors on the course they have designed. Therefore,

there is a new role that will lead to a reflexion on a

tool providing communication between authors and

tutors, to integrate a user-centered design in the

same design and content of their scenario.

4.3 Tutors

The tutor is responsible for the supervision and

monitoring of learners in different learning activities

until the final test. For this, he/she is supposed to

follow the teaching scenario advocated by the author

of course, document and manage learners, assist

them in their understanding of the course and

provide an hour of chat per week for 10 weeks to the

whole group.

Tutors have tools for synchronous and

asynchronous communication, an area of exchange

of documents and administration tools to manage

rights, view profiles and traces of learners.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

240

Since the beginning of training, tutors platform

connections have decrease over the years. Hence, the

behaviour of tutors has changed. Several factors

justify this result:

 Appropriateness of tools of the platform over

time;

 Design of course when the training starts;

 Decrease of the number of tutors and students

(21 in 2006, 17 in 2007, 12 in 2007-2008 and 18 in

2008-2009 and for the current year).

In parallel, evidence suggests that some tutors

modified the initial pedagogical choices. One of

them, a tutor of the course on a tool for developing

animations, asked to cancel the chat sessions to

communicate with students via email on individual

productions. It is therefore very difficult to know,

through the export of data to the platform, if tutors

use the tools differently, if this difference is related

to a pedagogical or technical choice, if there are

patterns of use. Trace of connections is quantitative

and global. A formalization of individual traces from

pre-defined categories would be necessary to

implement on the platform.

Thus, the analysis of tutors trace connections

allows us to assume appropriateness of the device,

pedagogical and technical, on their part. However,

this type of trace does not inform on the process that

has allowed this appropriation process. To do this,

more qualitative and individual traces would be

required, through an export platform or through

interviews conducted to determine better what has

changed in how they work. The results could

provide elements for designers to improve the

contractual documents for tutors, setting out their

tasks, and offer new or different settings on the

platform.

4.4 Learners

During the training, learners use the tools at their

disposal on the platform to assimilate the content. As

authors and tutors, they sign a contractual document

that describes the activities, whether pedagogical or

technical, that may result in the training. They must

respect the planning for chat sessions. In case of

difficulty, they can contact the tutor by email. They

participate in collaborative meetings held by the

tutor.

Figure 1 presents the development of learners'

connections to the platform, the number of mails

sent and forums consultation. In contrast to the

tutors, there is an increase in connections over the

years, suggesting that learners operate more tools

from the platform (forum, chat, exchange area,

documents...). The results also show that the

evolution of the use of each tool is different at any

given time. While connections to the platform are

important at the beginning of the semester, and then

decrease with time, the number of mails is relatively

stable during the first months, while access to the

forum is important in the middle of the semester.

Platform connections

Forum consultations

E-mail sendings

Figure 1: Use evolution of the tools of the platform by

learners.

The designer may use these results to rewrite the

learner and tutor contractual documents and think to

the organization of the platform. For example, to

increase the number of learners attending the chat

sessions, the strategy was to schedule these sessions

in two days of the week and not spread throughout

the week for working students. The important use of

the forum in the middle of the semester coincides

with the start of the course "Realization" that

requires teamwork, which suggests that learners

promote the forum as a means of communication.

Currently, all messages from all the teams are stored

in the same space, this is one problem. To facilitate

USING INTERACTION TRACES FOR EVOLUTIONARY DESIGN SUPPORT - Application on the Virtual Campus

VCIel

241

access to data, it would be best for each team to

create a space for exchange dedicated to it.

5 DISCUSSION

AND CONCLUSIONS

This article presents a design approach based on

traces of interaction. The aim is to evolve the design

process by observing and analyzing activities of

different actors. The advantage of this approach is

that it takes into account the use to re-examine the

design. Indeed, the trace to assist in understanding

the activity of each actor and identify new needs.

This is particularly interesting for the design of

complex projects such as VCIel. However, the trace

does not always identify the problem specifically for

use because it is often of a quantitative nature.

Indeed, most of our interpretations, even if they

come from questions about quantitative data, are

derived from qualitative data. The quantitative

aspect, as suggested by data from the platform, is

therefore not enough to interpret the reasons for the

observed behaviour. We propose therefore to bind

traces of a different nature, which would deepen the

analysis but also to identify specific profiles of use.

In addition to improve the overall design process,

the traces can be considered as sources of

knowledge on learning of the learners as individuals

(reflexivity) or as a group (collaboration, sharing,

coordination, etc). In this sense, the analysis of the

trace can improve the pedagogical scenario and

adapt to the situation. Eventually, the trail could also

be used for the sharing of experience between actors

and indirectly allow a return on the design.

The theory has enabled us to glimpse the

complexity of the design process in various ways:

poor definition of the problem, collaboration

between members of the team of designers, the

importance of integrating the end-user at various

levels of the process. VCIel experienced this

complexity, and this shows how the creation of

online training courses can be considered as the

design of products studied in ergonomics: industrial

products... With regard to the poor definition of the

problem, VCIel is actually poorly defined in the

sense that it is innovative with actors-designers

beginners to this type of work. The user-centered

design proposed by (Norman, 1999) seems to be

entirely appropriate to address these difficulties.

Indeed, during the process, it was found that if the

user has been interviewed, tested, and he/she is

integrated in the development of tools, it will allow a

certain quality tools at the end of the cycle.

ACKNOWLEDGEMENTS

The authors would like to thank all tutors of VCIel

training for their help.

REFERENCES

Clauzel, D., Sehaba, K., Prié, Y. 2009. Modelling and

visualising traces for reflexivity in synchronous

collaborative systems. In International Conference on

Intelligent Networking and Collaborative Systems.

IEEE Computer Society, pp. 16–23.

Clauzel, D., Sehaba, K., Prié, Y. 2010. Enhancing

synchronous collaboration by using interactive

visualisation of modelled traces. Simulation Modelling

Practice and Theory Journal. (To appear).

Cram, D., Jouvin, D., Mille, A. 2007. Visualizing

interaction traces to improve reﬂexivity in

synchronous collaborative e-learning activities. In 6th

European Conference on e-Learning. A. C. Limited,

Ed., Oct. 2007, pp. 147–158.

Eason, K. 1987. Information technology and

organizational change. London: Taylor and Francis.

ISO 13407 (1999) User centered design for interactive

systems.

Norman, D. A. 1999. Invisible Computer: Why Good

Products Can Fail, the Personal Computer Is So

Complex and Information Appliances Are the Solution.

London, MIT Press.

Renaut, C., Batier, C., Flory, L., Heyde, M., 2006.

Improving web site usability for a better e-learning

experience, In A. Méndez-Vilas, A. Solano Martín, J.

A. Mesa González and J. Mesa González (Eds.)

Current Developments in Technology-Assisted

Education, Formatex, Vol. 2, pp. 891-896.

Schön, D.A., 1983. The Reﬂective Practitioner: How

Professionals Think in Action. Basic Books, june

1983.

Settouti, S., Prié, Y., Marty, J.-C., and Mille, A. 2009. A

trace-based system for technology-enhanced learning

systems personalisation. In The 9th IEEE International

Conference on Advanced Learning Technologies, Jul.

2009.

Soller, A., Martinez, A., Jermann, P., Muehlenbrock, M.,

2005. From mirroring to guiding: a review of the state

of the art technology for supporting collaborative

learning. International Journal of Artiﬁcial

Intelligence in Education 15 (2005) 261–290.

Visser, W. 1991. The cognitive psychology viewpoint on

design: Examples from empirical studies. In J. S. Gero

(Ed.), Artificial Intelligence in Design '91 (pp. 505–

524). Oxford, England: Butterworth-Heinemann.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

242