User Centered Approach for Learning Analytics Dashboard Generation

Dabbebi Ines

1,2

, Gilliot Jean-Marie

and Iksal Sebastien

UBL, Le Mans University, LIUM Laboratory, Laval-LeMans, France

UBL, IMT Atlantique, Lab-STICC Laboratory, Brest, France

Keywords:

Learning Analytics, Dashboard, Generation Process, Decision Making, Business Intelligence.

Abstract:

The use of learning dashboards with analytics might help users to gain insight into their learning process and

then to make decision. However, designing meaningful Learning Analytics dashboard (LAD) is still a complex

process that requires an explicit understanding of the user needs. For this reason, we carried out a user-centered

design (UCD) approach with the aim to provide users with adapted LADs to support their decision-making.

Hence, we develop a UCD process composed of 4 steps: (i) we propose a participatory-based design tool for

capturing contextualized needs (ii) these identiﬁed needs will be described using an independent formalism

and LAD models in order to capitalize on them (iii) to automate these models, we propose a LAD generation

process (iv) ﬁnally, we carried out an evaluation phase with the aim to review and reﬁne our models. During

our process development, an iterative user needs reﬁnement conﬁrmed that decision is considered as a centered

element for LAD generations.

1 INTRODUCTION

In recent years, Learning Analytics (LA) has emerged

as a research domain. LA is deﬁned as the measure-

ment, collection, analysis and reporting of data about

learners and their contexts(Siemens and Long, 2011).

It aims to propose the means to collect and model

digital traces of e-learning, and implement learning

indicators and visualizations. Nevertheless, LA is

confronted with heterogeneity problems(Chatti et al.,

2012). Therefore, analysis processes must be re-

deﬁned when their implementation context changes:

they cannot be reused, shared or easily improved

(Lebis et al., 2016). Since data visualization is part of

the analysis process, the deﬁnition of Learning An-

alytics Dashboard (LAD) models is necessary as a

means toward the capitalization of LAD’s structure.

Actually, LAD is deﬁned as a single display that ag-

gregates different indicators about learner (s), learn-

ing process(es) and/or learning context(s) into one or

multiple visualizations(Schwendimann et al., 2017).

In this paper, we aim to propose an independent

formalism and models to describe LAD. We propose

also an automatic process for generating LADs. Our

aim is therefore to capture needs in the most exhaus-

tive way possible, to capitalize on this capture through

explicit models, and to propose a generation of learn-

ing dashboards based on these models. LAD has been

studied by various researchers. Some existing works

proposed LADs to help teachers during their learning

activities(Xhakaj et al., 2017) or to study the effect

of LA on student performance (Arnold and Pistilli,

2012). All these works occur in predeﬁned contexts

while presenting speciﬁc indicators with the aim to

lead users to visualize what they want. Hence, we

work on generating LAD in a dynamic way with re-

spect to users contexts. The main idea is to support

users identifying which and when the decision should

be made. Therefore, we decided to follow a user-

centered design (UCD) approach. We present there-

fore our works dedicated to answer these following

research questions that focuses on:

How to capture user needs and contexts? We assume

that users can make decision using LAD. Thus, LAD

content should meet user requirements and answer

their visualization objectives.

How to generate an adapted LAD? Our works aim

to design LAD generation process in a dynamic and

contextual way. The generation process should take

into account a generic LAD model while highlight-

ing the links between different key elements: learning

indicators, visual displays and user contexts.

260

Ines, D., Jean-Marie, G. and Sebastien, I.

User Centered Approach for Learning Analytics Dashboard Generation.

DOI: 10.5220/0007693102600267

In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 260-267

ISBN: 978-989-758-367-4

2 USER-CENTERED DESIGN

APPROACH

LAD design requires a well-understanding of user

needs and their contexts so that we can help them

make a decision. We conducted a User-Centered De-

sign (UCD) approach. It is a well-known design ap-

proach that aims to develop, in an iterative way, an

explicit comprehension of users(Jokela et al., 2003).

UCD can help a designer to establish user needs, pref-

erences and limitations, to discover design opportu-

nities throughout the design process where user sat-

isfaction becomes a key measure of design success

(Lanter and Essinger, 2016), which helps to evalu-

ate both the effectiveness and relevance of the tools

(Miksch and Aigner, 2014), leading to a ﬁnal valida-

tion by the user (Doroftei et al., 2017). UCD have

four main phases:

Phase 1: Understand Users and their Context. The

objective is to identify relevant method for captur-

ing contextualized needs. As a result, we proposed

a participatory-based tool.

Phase 2: Specify User Requirements. Capturing

user needs is a means to create and reﬁne our LAD’s

models mutually. This phase is considered as a direct

interpretation of user needs.

Phase 3: Produce Design Solutions. We have car-

ried out an automatic LAD generation from these

models. In fact, we automate this phase with a LAD

generation process (DGP).

Phase 4: Evaluate design proposal against require-

ments The evaluation aims, ﬁrstly, to review and re-

ﬁne our LAD’s models and secondly, to capture new

users’ needs in order to provide adapted LADs.

3 PHASE 1: UNDERSTAND

USERS AND THEIR CONTEXTS

We detail in this section, three iterations used to un-

derstand and to reﬁne user needs: (i) We conducted

a needs assessment study that demonstrated the difﬁ-

culty to capture user needs (ii) We carried out partic-

ipatory meetings with teachers and proposed sketch

activities to enhance the user needs expression and ﬁ-

nally (iii), we proposed a participatory-based tool.

3.1 First Iteration: Needs Assessment

Study

The focus of this iteration was to develop an explicit

comprehension of users contexts. This study includes

survey and semi-structured interviews(Dabbebi et al.,

2017). We organized the study around ﬁve ques-

tions based on the Five W’s and How methods. Each

question refers to a contextual dimension:”Who, Why,

What, How, When”.

”Who” dimension is used to specify users and their

roles.

”Why” dimension relates to the identiﬁcation of

users’ objectives. In other words, it aims at under-

standing why it is important for the user to use LAD.

”What” dimension is dedicated to identifying LAD

content especially indicators.

”How” dimension refers to the description of LAD

structure.

”When” dimension is used to specify the way in

which the dashboard is used.

Following this iteration, we noted that users found

difﬁculties in expressing their needs clearly through

a simple questioning. Actually, users couldn’t eas-

ily project themselves into use of LAD or imagin-

ing the possibilities especially when asking teachers

about the use of digital data and indicators. For this

reason for the second iteration, we organized partic-

ipatory workshops with the aim of stimulating their

imagination and leading a brainstorming.

3.2 Second Iteration: Participatory

Workshops

We organized participatory workshops with 8 elemen-

tary and high school teachers. This group of teachers

use the same learning platforms called the Tactileo

Map application (Sanchez et al., 2015). The work-

shop was organized around two main activities:

1) The sketch activity is making teachers, through a

simple sketch, express their ideas, visualize what they

imagine, share and discuss with others. This activity

aims to gather additional information around the ﬁve

contextual dimensions. During this activity, teachers

start by identifying their main objective which was

monitoring. The choice of this objective was logical.

Indeed, (Schwendimann et al., 2017) did identify two

main LAD objectives: self-monitoring or monitoring

other. During this workshop, we noted that teachers

did discuss this objective according to different levels

of analysis. Actually, (Siemens and Long, 2011) did

identify three levels of analysis: individual, class and

departmental level. Teachers choose to work together

on a single LAD description. The chosen LAD is ded-

icated to help teachers monitor their student activities

according to class level.

2) Wireframe activity is a visual guide that represents

an abstract LAD structure including content organi-

zation. During this activity, we extract two informa-

tion: a) indicators related one to another are grouped

User Centered Approach for Learning Analytics Dashboard Generation

261

into the same section, b) indicators placed regarding

their importance. During this iteration, we noticed

that users project more easily in their decision to be

made rather than their objective. Like (Xhakaj et al.,

2017), we consider that the expression of decision-

making related to LAD might inﬂuence its design. In

fact, the dashboard, as a key concept in Business In-

telligence (BI), is deﬁned as a support that improves

decision-making and user performance (Rasmussen

et al., 2009). Our focus on BI is related mainly to

the fact that LA at the beginning was emerged from

BI (Siemens and Long, 2011).

We also noted the diversity of the user’s needs

that is difﬁcult to capture. To go further, an adapted

tool was therefore necessary and a participatory de-

sign method needs to be instrumented (Sanders et al.,

2010).

3.3 Third Iteration: A

Participatory-based Design Tool

Inspired by the conduct of our participatory work-

shops, we propose therefore this time a tool-based

method. Despite its low use in LA (Abel and Evans,

2013), participatory tools allow a better expression of

user needs and the exploration of design alternatives

and it helps tools appropriation (Knibbe, 2016). We

propose a design space (Shaw, 2012) through a tool

to support the participatory LAD design. We have

designed our design space around the same contex-

tual dimensions (Gilliot et al., 2018). We represents

this design space with cards and boards. As results

of previous iterations, we decided to represent the de-

cision to be taken as a centered element. The choice

of decision must be explained and then characterized.

Then, we represent the context of use. This is about

characterizing elements around the speciﬁed decision

to be taken. We propose to participants the opportu-

nity to deﬁne the data needed to be visualized and to

associate it with visual displays. Finally, to organize

and deﬁne a LAD structure, we propose to partici-

pants a second board represent a paper prototyping

steps. This participatory-based design tool was tested

in two different contexts. A ﬁrst test workshop was set

up with teachers in computer science, and the other

one was proposed to national education supervisory

staff. The workshop process was observed and ﬁlmed,

with the agreement of the participants. A satisfaction

questionnaire was also used to gather the participants’

opinions. We proposed seven questions requiring a

level of agreement on a 5-level Lickert scale. Overall,

the level of satisfaction we consider to be high (all

answer was higher than 3). The ﬁrst feedback from

a test workshop is encouraging, and shows that this

form of workshop effectively facilitates cooperation

and further exploration of alternatives.

3.4 Findings

During these iterations, we identify four points:

First, the context in which LAD is used inﬂuences

its organization. It is therefore necessary to be able to

formalize this context, both to help users reﬂect and to

capitalize on these uses for reuse. Even if each LAD

depends on a particular user’s objective, we noted it

can be related to the user’s contexts. In our exam-

ple, we noticed that teachers are more interested in

monitoring their students activities while administra-

tive staff aims to monitor courses popularity(Dabbebi

et al., 2017).

Second, we consider that LAD reﬂects users dur-

ing their learning situation. So, we noted that the ex-

pression of decision-making related to LAD visual-

ization can affect its design.

Third, we found out that representing indicators

in LAD is connected to different levels of analy-

sis. Teachers have organized their proposed indica-

tors based on the three level of analysis: departmen-

tal, class and individual level.

Finally, LAD displays different indicators

(Schwendimann et al., 2017). These indicators can

be visualized through simple textual representation,

or through the use of sophisticated visual display.

The choice of visual displays depends on the user’s

preferences, the context of use and the choice of

indicators itself. In the end, we noted that our

participatory-based design tools take into account the

identiﬁcation of these ﬁndings. Actually, this tool

is being used from need capture to LAD modeling,

which it will be discussed in the next phase.

4 PHASE 2: SPECIFY USER

REQUIREMENTS

The two phases ”understand users and their contexts”

and ”specify user requirements were tightly coupled

as our models were reﬁned at each iteration. The main

objective of these iterations is to specify all the ele-

ments that make it possible to characterize the rele-

vant LAD’s structure(s) and gradually reﬁne all our

conceptual models. The modeling phase can be used

to formalize and to evaluate the identiﬁed needs with

the aim to make it operational. LAD can be consid-

ered as an aggregation of a set of components, includ-

ing indicators and visual displays that serve to an-

swer user context (Schwendimann et al., 2017). In-

deed, identifying these components separately can be

CSEDU 2019 - 11th International Conference on Computer Supported Education

262

used to promote their reuse in similar contexts. In our

work, LAD reuse is maintained by three main points:

1. A meta-model used to describe LAD components

in a generic way to ensure its reuse.

2. A LAD pattern used to specify how to organize

LAD components regarding user contexts.

3. A generic description of LAD key elements to en-

sure its identiﬁcation and its reuse in similar con-

texts.

4.1 LAD Meta-model

We proposed meta-model in order to describe mainly

the organization of LAD components. In other words,

it represents explicitly LAD structure.

We propose an hierarchical structure to better or-

ganize displays and to facilitate generations according

to user needs. In our work, we focus on screen visu-

alizations. So, we adopt a classic document structure.

We propose LAD meta-model based on three hierar-

chical components: widget, section and view.

A widget is an atomic component which repre-

sents visual elements such as a graph, grid, table or

image. It is usually represented by a pair of informa-

tion indicator,visual display.

A section represents an intermediate component.

It aims to regroup a set of related widgets in order to

highlight their relationships.

Finally, the view is considered as the root compo-

nent. It may be represented as a graphical container. It

was created mainly to provide a context in which the

various components can be arranged and displayed.

The deﬁnition of a view is about to regroup and orga-

nize a set of sections to meet a speciﬁc objective.

4.2 LAD Pattern Modeling

A pattern is used to group a set of generic speciﬁca-

tions around the selection and organization of LAD

in a speciﬁc context of use. The pattern is about de-

scribing both LAD structure and content in a generic

way. Actually, it is a main element to assist LAD

to data storytelling (Knaﬂic, 2015) in order to facil-

itate decision-making. This generic speciﬁcation is

designed mainly to ensure LAD reuse in similar con-

texts. We propose then to associate a pattern to LAD

view to guide its structuring.

Decision Guidance. The choice of pattern is mainly

related to user decision. In fact,in BI, decisions are

made at different levels in the organization’s hier-

archy: Operational, tactical and strategic(Golfarelli

et al., 2004).

The operational decision relates to the day-to-day

running of an activity. They are mainly dedicated to

the different business lines, which aims at measuring

the performance of a speciﬁc action or process.

The tactical decision level, which is narrow in

scope and short-term in nature, helps to implement

the strategy. These kinds of dashboards are usually

built for middle management.

The strategic decisions are long-term in their im-

pact. They affect and shape the direction of the whole

organization. They are generally made for senior

managers. Decision levels in general depends on (1)

the frequency of dashboard visualization and (2) on

the level of detail. (1)The frequency indicates the pe-

riod of time in which indicators should be presented.

(2)In LA, we identify mainly three level of analysis

that may be represented by three levels of detail which

they are individual, class or departmental level. The

choice of decision level inﬂuence the choice of indi-

cators. Only key performance indicators (KPI) should

be providing it to users(Pauwels et al., 2009). To as-

sist LAD to data storytelling, we need to identify deci-

sion levels and to help users to focus their attention on

a particular target. Indicator can be focused on learn-

ing activity, data producers, time and learning context

(Schwendimann et al., 2017). Thus, to guide decision,

we need to identify decision level and LAD targets.

4.3 LAD Key Elements

LAD generation requires three mandatory key ele-

ments to be modeled (Dabbebi et al., 2017): indica-

tors, visual displays and user context.

Indicator Modeling. In our case, indicators are al-

ready calculated by several analysis platforms. These

indicators are focused on a particular learning objec-

tive. This model answer mainly to the ”What” dimen-

sion. Indicators modeling aim to index and to catego-

rize indicators. This model includes the type of indi-

cator, the context of use (Learning objective, learning

situation, etc.) and its technical characteristics (data

quantity, dimensions, etc.). In our model, indicators

description should be sufﬁciently detailed in order to

facilitate their representation regarding the adequate

level of analysis, the level of detail and to facilitate

visual displays selection.

Visual Display Modeling. Data visualization helps

users to get into indicators and understand what is

happening in a simple way. The choice of an adequate

visual display may be critical. Indeed, presenting a

wrong visual display or providing a sophisticated one

can cause a wrong data interpretation. We, therefore,

propose a visual display model with the aim to de-

scribe and to index the existing visualization meth-

User Centered Approach for Learning Analytics Dashboard Generation

263

ods. The choice of visual display depends mainly on

user preference and on the characteristics of indica-

tors. Visual display can be used to empower indi-

cators objectives. This model represents mainly the

”How” dimension.

User Context Modeling. LAD generation requires

to understand who is the user. Each user has their

role, objectives, preferences, learning environment,

and which decision needs to take. This model will

help our generation process to provide the user with

an adapted LAD to make decision easily.

Using these different models in general aim to

assist and guide LAD generations. Thus, identify-

ing, organizing and presenting these key elements will

form the main steps of the generation process.

5 PHASE 3: LAD GENERATION

PROCESS

In this section, we present our LAD generation pro-

cess (DGP). The purpose is to produce LAD in dif-

ferent contexts in a dynamic way. We deﬁne in this

section the steps followed for LAD generations.

5.1 Process Description

DGP is organized around the same contextual dimen-

sions. Each dimension is represented within the dif-

ferent LAD models that will be operated by DGP.

In fact, DGP should respect LAD meta-model and

the chosen pattern to ensure the links between LAD

key elements. Hence, LAD should present a group

of learning indicators in line with the user context.

These indicators should be well-organized in a way to

respect to LAD meta-model and the pattern descrip-

tion. Then, to make these indicators easy to under-

stand, the identiﬁcation of an appropriate visual dis-

play is required.

We can resume DGP workﬂow in three steps:1)

Content identiﬁcation, 2) Content organization and

3)Dashboard Presentation;

5.1.1 First Step: Content Identiﬁcation

This step of content identiﬁcation is detailed in Fig1.

1) Identifying User Context: Based on user con-

text model, DGP start by identifying the user role,

learning platforms and learning activities and learn-

ing objectives (etc.) in order to be able to prepare

the dedicated learning indicators and to understand

his preferences. 2) Identifying user decision: De-

pending on expressed users objectives and on what

and when the decision needs to be made, DGP select

Identify user

decision

WHO? WHY?

Verify

Extract Key

Indicators

(KI)

Identify

decision

indicators

Identify LAD

pattern

Identify user

context

WHAT?

HOW?

WHAT?

Figure 1: Content identiﬁcation step.

LAD patterns which describe the level of decision in-

cluding the level of detail and the decision target. 3)

Extract Key indicators (KI): The selected pattern ver-

ify whether the selected indicators are considered as

key indicators (KI) or not. In BI, KI is deﬁned as

a set of measures that are considered most important

for an organization. Following the same logic, DGP

should identify only important measures. To evaluate

its importance, DGP should follow the pattern indica-

tion including mainly the decision target, the level of

detail. For example, when user decides to plan their

learning activities dedicated to a speciﬁc class, DGP

selects only the indicators related to the learning ac-

tivity as a decision target and it should extracts and

regroup all measures to represent a synthetic view of

a speciﬁc class activities. At this stage, LAD content

is ready to be used.

5.1.2 Second Step: Content Organization

This step is about establishing relationships between

indicators according to LAD meta-model. This step

is based mainly on a set of organizational rules with

the aim to:

• Propose an elaboration rule for representing a

pair of KI when one gives additional information

related to another KI. This rule can be applied

when the two KI share the same context, the same

source of data (same learning activity).

• Group indicators together when one provides a

short summary, it can be a textual indicator or

even a global indicator that summary the result vi-

sualized in another indicator.

• Propose a group of indicators together when they

represent all the same information with a different

level of detail.

The purpose is to transform LAD from a simple in-

formation visualization (which is a minimum LAD

requirement) to a decision-making tool by relying on

cause-and-effect relationships. By identifying these

relationships, DGP will be able to constitute and or-

ganize LAD components: widget, section and view.

This step ends by adjusting the chosen pattern to be

more adapted to user preferences.

CSEDU 2019 - 11th International Conference on Computer Supported Education

264

5.1.3 Third Step: LAD Presentation

This step is about preparing a list of possible visual

displays and populating LAD with data. 1) Selecting

Extracted

key

Indicators

Select Visual

display

Populate

widgets

Reselect(Purposes)

Generated

LAD

WHAT?

Adjust data

format

HOW?

WHAT?

Figure 2: LAD Presentation’s Step.

visual displays are based on different criteria. Firstly,

DGP should connect both indicator and visual display

models. Thus, it compares KI descriptions with those

of visual displays like their dimensions and data quan-

tity. Secondly, DGP will check users model to iden-

tify user preference. In the case of having more than

one visual displays for each KI, DGP propose visual

display by default. In case of failure, for example,

when kI measures represent the time evolution, but

DGP select a bar chart rather than a timeline chart be-

cause of the technical choices (regarding dimensions

or amount of data). Thus, DGP will reselect another

visual display sharing the same purpose as a bar chart

and represent the time evolution. Actually, this step is

based mainly on the kI visualization where each ex-

tracted key indicators will be presented into adequate

visual displays.

2) Adjust data format: The adjustment step is per-

formed automatically by the system. Considering that

each visual display has different data format require-

ments, it is necessary to adapt the kI measures. Then

the KI measures can be reprocessed individually ac-

cording to the required format. In case of failure,

DGP choose another visual display sharing the same

purpose.

3) Populate widget : After the KI measures are

adjusted based on the format, they are sent to the cor-

responding visualization algorithms. Actually, DGP

will work on populating every widget with their re-

spective data (measures and dimensions). Data will

be represented according to the chosen period of time

and LAD level of detail.

6 PHASE 4: EVALUATE DESIGN

PROPOSAL AGAINST

REQUIREMENTS

This phase is about evaluating the design proposal

throughout the iterative cycle. This is used mainly

to acquire feedback on DGP and to reﬁne our mod-

els. Moreover, we tried to highlight decision impacts

on LAD generations. Actually, the need to establish

an iterative evaluation phase is justiﬁed by our need

to ensure the effectiveness of our DGP. Indeed, (Dick

et al., 2005) recommends the use of rapid prototyping

throughout iterative cycle as a strategic and effective

approach to ensure product effectiveness. We con-

ducted this evaluation phase by implementing LADs

to TactileoMap teachers. We carried out two LAD

generation iterations. The ﬁrst iteration was about en-

suring an automatic LAD generation. The second it-

eration was about taking into account user decisions.

LADs was evaluated during focus groups. These fo-

cus groups were semi-structured. The idea is to guide

participants to evaluate and analyze LAD’s compo-

nents, although the discussion was open.

6.1 First Iteration: LAD Dynamic

Generation

The ﬁrst LAD was designed and developed for the

same group of teachers. We proposed LAD to help

teachers in monitoring student performance (Fig3).

The choice of LAD content and structure depends

mainly on teacher needs. Thereafter, we carried out

Tableau de bord

Génération dynamique

Section 1:

Top Ranking students

Section 2.1:

Synthetic view of Interaction

between groups of learners

Section 2.2:

Detailed view by group of

learners

Section 3:

Representation of learning activities

Figure 3: First LAD Generation.

an evaluation session based on focus group method.

We organized this session with two groups of teach-

ers(G1 and G2). Each group should evaluate LAD

content and structure. In other words, they should

evaluate, ﬁrst, the usefulness of learning indicator in

regard to their learning objective. Then, they evaluate

whether LAD presentation was easy to interpret.

Findings: Therefore, we detail teacher feedback:

Structure: Both G1 and G2 shared the same idea about

LAD structure. Actually, they found LAD organiza-

tion conforms to what they expected.

Content:

The evaluation includes the indicators and

visual displays.

We noted that each group had its own vision about

LAD content. The ﬁrst group (G1) need more de-

User Centered Approach for Learning Analytics Dashboard Generation

265

tailed indicators. While the second group (G2) asked

to have fewer indicators with synthetic data.

We have deduced that visual display choice was

affected by LAD level of detail and indicators. In fact,

G1 preferred to move from using simple histograms to

drill-down histograms with the aim to visualize more

detailed indicators.

We concluded that LAD generation was fully ade-

quate for both groups. We noticed that LAD was eval-

uated based on if it helps teachers take decisions and

not based on only their objective. In fact, we noted

even if teachers share the same learning objective,

they did not share the same need to make the same

decision. Whereas, the use of dashboard is clearly

illustrated in BI process. Thus, we noted that this it-

eration conﬁrms the need to adopt the BI approach.

6.2 Second Iteration: LAD Generation

of Decision-making

We carried out another LAD generation with a reﬁned

LAD model. This iteration guided by decision and

pattern speciﬁcations. We propose two versions of

LAD where each version represents a speciﬁc deci-

sion level. As we mentioned, decision level is deﬁned

by the level of detail and the frequency of use (deci-

sion horizon). These LADs are dedicated to answer

the same objective about monitoring learner perfor-

mance. However, each one of them will be guided by

different decisions (decision target and level).

We proposed for G1 an operational LAD (Fig4).

This choice was justiﬁed by the need to visualize de-

tailed indicators daily. Thus, we offered G1 the same

indicators with detailed view. In addition to decision

level, DGP presented kI according to G1 decision tar-

get which is learners. Actually, kI should be, on the

one hand, about learners. On the other hand, measures

that represent learners should be presented mainly on

x-axis. This will guide teachers to focus more on

learner performance rather than another target.

For the second group of teachers (G2), we pro-

posed a tactical LAD (Fig5). This decision level was

justiﬁed by the need to visualize relatively frequently

of fewer indicators. This is a medium-term decision.

Therefore, DGP selected only two indicators. The

purpose is to help teachers to evaluate and to organize

learners into teams for their future activities. Thus,

LAD decision target is the same as G1. Since the

frequency of use is different and G2 will not be able

to visualize daily indicators, DGP propose presenting

measures over a period of time to provide more syn-

thetic view. To conclude, the evaluation phase was

used to validate the effectiveness of our DGP. While

the ﬁrst iteration was a way to reﬁne our models, the

Operational

LAD

Learners Performance

over Time

Number of actions

performed by Learners

Detailed view of learners

activity

General Information about

learners activity

Figure 4: Operational LAD: Monitoring learners perfor-

mance over time.

Tactical LAD

Number of actions performed

by Learners

Detailed view of learners

activity

Figure 5: Tactical LAD: Monitoring learners performance

over a period of time.

second iteration demonstrated the impact of the de-

cision on LAD’s generation. Thus, we can conclude

that decision is considered as a centered element for

LAD generations. Actually, by changing only the de-

cision, DGP proposes a different dashboard automat-

ically.

7 CONCLUSION

A key assumption of our work is that users can make

decision using LAD. The idea is LAD can be more ef-

fective if it is designed with a thorough understanding

of user needs and contexts. In this paper, we carried

out a complete user-centered design approach with

the aim to capitalize, in an iterative way, an explicit

comprehension of users. We propose an equipped ap-

proach from capturing the user needs to LAD genera-

tion thanks a dedicated DGP which allows rapid pro-

totyping to have user evaluations. During our ﬁrst it-

CSEDU 2019 - 11th International Conference on Computer Supported Education

266

eration, we identiﬁed the users’ difﬁculties in express-

ing their needs. We have made the choice to introduce

the decision as a dimension of the process. The intro-

duction of the decision provided a number of informa-

tion: During the ﬁrst phase, we found a richer expres-

sion of needs. During the third phase of LAD gener-

ation, we found that introducing decision contribute

to the generation of different LADs. During the last

phase, we validate that these different LADs were rel-

evant to the decision identiﬁed. We therefore conﬁrm

that considering the decision as a centered dimension

in our work has a positive impact on the design of

LAD. The next stage of our work is to evaluate our

DGP with other users while reﬁning decision making

modeling. We also aim to capitalize LADs to enhance

the ability of our process to propose alternatives.

ACKNOWLEDGEMENTS

We thank all participants who took part in our stud-

ies. This work has been supported by the HUBBLE

project (ANR-14-CE24-0015).

REFERENCES

Abel, T. D. and Evans, M. (2013). Cross-disciplinary par-

ticipatory & contextual design research: Creating a

teacher dashboard application. IxD&A, 19:63–76.

Arnold, K.-E. and Pistilli, M.-D. (2012). Course signals at

Purdue: using learning analytics to increase student

success. The 2nd International Conference on Learn-

ing Analytics and Knowledge, pages 267–270.

Chatti, M. A., Dyckhoff, A. L., Schroeder, U., and Th

us,

H. (2012). A reference model for learning analytics.

International Journal of Technology Enhanced Learn-

ing, 4(5-6):318–331.

Dabbebi, I., Iksal, S., Gilliot, J.-M., May, M., and Garlatti,

S. (2017). Towards adaptive dashboards for learning

analytic: An approach for conceptual design and im-

plementation. pages 120–131.

Dick, W., Carey, L., and Carey, J. O. (2005). The systematic

design of instruction.

Doroftei, D., De Cubber, G., Wagemans, R., Matos, A.,

Silva, E., Lobo, V., Cardoso, G., Chintamani, K.,

Govindaraj, S., Gancet, J., et al. (2017). User-centered

design. In Search and Rescue Robotics-From Theory

to Practice. InTech.

Gilliot, J.-M., Iksal, S., Medou, D., and Dabbebi, I.

(2018). Conception participative de tableaux de bord

d’apprentissage. In 30eme conf

erence francophone

sur l’interaction homme-machine, page 9p.

Golfarelli, M., Rizzi, S., and Cella, I. (2004). Beyond data

warehousing: what’s next in business intelligence? In

Proceedings of the 7th ACM international workshop

on Data warehousing and OLAP, pages 1–6. ACM.

Jokela, T., Iivari, N., Matero, J., and Karukka, M. (2003).

The standard of user-centered design and the standard

deﬁnition of usability: analyzing iso 13407 against iso

9241-11. pages 53–60.

Knaﬂic, C. N. (2015). Storytelling with data: A data visu-

alization guide for business professionals. John Wiley

& Sons.

Knibbe, C. (2016). Concevoir avec des technolo-

gies

emergentes pour la construction conjointe

des pratiques et des artefacts : apports d’une

ethodologie participative

a l’innovation tech-

nologique et p

edagogique. Theses, Conservatoire na-

tional des arts et metiers - CNAM.

Lanter, D. and Essinger, R. (2016). User-centered design.

International Encyclopedia of Geography: People,

the Earth, Environment and Technology, pages 1–4.

Lebis, A., Lefevre, M., Luengo, V., and Guin, N. (2016).

Towards a capitalization of processes analyzing learn-

ing interaction traces. In European Conference

on Technology Enhanced Learning, pages 397–403.

Springer.

Miksch, S. and Aigner, W. (2014). A matter of time: Ap-

plying a data–users–tasks design triangle to visual an-

alytics of time-oriented data. Computers & Graphics,

38:286–290.

Pauwels, K., Ambler, T., Clark, B. H., LaPointe, P., Reib-

stein, D., Skiera, B., Wierenga, B., and Wiesel, T.

(2009). Dashboards as a service: why, what, how, and

what research is needed? Journal of Service Research,

12(2):175–189.

Rasmussen, N. H., Bansal, M., and Chen, C. Y. (2009).

Business dashboards: a visual catalog for design and

deployment. John Wiley & Sons.

Sanchez, E., Kalmpourtzis, G., Cazes, J., Berthoix, M., and

Monod-Ansaldi, R. (2015). Learning with Tactileo

Map: From Gamiﬁcation to Ludicization of Field-

work. GI Fourm Journal for Geographic Information

Science.

Sanders, E. B.-N., Brandt, E., and Binder, T. (2010). A

framework for organizing the tools and techniques of

participatory design. In Proceedings of the 11th Bien-

nial Participatory Design Conference, PDC ’10, pages

195–198, New York, NY, USA. ACM.

Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk,

A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet,

D., and Dillenbourg, P. (2017). Perceiving learning

at a glance: A systematic literature review of learning

dashboard research. IEEE Transactions on Learning

Technologies, 10(1):30–41.

Shaw, M. (2012). The role of design spaces. IEEE software,

29(1):46–50.

Siemens, G. and Long, P. (2011). Penetrating the fog: Ana-

lytics in learning and education. EDUCAUSE review,

46(5):30.

Xhakaj, F., Aleven, V., and McLaren, B. M. (2017). Ef-

fects of a teacher dashboard for an intelligent tutor-

ing system on teacher knowledge, lesson planning,

lessons and student learning. In European Conference

on Technology Enhanced Learning, pages 315–329.

Springer.

User Centered Approach for Learning Analytics Dashboard Generation

267