PANDA

A Platform for Open Learning Analytics

Marcus Hamann

, Christian Saul

and Heinz-Dietrich Wuttke

Ilmenau University of Technology, Integrated Communication Systems Group, Ilmenau, Germany

Fraunhofer Institute for Digital Media Technology,

Technology-enhanced Learning and Training Group, Ilmenau, Germany

Keywords:

Open Learning Analytics, PANDA, xAPI, LRS, Personalization.

Abstract:

Learning Analytics (LA) has emerged as a signiﬁcant area of research in the ﬁeld of technology-enhanced

learning. It automatically analyzes educational data in order to enhance students learning experience and to

foster their learning. Open learning analytics (OLA) extends this ﬁeld in that it integrates data from distributed

and heterogeneous sources, serves different stakeholders with very diverse interests and needs, and leverages

a variety of statistical, visual and computational tools, methods and methodologies. This paper presents an

OLA platform called PANDA that is currently being developed as part of a German research and development

project. The platform allows different learning systems to publish data about learners and their contexts, and

applies different methods and techniques for information visualization and discovery to analyze the collected

data and to detect interesting patterns within.

1 INTRODUCTION

Over the last decade, Learning Analytics (LA) has

emerged as a signiﬁcant area of research in the ﬁeld

of technology-enhanced learning. It has been consid-

ered as one of the fastest growing areas of research

related to education and technology (Broadfoot et al.,

2012). According to the 1st International Conference

on Learning Analytics and Knowledge (LAK’11), LA

is deﬁned as the ”measurement, collection, analysis

and reporting of data about learners and their con-

texts, for purposes of understanding and optimizing

learning and the environments in which it occurs”. It

was recognized very early that the ﬁeld of LA offers

promising possibilities for education and assessment.

LA provides a variety of information that can be used

to support monitoring and analysis, prediction and in-

tervention, tutoring and mentoring, adaptation, per-

sonalization and recommendation as well as aware-

ness and reﬂection (Chatti et al., 2014).

Even though LA has been cited in the latest NMC

Horizon Report (2014 Higher Education Edition) as

an important development with a time-to-adoption

horizon of one year and less, a great deal of research is

still required in this area. A particularly rich area for

future research is Open Learning Analytics (OLA)

http://solaresearch.org/initiatives/ola/

OLA extends the ﬁeld of LA in that it integrates data

from distributed and heterogeneous sources, serves

different stakeholders with very diverse interests and

needs, and supports a variety of statistical, visual

and computational tools, methods and methodologies.

Following up on this, this paper proposes PANDA, an

open learning analytics platform that is currently be-

ing developed as part of a German research and de-

velopment project with the same name.

The remainder of the paper is organized as fol-

lows: The second chapter gives an insight into the

project and its goals. The third chapter describes

our proposed solution. Following up on this, chapter

four gives an insight into the collecting and utiliza-

tion of information about learning experience during

using the PANDA environment. Finally, concluding

remarks and references complete the paper.

2 THE PANDA PROJECT

PANDA is a two-year research and development

project funded by the German government (Federal

Ministry for Economic Affairs and Energy). Part-

ners involved in the project are Ilmenau University of

Technology, the Fraunhofer Institute for Digital Me-

dia Technology (IDMT) and the company Magh &

467

Hamann M., Saul C. and Wuttke H..

PANDA - A Platform for Open Learning Analytics.

DOI: 10.5220/0005489804670473

In Proceedings of the 7th International Conference on Computer Supported Education (CSEDU-2015), pages 467-473

ISBN: 978-989-758-107-6

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

Boppert. Each of the partners brings in an own learn-

ing system (Moodle, askMe! and Avendoo) with own

data and different stakeholders with diverse interests

and needs.

The main goal of the project is to develop a cen-

tralized platform that allows different learning sys-

tems to publish data about learners and their con-

texts anonymously with the beneﬁt of getting useful

information to foster learning in return. The PANDA

platform applies different LA methods and techniques

to analyze the collected data and to detect interest-

ing patterns within. This includes on one hand meth-

ods and techniques that provide statistics in forms of

reports and tables, visual representation in forms of

charts, maps, etc., but on the other hand methods and

techniques that extract information from a data set

using methods from artiﬁcial intelligence, machine

learning, data mining, etc.

The information is then made available to the

different systems via well-deﬁned interfaces (APIs).

They are not only used to support tutoring and men-

toring, but also allows the systems involved to ad-

dress the individual user more speciﬁcally (personal-

ization).

3 PROPOSED SOLUTION

The proposed PANDA platform is depicted in Figure

1. Basically, it consists of two components: a compo-

nent that stores students’ learning experiences and a

second component that analyzes this information and

applies LA methods and techniques. The (learning)

data storage in the proposed platform is done using

a Learning Record Store (LRS). It allows storing and

retrieving learning experiences using the xAPI

. Be-

side the actual platform, there are different systems

connected to it namely Avendoo

, Moodle

, askMe!

and RemoteLab. From students point of view, they

form a uniﬁed environment for learning (Avendoo,

Moodle) and knowledge testing (askMe!, Remotelab)

by providing course materials and interactive content

objects (ICOs) like simulations and remote experi-

ments.

The systems beneﬁt from the PANDA platform

in that it serves as connection point for the commu-

nication of learning experience. The modular de-

sign of the platform together with the usage of well-

deﬁned interfaces (xAPI) increases the compatibil-

ity with other systems and supports its prompt and

http://www.adlnet.gov/tla/experience-api/

http://www.avendoo.de/

https://moodle.org/

http://www.idmt.fraunhofer.de/askme

widespread adoption. The following subsections de-

scribe the LRS and the LA component in more detail.

3.1 Learning Record Store

The Learning Record Store plays a signiﬁcant role in

the PANDA platform because it stores information

about students’ learning activities performed in

different systems. It is a type of data repository

designed to store learning experiences in so called

statements generated by xAPI-compliant systems or

reporting tools. The interface to the LRS is realized

by the xAPI speciﬁcation. xAPI is a communication

mechanism based on activity streams that facilitates

and integrates all types of learning and training and

makes it possible to collect data about the wide range

of experiences a person has (online and ofﬂine). The

experiences are expressed as statements. Statements

are the core of xAPI. In general, a statement has the

following structure:

The actor is represented by the student identiﬁed

only by an ID. Verbs can be information about the us-

age of the system like ”viewed”, ”passed” or ”failed”.

The object, ﬁnally, addresses the element which is

handled by the actor e.g. ”Moodle chapter X”, ”ques-

tion Y” or ”ICO Z”. These statements are human

readable, meaningful if printed on an interface and

also machine readable. By means of this structure, we

can easily classify users by different verbs or objects

in order to provide a high degree of adaptivity and

personalization for our learning management system.

In the context of the PANDA project, the LRS

stores the whole amount of information about the ac-

tivities of the user such as the number of successfully

ﬁnished tests or the amount of viewed learning ma-

terial. By splitting the information into small infor-

mation units these data are hidden for unauthorized

persons because of their missing context.

3.2 Learning Analytics

The LA component plays a signiﬁcant role in the

PANDA platform because it analyzes data stored in

the LRS and tries to discover patterns within. Then,

these ﬁndings can be used by other systems to support

tutoring, mentoring, adaptation, personalization, etc.

Mainly four groups of techniques have received par-

ticular attention in the LA literature in the last couple

of years (Chatti et al., 2014):

1. Statistics: This group includes all methods and

CSEDU2015-7thInternationalConferenceonComputerSupportedEducation

468

Avendoo

Moodle

askMe!

RemoteLab

LRS

The PANDA platform

Learning

Analytics

xAPI

Provision of

statements, states,

activities, etc.

Storage of

statements, states,

activities, etc.

Provision of

statistics, visuali-

zations, etc.

Provision of

adaptation

information

Data

Figure 1: Overview of the PANDA platform.

techniques that provide statistics in forms of re-

ports and tables.

2. Information visualization: This group includes all

methods and techniques that provide visual repre-

sentation in forms of charts, scatter-plots, maps,

etc.

3. Data mining: This group includes all methods and

techniques that discover useful patterns or knowl-

edge from data sources.

4. Social network analysis: This group includes all

methods and techniques that analyze the relation-

ships between individuals or organizations.

The methods and techniques, the LA component

applies, can mainly be assigned to the ﬁrst three

groups. For statistics and information visualization,

the LA component uses the data stored in the LRS

and provides them in a clear and understandable for-

mat. These visualizations (see Figure 2) can then be

integrated on the own website or learning platform

using e.g. a communication technique like JSONP

Statistics and visualizations can be both descriptive

and predictive. Descriptive means that they are in ac-

cordance with the facts and without any interpreta-

tion or assessment. Exemplary: Student A reached

5 out of 10 points. In contrast, predictive methods

try to make statements about facts or the occurrence

of events with a given probability. Exemplary: It is

very likely that student B is not going to pass the ﬁ-

nal test successfully. In order to predict these future

events, prediction models (mainly classiﬁcation meth-

ods (Ming and Ming, 2012)) are going to be used.

In addition, clustering algorithms are intended to be

used to group students and student actions as well as

sequential pattern mining to identify whether an event

(or any observed construct) was the cause of another

event (or observed construct) (Baker and Siemens,

2011).

http://www.json-p.org/

The application of these methods using the data

from the LRS will be used to improve both the learn-

ing and teaching methods. On the one hand we can

use the information to support the user (e.g., give im-

mediate feedback to wrong answers, links to relevant

lectures, etc.), while on the other hand the teacher

proﬁts by the learning process that each user has

passed. In that way we can accompany the user in

the learning process in order to recognize user weak-

nesses and to motivate the user. The beneﬁt for the

teacher is mainly feedback of users weaknesses and

learning material.

3.3 Data Privacy

A big concern of learning analytics is data privacy.

However, it might not be in the interest of the student

to disclose collected and stored personally identiﬁ-

able information or other sensitive information. For

that reason, the PANDA platform uses different ap-

5 / 12

tests

Your learning progress

(in comparison to others)

You completed 3 out of 7 tests

Your score per test

(in comparison to others)

Risk for a successful

completion

Figure 2: PANDA visualizations.

PANDA-APlatformforOpenLearningAnalytics

469

proaches to achieve a high level of anonymity. The

most important precaution for protection of data pri-

vacy is the LRS. With the help of the LRS, we col-

lect data anonymously and centralized outside of the

learning environments (Avendoo, Moodle, askMe!,

RemoteLab). In that way, each user is represented

by an ID on an external server that provides the LRS.

Moreover, the learning environments are not able

to exchange data about the user. Rather, each sys-

tem collects special information about each user and

transfer it to the LRS independently.

In addition to the above mentioned aspects the

central data storage offers another advantage. From

the anonymous data we can build user groups which

can be evaluated separately. This user group spe-

ciﬁc evaluation, for instance according to skill level of

each user, can be used to reach a high level of adaptiv-

ity and personalization of the whole system. The big

advantage is that we do not have to poll each subsys-

tem to receive all relevant information about one user

or user group but that we can obtain all pertinent data

from the LRS.

4 LEARNING

EXPERIENCE - COLLECTING

AND UTILIZATION DATA

In order to make the PANDA platform to work, in par-

ticular the LA component, information about learn-

ers interactions (i.e., learning experiences) need to be

gathered from different systems. In the context of this

project, the systems that mainly provide information

about learning experiences are the Moodle learning

platform, the remote laboratory hosted at the Ilmenau

University of Technology and the Avendoo learning

platform. In contrast, the system that take advantage

of the PANDA platform (and its data) are mainly the

askMe! system. In the following, these systems are

described in more detail.

4.1 Moodle

Moodle is an open source learning platform originally

developed by Martin Dougiamas. Today, the Moodle

project is supported by a big community and is grow-

ing continuously. It combines a lot of modules such

as ”books”, ”forums”, ”quizzes” and many more in

order to support learning activities. In addition, it is

highly adaptable in terms of including own plugins.

These facts make Moodle to a valuable learn-

ing platform and an important part of the PANDA

project. In the context of the PANDA project, Moodle

Figure 3: Moodle system.

is mainly used to provide learning content for the user.

It allows structuring lectures and further information

very generic. We are able to include external links

as well as ICOs and the user can decide for himself

which content is relevant.

Most advantageous for the PANDA project is the

customizability of Moodle. Therefore, we have estab-

lished an interface to the LRS. Thus, we can record

the learning process of each user. After recording

of data we can analyze users in order to provide

adaptation information which in turn can be used by

the Moodle system to improve the learning process.

Thus, we can e.g. hide all other parts of the learning

path to assist the user with a speciﬁc problem.

In summary, that implies, if a user acquired

knowledge about one part of a lecture with the help of

Moodle content and ICOs and passed a test about the

knowledge, this information is recorded in the LRS

without deeper knowledge about the user.

One excerpt of a course material inside the Moo-

dle system, more precisely, an ICO is depicted in Fig-

ure 3. With the help of this ICO the student should

CSEDU2015-7thInternationalConferenceonComputerSupportedEducation

470

Figure 4: Remote Lab of Ilmenau University of Technology.

get deeper knowledge about Boolean set algebra. For

that a random Venn diagram, representing sets of in-

put combinations, will be presented to the student and

he or she has to ﬁnd a correct truth table matching the

Venn diagram. During solving this problem the stu-

dent can compare his or her solution with the correct

one. This information will be sent to the LRS in or-

der to support the student with the help of Learning

Analytics.

4.2 Remote Laboratory

In general, a remote laboratory uses telecommunica-

tions to remotely conduct real experiments at a phys-

ical location whilst the scientist is utilizing this tech-

nology from a separate location. With the Remote

Lab, called GOLDi (Grid of Online Lab Devices Il-

menau), the University of Technology wants to offer

to students a working environment that is as close as

possible to a real world laboratory.

The remote laboratory of Ilmenau University of

Technology (Henke et al., 2012; Henke et al., 2013;

Henke et al., 2014) allows students to design, verify

and implement digital circuits and control systems.

The lab consists of different programmable control

units (embedded systems or FPGAs) and physical

systems (e.g., an elevator, a conveyor or a high rack

warehouse) that provide real-time experiments with

real hardware equipment or simulations to students.

All of these experiments empower students to solve

complex design tasks, makes the learning environ-

ment very powerful and allows a very effective learn-

ing process.

Under real laboratory conditions disturbances can

appear and lead to failures of the control algorithm

that cannot be detected under virtual lab conditions.

While the student is interacting with the Remote

Lab, it is able to generate fault messages in case of

faulty inputs to the LRS which can be used by meth-

ods of Learning Analytics in order to provide im-

provements of the learning process. Figure 4 shows

the graphical user interface of the Remote Lab in

more detail.

4.3 askMe!

askMe! is a web-based e-assessment system being

developed at the Fraunhofer IDMT that covers the

whole life-cycle of e-assessments starting from cre-

ating questions, presenting them to the students up to

preparing the results and presenting them to teachers,

tutors, etc. The questions and tests can consider in-

dividual aspects so that e-assessments and their feed-

back can perfectly be tailored to students or groups of

students (Saul and Wuttke, 2013). Moreover, the au-

thor of the adaptive tests is not limited to traditional

question types such as multiple-choice, but can use

Interactive Content Objects (ICOs) to create sophis-

ticated (interactive) e-assessments (Saul and Wuttke,

2012). The latter aspect takes into account the as-

PANDA-APlatformforOpenLearningAnalytics

471

Figure 5: Knowledge dashboard of the askMe! system.

sumption that learning is the result of interaction and

more speciﬁcally, the result of engagement with the

subject matter. In order to deal with the different

ICOs located elsewhere in the Web, a communication

mechanism based on the xAPI speciﬁcation has been

developed.

In the context of the PANDA project, the LA as-

pect of the askMe! system is being further devel-

oped. Currently, the system focuses on information

visualizations that are provided to both students and

authors (Saul and Wuttke, 2014). When a student

has completed a test, he/she will not be confronted

with an abstract score, but will get detailed feedback

on his/her strengths and weaknesses, which allows

him/her to efﬁciently address speciﬁc deﬁcits after-

ward. This information is presented in his/her knowl-

edge dashboard (cf. Figure 5). This component not

only provides students with a tabular and graphical

overview of his/her testing results, but also with a de-

tailed overview about his/her knowledge level accord-

ing to the topics addressed by the respective test. In

this way, the askMe! system is able to increase stu-

dents self-awareness signiﬁcantly. However, the pre-

sentation of statistics in askMe! is not limited to sup-

port students, but is also provided to authors. The in-

formation for this user group is presented in user and

test statistics. This component presents an overview

of students results in (adaptive or non-adaptive) tests

as well as their individual learning progress.

These information visualizations in forms of

charts and tables are now being expanded and sup-

ported with the help of PANDA. This includes the

use of prediction methods in order to predict, which

students are struggling with the content. Tutors and

teachers can use this information (e.g., visualized as

trafﬁc light) to intervene either online or face-to-face.

Other methods including heat maps, bar charts and

line graphs show students’ activities and testing re-

sults over time and support teachers in their decision-

making. The basis of all these methods are students’

interactions with the askMe! system, which are com-

municated to the LRS and processed by LA.

4.4 Avendoo

Avendoo is a learning platform developed by the com-

pany Magh & Boppert. In the frame of the PANDA

project, Avendoo will not only provide information

about students’ learning activities, but will also use

this information (together with information obtained

from other systems) to adapt the learning courses

(adaptive navigation support, (Brusilovsky, 2001))

and the system accordingly. For this, the system

will make use of the LA component and the infor-

mation it provides to implement adaptive link hiding

(Brusilovsky, 2004).

5 CONCLUSIONS AND FUTURE

WORK

This paper has proposed an OLA platform called

PANDA. It supports both learning experience trac-

ing and learning experience utilization. The learning

experience tracing is realized by the Moodle learn-

CSEDU2015-7thInternationalConferenceonComputerSupportedEducation

472

ing platform and the Remote Laboratory. The objec-

tive here is to gather information about learners in-

teractions such as learning experience. This part of

the PANDA platform serves as input for the follow-

ing learning experience utilization in order to process

all relevant data for information visualization and dis-

covery to ﬁnd patterns in the learning process. The

learning experience utilization is mainly realized by

the askMe! system and the Avendoo learning plat-

form.

Future work includes the (full) implementation of

the LA component. This allows us to obtain valu-

able information from the PANDA platform and to

use them for adaptation, personalization or individ-

ual support of the learner (e.g., to give better hints).

Moreover, in order to handle the huge amount of data

due to the number of users, an authoring tool is cur-

rently being developed. In addition, to cover also the

higher order thinking skills while learning we intend

to integrate the Remote Laboratory into the learning

environment. Furthermore, a major challenge is to

test our system with other learning content as well as

other target groups.

Finally, it can be stated that the PANDA platform

reﬂects the fact that learning takes place everywhere:

in traditional education settings (e.g., LMS) as well

as in more open-ended and less formal learning set-

tings (e.g., PLEs, MOOCs). It has the potential to

deal with the challenges in increasingly complex and

fast-changing learning environments.

ACKNOWLEDGEMENTS

The research presented in this paper has been ﬁ-

nancially supported by the German Federal Min-

istry for Economic Affairs and Energy (BMWi)

within the project ”Entwicklung einer Software

ur die Personalisierung von Lernprozessen durch

Adaptivit

at, Nutzermodellierung und Datenanalyse

der Lerner-Aktionen (PANDA)” under contract no.

KF2329704KM3 and KF2250116KM3.

REFERENCES

Baker, R. and Siemens, G. (2011). Educational Data Min-

ing and Learning Analytics. In Cambridge Handbook

of the Learning Sciences, page 19. Cambridge Univer-

sity Press, Cambridge.

Broadfoot, P., Timmis, S., Payton, S., Oldﬁeld, A., and

Sutherland, R. (2012). Rethinking Assessment -

Learning analytics and technology enhanced assess-

ment (TEA). Technical report, University of Bristol.

Brusilovsky, P. (2001). Adaptive Hypermedia. User Mod-

eling and User-Adapted Interaction, 11(1):87–110.

Brusilovsky, P. (2004). Adaptive Navigation Support: From

Adaptive Hypermedia to the Adaptive Web and Be-

yond. PsychNology Journal, 2(1):7–23.

Chatti, A., Lukarov, V., Th

us, H., Muslim, A., Yousef,

F., Wahid, U., Greven, C., Chakrabarti, A., and

Schroeder, U. (2014). Learning Analytics: Challenges

and Future Research Directions. eleed, 10(1).

Henke, K., Ostendorff, S., Vogel, S., and Wuttke, H.-D.

(2012). A Grid Concept for Reliable, Flexible and Ro-

bust Remote Engineering Laboratories. International

Journal of Online Engineering (iJOE), 8:42–49.

Henke, K., Ostendorff, S., Wuttke, H.-D., Vietzke, T., and

Lutze, C. (2013). Fields of Applications for Hybrid

Online Labs. International Journal of Online Engi-

neering (iJOE), 9:20–30.

Henke, K., Wuttke, H.-D., Vietzke, T., and Ostendorff, S.

(2014). Using Interactive Hybrid Online Labs for

Rapid Prototyping of Digital Systems. 11th Interna-

tional Conference on Remote Engineering and Virtual

Instrumentation (REV), 11:57–63.

Ming, N. and Ming, V. (2012). Predicting student outcomes

from unstructured data. CEUR Workshop Proceed-

ings, 872:11–16.

Saul, C. and Wuttke, H.-D. (2012). Towards a high-level

Integration of interactive Tools with E-Assessments.

In Proceedings of the 12th IEEE International Con-

ference on Advanced Learning Technologies (ICALT

2012), pages 597–598, Rome.

Saul, C. and Wuttke, H.-D. (2013). E-Assessment meets

Personalization. In Proceedings of the IEEE Global

Engineering Education Conference (EDUCON 2013),

pages 200–206, Berlin.

Saul, C. and Wuttke, H.-D. (2014). Turning Learners into

effective better Learners: The Use of the askMe! Sys-

tem for Learning Analytics. In Extended Proceedings

of the 22th International Conference on User Mod-

eling, Adaptation and Personalization (UMAP 2014),

pages 57–60, Aalborg. CEUR-WS.

PANDA-APlatformforOpenLearningAnalytics

473