PANDA
A Platform for Open Learning Analytics
Marcus Hamann
1
, Christian Saul
2
and Heinz-Dietrich Wuttke
1
1
Ilmenau University of Technology, Integrated Communication Systems Group, Ilmenau, Germany
2
Fraunhofer Institute for Digital Media Technology,
Technology-enhanced Learning and Training Group, Ilmenau, Germany
Keywords:
Open Learning Analytics, PANDA, xAPI, LRS, Personalization.
Abstract:
Learning Analytics (LA) has emerged as a significant area of research in the field of technology-enhanced
learning. It automatically analyzes educational data in order to enhance students learning experience and to
foster their learning. Open learning analytics (OLA) extends this field in that it integrates data from distributed
and heterogeneous sources, serves different stakeholders with very diverse interests and needs, and leverages
a variety of statistical, visual and computational tools, methods and methodologies. This paper presents an
OLA platform called PANDA that is currently being developed as part of a German research and development
project. The platform allows different learning systems to publish data about learners and their contexts, and
applies different methods and techniques for information visualization and discovery to analyze the collected
data and to detect interesting patterns within.
1 INTRODUCTION
Over the last decade, Learning Analytics (LA) has
emerged as a significant area of research in the field
of technology-enhanced learning. It has been consid-
ered as one of the fastest growing areas of research
related to education and technology (Broadfoot et al.,
2012). According to the 1st International Conference
on Learning Analytics and Knowledge (LAK’11), LA
is defined as the ”measurement, collection, analysis
and reporting of data about learners and their con-
texts, for purposes of understanding and optimizing
learning and the environments in which it occurs”. It
was recognized very early that the field of LA offers
promising possibilities for education and assessment.
LA provides a variety of information that can be used
to support monitoring and analysis, prediction and in-
tervention, tutoring and mentoring, adaptation, per-
sonalization and recommendation as well as aware-
ness and reflection (Chatti et al., 2014).
Even though LA has been cited in the latest NMC
Horizon Report (2014 Higher Education Edition) as
an important development with a time-to-adoption
horizon of one year and less, a great deal of research is
still required in this area. A particularly rich area for
future research is Open Learning Analytics (OLA)
1
.
1
http://solaresearch.org/initiatives/ola/
OLA extends the field of LA in that it integrates data
from distributed and heterogeneous sources, serves
different stakeholders with very diverse interests and
needs, and supports a variety of statistical, visual
and computational tools, methods and methodologies.
Following up on this, this paper proposes PANDA, an
open learning analytics platform that is currently be-
ing developed as part of a German research and de-
velopment project with the same name.
The remainder of the paper is organized as fol-
lows: The second chapter gives an insight into the
project and its goals. The third chapter describes
our proposed solution. Following up on this, chapter
four gives an insight into the collecting and utiliza-
tion of information about learning experience during
using the PANDA environment. Finally, concluding
remarks and references complete the paper.
2 THE PANDA PROJECT
PANDA is a two-year research and development
project funded by the German government (Federal
Ministry for Economic Affairs and Energy). Part-
ners involved in the project are Ilmenau University of
Technology, the Fraunhofer Institute for Digital Me-
dia Technology (IDMT) and the company Magh &
467
Hamann M., Saul C. and Wuttke H..
PANDA - A Platform for Open Learning Analytics.
DOI: 10.5220/0005489804670473
In Proceedings of the 7th International Conference on Computer Supported Education (CSEDU-2015), pages 467-473
ISBN: 978-989-758-107-6
Copyright
c
2015 SCITEPRESS (Science and Technology Publications, Lda.)
Boppert. Each of the partners brings in an own learn-
ing system (Moodle, askMe! and Avendoo) with own
data and different stakeholders with diverse interests
and needs.
The main goal of the project is to develop a cen-
tralized platform that allows different learning sys-
tems to publish data about learners and their con-
texts anonymously with the benefit of getting useful
information to foster learning in return. The PANDA
platform applies different LA methods and techniques
to analyze the collected data and to detect interest-
ing patterns within. This includes on one hand meth-
ods and techniques that provide statistics in forms of
reports and tables, visual representation in forms of
charts, maps, etc., but on the other hand methods and
techniques that extract information from a data set
using methods from artificial intelligence, machine
learning, data mining, etc.
The information is then made available to the
different systems via well-defined interfaces (APIs).
They are not only used to support tutoring and men-
toring, but also allows the systems involved to ad-
dress the individual user more specifically (personal-
ization).
3 PROPOSED SOLUTION
The proposed PANDA platform is depicted in Figure
1. Basically, it consists of two components: a compo-
nent that stores students’ learning experiences and a
second component that analyzes this information and
applies LA methods and techniques. The (learning)
data storage in the proposed platform is done using
a Learning Record Store (LRS). It allows storing and
retrieving learning experiences using the xAPI
2
. Be-
side the actual platform, there are different systems
connected to it namely Avendoo
3
, Moodle
4
, askMe!
5
and RemoteLab. From students point of view, they
form a unified environment for learning (Avendoo,
Moodle) and knowledge testing (askMe!, Remotelab)
by providing course materials and interactive content
objects (ICOs) like simulations and remote experi-
ments.
The systems benefit from the PANDA platform
in that it serves as connection point for the commu-
nication of learning experience. The modular de-
sign of the platform together with the usage of well-
defined interfaces (xAPI) increases the compatibil-
ity with other systems and supports its prompt and
2
http://www.adlnet.gov/tla/experience-api/
3
http://www.avendoo.de/
4
https://moodle.org/
5
http://www.idmt.fraunhofer.de/askme
widespread adoption. The following subsections de-
scribe the LRS and the LA component in more detail.
3.1 Learning Record Store
The Learning Record Store plays a significant role in
the PANDA platform because it stores information
about students’ learning activities performed in
different systems. It is a type of data repository
designed to store learning experiences in so called
statements generated by xAPI-compliant systems or
reporting tools. The interface to the LRS is realized
by the xAPI specification. xAPI is a communication
mechanism based on activity streams that facilitates
and integrates all types of learning and training and
makes it possible to collect data about the wide range
of experiences a person has (online and offline). The
experiences are expressed as statements. Statements
are the core of xAPI. In general, a statement has the
following structure:
<actor (student)><verb><object>with <result>in <context>
The actor is represented by the student identified
only by an ID. Verbs can be information about the us-
age of the system like ”viewed”, ”passed” or ”failed”.
The object, finally, addresses the element which is
handled by the actor e.g. ”Moodle chapter X”, ”ques-
tion Y” or ”ICO Z”. These statements are human
readable, meaningful if printed on an interface and
also machine readable. By means of this structure, we
can easily classify users by different verbs or objects
in order to provide a high degree of adaptivity and
personalization for our learning management system.
In the context of the PANDA project, the LRS
stores the whole amount of information about the ac-
tivities of the user such as the number of successfully
finished tests or the amount of viewed learning ma-
terial. By splitting the information into small infor-
mation units these data are hidden for unauthorized
persons because of their missing context.
3.2 Learning Analytics
The LA component plays a significant role in the
PANDA platform because it analyzes data stored in
the LRS and tries to discover patterns within. Then,
these findings can be used by other systems to support
tutoring, mentoring, adaptation, personalization, etc.
Mainly four groups of techniques have received par-
ticular attention in the LA literature in the last couple
of years (Chatti et al., 2014):
1. Statistics: This group includes all methods and
CSEDU2015-7thInternationalConferenceonComputerSupportedEducation
468
Avendoo
Moodle
askMe!
RemoteLab
LRS
The PANDA platform
Learning
Analytics
xAPI
Provision of
statements, states,
activities, etc.
Storage of
statements, states,
activities, etc.
Provision of
statistics, visuali-
zations, etc.
Provision of
adaptation
information
Data
Data
Figure 1: Overview of the PANDA platform.
techniques that provide statistics in forms of re-
ports and tables.
2. Information visualization: This group includes all
methods and techniques that provide visual repre-
sentation in forms of charts, scatter-plots, maps,
etc.
3. Data mining: This group includes all methods and
techniques that discover useful patterns or knowl-
edge from data sources.
4. Social network analysis: This group includes all
methods and techniques that analyze the relation-
ships between individuals or organizations.
The methods and techniques, the LA component
applies, can mainly be assigned to the first three
groups. For statistics and information visualization,
the LA component uses the data stored in the LRS
and provides them in a clear and understandable for-
mat. These visualizations (see Figure 2) can then be
integrated on the own website or learning platform
using e.g. a communication technique like JSONP
6
.
Statistics and visualizations can be both descriptive
and predictive. Descriptive means that they are in ac-
cordance with the facts and without any interpreta-
tion or assessment. Exemplary: Student A reached
5 out of 10 points. In contrast, predictive methods
try to make statements about facts or the occurrence
of events with a given probability. Exemplary: It is
very likely that student B is not going to pass the fi-
nal test successfully. In order to predict these future
events, prediction models (mainly classification meth-
ods (Ming and Ming, 2012)) are going to be used.
In addition, clustering algorithms are intended to be
used to group students and student actions as well as
sequential pattern mining to identify whether an event
(or any observed construct) was the cause of another
event (or observed construct) (Baker and Siemens,
2011).
6
http://www.json-p.org/
The application of these methods using the data
from the LRS will be used to improve both the learn-
ing and teaching methods. On the one hand we can
use the information to support the user (e.g., give im-
mediate feedback to wrong answers, links to relevant
lectures, etc.), while on the other hand the teacher
profits by the learning process that each user has
passed. In that way we can accompany the user in
the learning process in order to recognize user weak-
nesses and to motivate the user. The benefit for the
teacher is mainly feedback of users weaknesses and
learning material.
3.3 Data Privacy
A big concern of learning analytics is data privacy.
However, it might not be in the interest of the student
to disclose collected and stored personally identifi-
able information or other sensitive information. For
that reason, the PANDA platform uses different ap-
5 / 12
tests
Your learning progress
(in comparison to others)
You completed 3 out of 7 tests
Your score per test
(in comparison to others)
Risk for a successful
completion
Figure 2: PANDA visualizations.
PANDA-APlatformforOpenLearningAnalytics
469
proaches to achieve a high level of anonymity. The
most important precaution for protection of data pri-
vacy is the LRS. With the help of the LRS, we col-
lect data anonymously and centralized outside of the
learning environments (Avendoo, Moodle, askMe!,
RemoteLab). In that way, each user is represented
by an ID on an external server that provides the LRS.
Moreover, the learning environments are not able
to exchange data about the user. Rather, each sys-
tem collects special information about each user and
transfer it to the LRS independently.
In addition to the above mentioned aspects the
central data storage offers another advantage. From
the anonymous data we can build user groups which
can be evaluated separately. This user group spe-
cific evaluation, for instance according to skill level of
each user, can be used to reach a high level of adaptiv-
ity and personalization of the whole system. The big
advantage is that we do not have to poll each subsys-
tem to receive all relevant information about one user
or user group but that we can obtain all pertinent data
from the LRS.
4 LEARNING
EXPERIENCE - COLLECTING
AND UTILIZATION DATA
In order to make the PANDA platform to work, in par-
ticular the LA component, information about learn-
ers interactions (i.e., learning experiences) need to be
gathered from different systems. In the context of this
project, the systems that mainly provide information
about learning experiences are the Moodle learning
platform, the remote laboratory hosted at the Ilmenau
University of Technology and the Avendoo learning
platform. In contrast, the system that take advantage
of the PANDA platform (and its data) are mainly the
askMe! system. In the following, these systems are
described in more detail.
4.1 Moodle
Moodle is an open source learning platform originally
developed by Martin Dougiamas. Today, the Moodle
project is supported by a big community and is grow-
ing continuously. It combines a lot of modules such
as ”books”, ”forums”, ”quizzes” and many more in
order to support learning activities. In addition, it is
highly adaptable in terms of including own plugins.
These facts make Moodle to a valuable learn-
ing platform and an important part of the PANDA
project. In the context of the PANDA project, Moodle
Figure 3: Moodle system.
is mainly used to provide learning content for the user.
It allows structuring lectures and further information
very generic. We are able to include external links
as well as ICOs and the user can decide for himself
which content is relevant.
Most advantageous for the PANDA project is the
customizability of Moodle. Therefore, we have estab-
lished an interface to the LRS. Thus, we can record
the learning process of each user. After recording
of data we can analyze users in order to provide
adaptation information which in turn can be used by
the Moodle system to improve the learning process.
Thus, we can e.g. hide all other parts of the learning
path to assist the user with a specific problem.
In summary, that implies, if a user acquired
knowledge about one part of a lecture with the help of
Moodle content and ICOs and passed a test about the
knowledge, this information is recorded in the LRS
without deeper knowledge about the user.
One excerpt of a course material inside the Moo-
dle system, more precisely, an ICO is depicted in Fig-
ure 3. With the help of this ICO the student should
CSEDU2015-7thInternationalConferenceonComputerSupportedEducation
470
Figure 4: Remote Lab of Ilmenau University of Technology.
get deeper knowledge about Boolean set algebra. For
that a random Venn diagram, representing sets of in-
put combinations, will be presented to the student and
he or she has to find a correct truth table matching the
Venn diagram. During solving this problem the stu-
dent can compare his or her solution with the correct
one. This information will be sent to the LRS in or-
der to support the student with the help of Learning
Analytics.
4.2 Remote Laboratory
In general, a remote laboratory uses telecommunica-
tions to remotely conduct real experiments at a phys-
ical location whilst the scientist is utilizing this tech-
nology from a separate location. With the Remote
Lab, called GOLDi (Grid of Online Lab Devices Il-
menau), the University of Technology wants to offer
to students a working environment that is as close as
possible to a real world laboratory.
The remote laboratory of Ilmenau University of
Technology (Henke et al., 2012; Henke et al., 2013;
Henke et al., 2014) allows students to design, verify
and implement digital circuits and control systems.
The lab consists of different programmable control
units (embedded systems or FPGAs) and physical
systems (e.g., an elevator, a conveyor or a high rack
warehouse) that provide real-time experiments with
real hardware equipment or simulations to students.
All of these experiments empower students to solve
complex design tasks, makes the learning environ-
ment very powerful and allows a very effective learn-
ing process.
Under real laboratory conditions disturbances can
appear and lead to failures of the control algorithm
that cannot be detected under virtual lab conditions.
While the student is interacting with the Remote
Lab, it is able to generate fault messages in case of
faulty inputs to the LRS which can be used by meth-
ods of Learning Analytics in order to provide im-
provements of the learning process. Figure 4 shows
the graphical user interface of the Remote Lab in
more detail.
4.3 askMe!
askMe! is a web-based e-assessment system being
developed at the Fraunhofer IDMT that covers the
whole life-cycle of e-assessments starting from cre-
ating questions, presenting them to the students up to
preparing the results and presenting them to teachers,
tutors, etc. The questions and tests can consider in-
dividual aspects so that e-assessments and their feed-
back can perfectly be tailored to students or groups of
students (Saul and Wuttke, 2013). Moreover, the au-
thor of the adaptive tests is not limited to traditional
question types such as multiple-choice, but can use
Interactive Content Objects (ICOs) to create sophis-
ticated (interactive) e-assessments (Saul and Wuttke,
2012). The latter aspect takes into account the as-
PANDA-APlatformforOpenLearningAnalytics
471
Figure 5: Knowledge dashboard of the askMe! system.
sumption that learning is the result of interaction and
more specifically, the result of engagement with the
subject matter. In order to deal with the different
ICOs located elsewhere in the Web, a communication
mechanism based on the xAPI specification has been
developed.
In the context of the PANDA project, the LA as-
pect of the askMe! system is being further devel-
oped. Currently, the system focuses on information
visualizations that are provided to both students and
authors (Saul and Wuttke, 2014). When a student
has completed a test, he/she will not be confronted
with an abstract score, but will get detailed feedback
on his/her strengths and weaknesses, which allows
him/her to efficiently address specific deficits after-
ward. This information is presented in his/her knowl-
edge dashboard (cf. Figure 5). This component not
only provides students with a tabular and graphical
overview of his/her testing results, but also with a de-
tailed overview about his/her knowledge level accord-
ing to the topics addressed by the respective test. In
this way, the askMe! system is able to increase stu-
dents self-awareness significantly. However, the pre-
sentation of statistics in askMe! is not limited to sup-
port students, but is also provided to authors. The in-
formation for this user group is presented in user and
test statistics. This component presents an overview
of students results in (adaptive or non-adaptive) tests
as well as their individual learning progress.
These information visualizations in forms of
charts and tables are now being expanded and sup-
ported with the help of PANDA. This includes the
use of prediction methods in order to predict, which
students are struggling with the content. Tutors and
teachers can use this information (e.g., visualized as
traffic light) to intervene either online or face-to-face.
Other methods including heat maps, bar charts and
line graphs show students’ activities and testing re-
sults over time and support teachers in their decision-
making. The basis of all these methods are students’
interactions with the askMe! system, which are com-
municated to the LRS and processed by LA.
4.4 Avendoo
Avendoo is a learning platform developed by the com-
pany Magh & Boppert. In the frame of the PANDA
project, Avendoo will not only provide information
about students’ learning activities, but will also use
this information (together with information obtained
from other systems) to adapt the learning courses
(adaptive navigation support, (Brusilovsky, 2001))
and the system accordingly. For this, the system
will make use of the LA component and the infor-
mation it provides to implement adaptive link hiding
(Brusilovsky, 2004).
5 CONCLUSIONS AND FUTURE
WORK
This paper has proposed an OLA platform called
PANDA. It supports both learning experience trac-
ing and learning experience utilization. The learning
experience tracing is realized by the Moodle learn-
CSEDU2015-7thInternationalConferenceonComputerSupportedEducation
472
ing platform and the Remote Laboratory. The objec-
tive here is to gather information about learners in-
teractions such as learning experience. This part of
the PANDA platform serves as input for the follow-
ing learning experience utilization in order to process
all relevant data for information visualization and dis-
covery to find patterns in the learning process. The
learning experience utilization is mainly realized by
the askMe! system and the Avendoo learning plat-
form.
Future work includes the (full) implementation of
the LA component. This allows us to obtain valu-
able information from the PANDA platform and to
use them for adaptation, personalization or individ-
ual support of the learner (e.g., to give better hints).
Moreover, in order to handle the huge amount of data
due to the number of users, an authoring tool is cur-
rently being developed. In addition, to cover also the
higher order thinking skills while learning we intend
to integrate the Remote Laboratory into the learning
environment. Furthermore, a major challenge is to
test our system with other learning content as well as
other target groups.
Finally, it can be stated that the PANDA platform
reflects the fact that learning takes place everywhere:
in traditional education settings (e.g., LMS) as well
as in more open-ended and less formal learning set-
tings (e.g., PLEs, MOOCs). It has the potential to
deal with the challenges in increasingly complex and
fast-changing learning environments.
ACKNOWLEDGEMENTS
The research presented in this paper has been fi-
nancially supported by the German Federal Min-
istry for Economic Affairs and Energy (BMWi)
within the project ”Entwicklung einer Software
f
¨
ur die Personalisierung von Lernprozessen durch
Adaptivit
¨
at, Nutzermodellierung und Datenanalyse
der Lerner-Aktionen (PANDA)” under contract no.
KF2329704KM3 and KF2250116KM3.
REFERENCES
Baker, R. and Siemens, G. (2011). Educational Data Min-
ing and Learning Analytics. In Cambridge Handbook
of the Learning Sciences, page 19. Cambridge Univer-
sity Press, Cambridge.
Broadfoot, P., Timmis, S., Payton, S., Oldfield, A., and
Sutherland, R. (2012). Rethinking Assessment -
Learning analytics and technology enhanced assess-
ment (TEA). Technical report, University of Bristol.
Brusilovsky, P. (2001). Adaptive Hypermedia. User Mod-
eling and User-Adapted Interaction, 11(1):87–110.
Brusilovsky, P. (2004). Adaptive Navigation Support: From
Adaptive Hypermedia to the Adaptive Web and Be-
yond. PsychNology Journal, 2(1):7–23.
Chatti, A., Lukarov, V., Th
¨
us, H., Muslim, A., Yousef,
F., Wahid, U., Greven, C., Chakrabarti, A., and
Schroeder, U. (2014). Learning Analytics: Challenges
and Future Research Directions. eleed, 10(1).
Henke, K., Ostendorff, S., Vogel, S., and Wuttke, H.-D.
(2012). A Grid Concept for Reliable, Flexible and Ro-
bust Remote Engineering Laboratories. International
Journal of Online Engineering (iJOE), 8:42–49.
Henke, K., Ostendorff, S., Wuttke, H.-D., Vietzke, T., and
Lutze, C. (2013). Fields of Applications for Hybrid
Online Labs. International Journal of Online Engi-
neering (iJOE), 9:20–30.
Henke, K., Wuttke, H.-D., Vietzke, T., and Ostendorff, S.
(2014). Using Interactive Hybrid Online Labs for
Rapid Prototyping of Digital Systems. 11th Interna-
tional Conference on Remote Engineering and Virtual
Instrumentation (REV), 11:57–63.
Ming, N. and Ming, V. (2012). Predicting student outcomes
from unstructured data. CEUR Workshop Proceed-
ings, 872:11–16.
Saul, C. and Wuttke, H.-D. (2012). Towards a high-level
Integration of interactive Tools with E-Assessments.
In Proceedings of the 12th IEEE International Con-
ference on Advanced Learning Technologies (ICALT
2012), pages 597–598, Rome.
Saul, C. and Wuttke, H.-D. (2013). E-Assessment meets
Personalization. In Proceedings of the IEEE Global
Engineering Education Conference (EDUCON 2013),
pages 200–206, Berlin.
Saul, C. and Wuttke, H.-D. (2014). Turning Learners into
effective better Learners: The Use of the askMe! Sys-
tem for Learning Analytics. In Extended Proceedings
of the 22th International Conference on User Mod-
eling, Adaptation and Personalization (UMAP 2014),
pages 57–60, Aalborg. CEUR-WS.
PANDA-APlatformforOpenLearningAnalytics
473