Learning Analytics as a Metacognitive Tool

Eva Durall

and Begoña Gros

Department of Media, School of Arts, Design and Architecture, Aalto University, Miestentie 3, Espoo, Finland

Department of Theory and History of Education, University of Barcelona, Barcelona, Spain

Keywords: Learning Analytics, Educational Data Mining, Self-directed Learning, Self-Regulated Learning.

Abstract: The use of learning analytics is entering in the field of research in education as a promising way to support

learning. However, in many cases data are not transparent for the learner. In this regard, Educational

institutions shouldn’t escape the need of making transparent for the learners how their personal data is being

tracked and used in order to build inferences, as well as how its use is going to affect in their learning. In

this contribution, we sustain that learning analytics offers opportunities to the students to reflect about

learning and develop metacognitive skills. Student-centered analytics are highlighted as a useful approach

for reframing learning analytics as a tool for supporting self-directed and self-regulated learning. The article

also provides insights about the design of learning analytics and examples of experiences that challenge

traditional implementations of learning analytics.

1 INTRODUCTION

The use of “big data” tools and methods is a

growing phenomenon in various fields ranging from

computer science, political science, medicine and

economics to physics and social sciences. “Big data”

analytics refers to the process of examining these

large amounts of data to uncover hidden patterns,

unknown correlations and other useful information.

Its rise coincides with new management and

measurement processes in corporations that aim to

develop “Business Intelligence” (BI) by

transforming raw data into meaningful information

that supports more efficient decision-making

processes).

In the education sector, analytics are also

perceived as reliable tools for decision-making, as

well as for achieving greater levels of adaptation and

personalization that are evidence-based (Harmelen

and Workman, 2012). Beyond BI, analytics in

education borrow techniques from different fields,

such as Educational Data Mining (EDM), Social

Network Analysis, web analytics and Information

Visualization in order to come up with tools and

methods that facilitate the exploration of data

coming from educational contexts. According to

Harmelen and Workman, the main potential uses of

analytics in education are (p.5):

● “Identify students at risk so as to provide

positive interventions designed to improve retention.

● Provide recommendations to students in

relation to reading material and learning activities.

● Detect the need for, and measure the results of,

pedagogic improvements.

● Tailor course offerings.

● Identify teachers who are performing well, and

teachers who need assistance with teaching methods.

● Assist in the student recruitment process”.

EDM and Learning Analytics (LA) are two

research areas with strong similarities. Both of them

seek to improve education by focusing on

assessment, the identification of problems and

interventions. The main differences can be found in

EDM’s emphasis on automated discovering and

automated adaptation, whereas LA seeks to inform

and empower instructors and learners in order to

better leverage human judgement (Siemens and

Baker, 2012).

LA research has been applied in two close and

related areas: learning and academia. Although both

of them use educational data, it is important to make

a distinction since the underlying motivation of each

one varies to great extent. According to the Society

for Learning Analytics Research, Learning Analytics

can be defined as “the measurement, collection,

analysis and reporting of data about learners and

their contexts, for purposes of understanding and

optimizing learning and the environments in which it

occurs. Learning analytics are largely concerned

380

Durall E. and Gros B..

Learning Analytics as a Metacognitive Tool.

DOI: 10.5220/0004933203800384

In Proceedings of the 6th International Conference on Computer Supported Education (CSEDU-2014), pages 380-384

ISBN: 978-989-758-020-8

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

with improving learner success” (SoLAR, 2013,

About). On the other hand, academic analytics could

be described as more business-oriented, since the

main purpose is to improve organizational

effectiveness through the use of learner, academic

and institutional data.

Lately, the high proportion of computer-

mediated interactions in learning has created an

interest about how data collected from the

interactions can be used to improve teaching and

learning. In this regard, the increasing offer of

Massive Open Online Courses (MOOCs) can be

presented as a case in which institutions take

advantage of data generated online in order to

achieve a better understanding of how people learn.

Despite its positive aims, LA still poses questions

about how learners can benefit from the data they

are generating in such learning platforms.

2 LEARNING ANALYTICS FOR

FOSTERING SELF-DIRECTED

AND SELF-REGULATED

LEARNING

In the knowledge society, self-directed learning

(SDL) and self-regulated learning (SRL) have

become particularly important for professional

development and lifelong learning.

SDL is an approach in which learners take

control of their own learning processes and

experiences. Tan et al. (2011) describe the processes

of SDL based on a series of requisites or qualities: a)

ownership of learning; b) self-management and self-

monitoring and c) extension of own learning. The

authors argue that providing opportunities to

establish and control one’s own learning objectives,

as well as to direct and monitor the associated

educational tasks, helps increase the subject’s

motivation and commitment to learning.

SRL is a process controlled by learners that may

be supported by social and environmental stimuli

related to setting objectives, self-monitoring

progress, searching for help, feedback, self-directed

reflection, time management and planning, etc.

According to Zimmerman’s (1989) definition, self-

regulation is conditioned by students’ active

involvement in metacognitive, motivational and

cognitive areas, in their own learning processes.

Self-regulation is very much a metacognitive

activity and a useful model to help understand

metacognition. According to Pilling-Cormick and

Garrison. (2007), metacognition goes to the core of

both SDL and SRL and is a link or bridge between

reflective inquiry and strategic task control.

The concepts of SDL and SRL are so similar that

on many occasions they have been used as

synonyms. Furthermore, the models proposed in

both approaches have many elements in common.

Loyens, Magda and Rickens (2008) conducted a

complete analysis of the similarities and differences

between the SDL and SRL models. Both imply

learners’ active involvement and goal-focused

behaviour. In addition, a series of activities can also

be identified as implicit in both models: setting

goals, analysing tasks, implementing the plan and

self-evaluating the learning process. According to

Loyens, Magda and Rickens (2008), the difference

between SDL and SRL basically relates to the

perspective adopted when studying learning

processes, depending on whether attention is focused

on the personal attributes and actions of the learners

and/or on the characteristics of the learning

environment. While SDL encompasses both

perspectives, SRL focuses more on the personal

characteristics and behaviours of the person or

people learning, including the cognitive, behavioural

and also emotional dimensions. One possible

explanation for this difference is the fact that while

SRL has been studied above all in an academic

context, the origins of the SDL concept lie in

studying adult learning in non-formal environments.

In recent years, particular attention has been paid

to the use of technology to support processes of SRL

and SDL. The design of digital environments to

support SDL and SRL processes aims to offer

specific help to learners for planning, organizing and

directing their research and exploration, as well as

for evaluating their own progress. Bartolomé and

Steffens (2011) propose a series of criteria that

technology-enhanced learning environments should

meet in order to support SRL processes: a)

encourage learners to plan their own learning

activity, including aspects linked to time

management (e.g. when to carry out an activity and

how long to spend on it), selecting communication

channels and ways of representing information and

using of the most suitable resources, b) provide

feedback on performance in learning activities to aid

monitoring and the correct direction of the learning

process and, c) provide learners with criteria for

evaluating the results of their learning in terms of the

objectives that were initially set and the type of

competences developed.

In order to successfully self-regulate and self-

direct learning it is necessary that students achieve

an understanding of their own cognitive process.

LearningAnalyticsasaMetacognitiveTool

381

Metacognition, understood as the knowledge of

one’s own thinking, is a central concept in self-

regulation and in self-directed learning since it

brings together central constructs of motivation,

management and monitoring (Pilling-Cormick and

Garrison, 2007). In this regard, Learning Analytics

can be a tool that offers opportunities to reflect about

learning and develop metacognitive skills.

Feedback has been considered as a key tool for

helping students improve performance. Traditional

feedback usually relates to learners’ mechanisms of

communication with their teachers and colleagues.

The use of technology adds new possibilities for

tracking learners’ activity and offers them more

immediate feedback about their learning

performance. However, most efforts to use learning

analytics focus on providing information for the

instructors in order to refine their pedagogical

strategies (Knight et al., 2013). Very rarely are

students considered the main receivers of the

learning analytics data or given the opportunity to

use the information to reflect on their learning

activity and self-regulate their learning more

efficiently.

Despite LA’s potential for improving teaching

and learning, scholars have expressed concerns

regarding the use of analytics in education. The main

criticisms deal with the commercialization of the

education sector, the use of outdated performance

indicators, simplistic uses of artificial intelligence,

as well as the ethics of the datasets and how they are

used (Slade and Prinsloo, 2013). Furthermore, some

authors warn that learning analytics could actually

disempower learners by making them reliant on

institutional feedback (Buckingham and Ferguson,

2011). Quoting Kruse and Pongsajapan, learning

analytics “perpetuates a culture of students as

passive subjects – the targets of a flow of

information – rather than as self-reflective learners

given the cognitive tools to evaluate their own

learning processes” (2012, p.2).

In response to the use of LA as a tool at the

service of teachers and the institution, an increasing

group of scholars have started to advocate for

student-centred analytics (Duval 2012; Clow, 2012,

Kruse and Pongsajapan 2012). In line with these

authors, we consider that learning analytics can and

should be used as a tool for reflection and

metacognition to support SDL and SRL. In this

regard, identifying the main challenges in the design

of learning environments that make use of learning

analytics to foster reflection is a key aspect. From

our perspective, the most urgent challenges to be

faced fall in two directions: data and visualization.

What sort of data is most meaningful for learners?

What types of visualization can foster reflection

most successfully?

3 LEARNING ANALYTICS

DESIGN

The demand for analytics that truly recognize users'

ownership is connected to a broader need for control

of the data that, as online users, we are constantly

generating. Considering this, student-centred

analytics share many aspects with Human-Data-

Interaction since, according to Haddadi et al. (2013)

“The term Human-Data-Interaction (HDI) arises

from the need, both ethical and practical, to engage

users to a much greater degree with the collection,

analysis, and trade of their personal data, in addition

to providing them with an intuitive feedback

mechanism” (p.3). In this regard, and in order to

support SDL and SRL, learning analytics should

provide mechanisms for learners to interact with

these systems explicitly. This requires learners to

adopt a questioning attitude and take part in the

interpretation of the data generated about them, but

they must also be offered the means to access,

understand and interact with the datasets.

The need for transparency and understandability

has also been faced by other areas that are closely

related to LA, such as Learner Models (LM). The

main difference between LA and LM lies in the type

of data monitored and its future use. So, while LA

often shows activity data (interaction time in

discussion; links in social networks or collaboration

tasks; performance data), LM use inferences drawn

from interaction in order to create a learner

information model that allows the system to be

highly personalized and adaptive. The appearance of

Open Learner Models (OLM) constitutes an

important effort towards making a student’s learner

model explicit with the aim of fostering self-

awareness and enhancing learning and learner

autonomy (Bull and Kay, 2008). An interesting case

of OLM is MyExperiences (Kump et al., 2012).

Here, the model has been designed to show users the

inferences about them, as well as the underlying

data, through a tree map visualization.

Another area that can provide interesting insights

for reframing LA is Personal Informatics (PI).

According to Li, Dey and Forlizzi (2010) PI can be

defined as a class of systems that allow people to

collect and reflect on personal information. In

contrast to LA, PI, also known as Quantified Self

CSEDU2014-6thInternationalConferenceonComputerSupportedEducation

382

(QS), requires the user to take an active role

throughout the five stages identified by Li et al.

(2010): preparation, collection, integration,

reflection and action. PI and QS have supported

informal learning in fields linked to sports and

health since they offer users opportunities to learn

about their progression and undertake new

challenges concerning healthy habits. Recently,

some scholars have noted that QS approaches can

support reflective learning and help people become

more aware of their own behaviour, make better

decisions, and change behaviour (Rivera-Pelayo et

al., 2012; Li et al., 2011; Durall and Toikkanen

2013). One important aspect to note when looking at

QS approaches is their voluntary nature. Even if QS

is used for monitoring chronic conditions, users who

self-track are motivated because they understand the

potential benefits that this practice will bring them.

In contrast, we cannot say that LA practices rely on

the learners’ voluntary participation. In this regard,

one way of encouraging learners to take an active

role in LA would consist of allowing them to choose

which data they are going to monitor from a flexible

and extendable set of indicators.

Transforming data into knowledge is a cognitive

process that can be supported by the way in which

data is made available. Information visualization has

been recognized as a tool for sense-making (Heer &

Agrawala, 2008) since it helps synthesize complex

information and facilitates comparisons and

inferences (Shneiderman, 1996; Tufte, 1990 and

1997). In the learning field, infovis has already been

recognized as a powerful tool for teachers and

students, especially through goal-oriented

visualizations such as dashboards (Duval, 2011). In

this regard, Govaerts (2010) notes that visualizations

of the learners’ activity has been used to improve

collaboration, increase awareness, support self-

reflection and find peer learners through social

network analysis. Some projects working along

these lines are CAMera, a tool for personal

monitoring and reporting (Schmitz, 2009) and

Moodog , a Moodle plug-in that visualizes data from

the activity logs to allow students to compare their

progress with others and teachers to visualize the

students’ activity in the online course (Zhang et al.,

2007).

A case study by Santos et al. (2012) using goal-

oriented visualizations of activity tracking is an

interesting experience of student-centred learning

analytics through visualizations. In this case, the

overall goal was to enable students to reflect on their

activity and compare it with their peers. With this

aim, data collected using different tools was

displayed in a goal-oriented visualization that

allowed students to filter the data by different

criteria and to compare it with their learning goals.

As the authors state, “linking the visualizations with

the learning goals can help students and teachers to

assess whether the goal has been achieved” (pp.

143). By enabling learners to filter what they want to

visualize, LA can generate metrics that relate to

what learners value in their learning process. This

way, they will be able to generate their own

questions and hypotheses that, later on, can be

contrasted through data. Learning analytics can be a

great tool for reflection since it offers students the

opportunity to revisit past experiences from a

different point of view. In order to explain the “new

situation”, it is necessary that learners recognize

their assumptions and change their perspective by

building new understandings. However, for

reflection to occur, it is important to keep in mind

that the situations “observed” must be relevant for

learners.

4 CONCLUSIONS

In this article, LA is recognized as a powerful tool

for helping students reflect on their learning activity

and, therefore, gain knowledge about their learning

processes. This is especially important since self-

knowledge can be considered as a key metacognitive

skill for SDL and SRL. Therefore, in order to truly

use analytics to help students become autonomous

learners, it is necessary to adopt a student-centred

approach.

Nowadays, the value of data requires careful and

critical reflection on issues relating to privacy, data

analysis, context of use and data ownership. In line

with other scholars, we support more transparency

and openness in LA (Clow 2012; Kruse and

Pongsajapan, 2012) since we are dealing with

sensitive information that ultimately belongs to the

learners. Therefore, educational institutions cannot

ignore the need for transparency and should ensure

that learners can see how their personal data is being

tracked and used in order to build inferences, and

how its use will affect their learning.

LA raises the issue about what is valued in the

learning process. Can learning be measured

according to, for instance, who logs into the system

most often, who engages most in group discussions

or uploads the tasks on time? There is a need to

rethink how learning indicators are selected and to

what extent they contribute to conceiving learning as

a process instead of in terms of outcomes (Clow,

LearningAnalyticsasaMetacognitiveTool

383

2012). In this regard, allowing students to decide

what aspects they are going to monitor and analyse

could help make LA a tool for reflection on learning

processes.

REFERENCES

Bartolomé, A. & Steffens, K., 2011. Technologies for self-

regulated learning. In R. Carneiro, P. Lefrere, K.

Steffens and Underwood, J. (Eds) Self-regulated

Learning in Technology Enhanced Learning

Environments: A European Review. Rotterdam: Sense

Publishers.

Bull, S., & Kay, J. (2008, June). Metacognition and open

learner models. In The 3rd Workshop on Meta-

Cognition and Self-Regulated Learning in Educational

Technologies, at ITS2008 (pp. 7-20).

Clow, D., 2012. The learning analytics cycle: closing the

loop effectively. In Proceedings of the 2nd

International Conference on Learning Analytics and

Knowledge (pp. 134-138). ACM.

Durall, E., & Toikkanen, T., 2013. Feeler: feel good and

learn better. A tool for promoting reflection about

learning and Well-being. In Proceedings of the 3rd

Workshop on Awareness and Reflection in

Technology-Enhanced Learning (pp.83-89). CEUR.

Duval, E., 2011. Attention please!: learning analytics for

visualization and recommendation. In Proceedings of

the 1st International Conference on Learning

Analytics and Knowledge (pp. 9-17). ACM.

Govaerts, S., Verbert, K., Klerkx, J., & Duval, E., 2010.

Visualizing activities for self-reflection and awareness.

In Advances in Web-Based Learning–ICWL 2010 (pp.

91-100). Berlin: Springer.

Haddadi, H., Mortier, R., McAuley, D., Crowcroft, J.,

2013. Human-Data Interaction. Technical Report, 837.

Cambridge: University of Cambridge, Computer

Laboratory.

van Harmelen, M., & Workman, D., 2012. Analytics for

Learning and Teaching. CETIS Analytics Series, 1(3).

Heer, J., Agrawala, M., 2008. Design considerations for

collaborative visual analytics. Information

Visualization, 7, 49-62.

Knight, S.; Shum, S.; Littleton, K., 2013. Collaborative

sensemaking in learning analytics. In: CSCW and

Education Workshop (2013): Viewing education as a

site of work practice, co-located with the 16th ACM

Conference on Computer Support Cooperative Work

and Social Computing (CSCW 2013), 23 February

2013, San Antonio, Texas.

Kolb, D. A., 1984. Experiential Learning: Experience as

the source of learning and development (Vol. 1).

Englewood Cliffs, NJ: Prentice-Hall.

Kruse, A., & Pongsajapan, R., 2012. Student-centered

learning analytics. Retrieved December 11, 2013 from

https://cndls.georgetown.edu/m/documents/thoughtpap

er-krusepongsajapan.pdf.

Kump, B., Seifert, C., Beham, G., Lindstaedt, S. N., &

Ley, T., 2012. Seeing what the system thinks you

know: visualizing evidence in an open learner model.

In Proceedings of the 2nd International Conference on

Learning Analytics and Knowledge (pp. 153-157).

ACM.

Li, I., Dey, A., & Forlizzi, J., 2010. A stage-based model

of personal informatics systems. In Proceedings of the

SIGCHI Conference on Human Factors in Computing

Systems (pp. 557-566). ACM.

Li, I., Dey, A. K., & Forlizzi, J., 2011. Understanding my

data, myself: supporting self-reflection with ubicomp

technologies. In Proceedings of the 13th international

conference on Ubiquitous computing (pp. 405-414).

ACM.

Loyens, S. M. M., Magda, J. & Rikers, R. M. J. P., 2008.

Self-Directed Learning in Problem-Based Learning

and its Relationships with Self-Regulated Learning.

EducPsychol Rev, 20, 411–427.

Pilling-Cormick, J., & Garrison, D. R., 2007. Self-directed

and self-regulated learning: Conceptual links.

Canadian Journal of University Continuing

Education, 33(2), 13-33.

Santos, J. L., Govaerts, S., Verbert, K., & Duval, E., 2012.

Goal-oriented visualizations of activity tracking: a

case study with engineering students. In Proceedings

of the 2nd International Conference on Learning

Analytics and Knowledge (pp. 143-152). ACM.

Schmitz, H. C., Scheffel, M., Friedrich, M., Jahn, M.,

Niemann, K., & Wolpers, M., 2009. CAMera for PLE.

In Learning in the Synergy of Multiple Disciplines (pp.

507-520). Berlin: Springer.

Siemens, G., & Baker, R. S. D., 2012. Learning analytics

and educational data mining: Towards communication

and collaboration. In Proceedings of the 2nd

International Conference on Learning Analytics and

Knowledge (pp. 252-254). ACM.

Slade, S., & Prinsloo, P., 2013. Learning analytics: ethical

issues and dilemmas. American Behavioral Scientist,

In-press.

Society for Learning Analytics. 2013. Retrieved December

11, 2013 from http://www.solaresearch.org/mission/

about/

Tan, S. C., Divaharan, S., Tan, L., & Cheah, H. M., 2011.

Self-directed learning with ICT: Theory, Practice and

Assessment. Singapore: Ministry of Education.

Tufte, E. R., 1990. Envisioning Information. Cheshire,

Connecticut: Graphic Press.

Tufte, E. R., 1997. Visual Explanations. Images and

Quantities, Evidence and Narrative. Cheshire,

Connecticut: Graphic Press.

Zhang, H., Almeroth, K., Knight, A., Bulger, M. & Mayer,

R., 2007. Moodog: Tracking students' online learning

activities. In World Conference on Educational

Multimedia, Hypermedia and Telecommunications

(Vol. 2007, No. 1, pp. 4415-4422).

Zimmerman, B. J., 1989. A social cognitive view of self-

regulated academic learning. Journal of Educational

Psychology, 81, 329–339.

CSEDU2014-6thInternationalConferenceonComputerSupportedEducation

384