Moodle Tools for Educational Analytics of the Use of Electronic

Resources of the University’s Portal

Olena G. Glazunova

1 a

, Maksym V. Mokriiev

1 b

, Olena H. Kuzminska

1 c

Valentyna I. Korolchuk

1 d

, Nataliia V. Morze

2 e

, Liliia O. Varchenko-Trotsenko

2 f

and

Roman A. Zolotukha

1 g

National University of Life and Environmental Sciences of Ukraine, 15 Heroyiv Oborony Str., Kyiv, 03041, Ukraine

Borys Grinchenko Kyiv University, 18/2 Bulvarno-Kudryavska Str., Kyiv, 04053, Ukraine

Keywords:

E-Learning, Learning Management Systems, Educational Analytics, E-Course, Analytics Tools CLMS,

Moodle, Higher Education.

Abstract:

The need for additional analysis of the effectiveness of e-learning implementation models and their resource

support in higher education institutions in the context of the COVID-19 pandemic has been actualized. An

overview of solutions and case studies in the context of selecting and analyzing the effectiveness of individual

services and learning management platforms is provided. It has been studied that in order to investigate the

effectiveness of using electronic resources to meet students’ educational needs, it is advisable to use quanti-

tative indicators in addition to student description results. This includes data from educational analytics on

the frequency and duration of students’ use of individual e-resources. Reviewed the functionality modules

“Course Comparison” of the Moodle LMS and “Statistics”, as well as the optional Analytics module. The

results of applying these modules to the analysis of e-learning courses of the National University of Life and

Environmental Sciences of Ukraine and Boris Grinchenko Kyiv University are presented. The reasons for

students’ low use of individual e-courses were investigated.

1 INTRODUCTION

The issue of the quality of e-learning resources in a

distance learning environment is extremely relevant

during the quarantine period associated with COVID-

19 (Vakaliuk et al., 2021). Despite the various qual-

ity assurance procedures for e-learning resources for

students, especially in forced distance learning set-

tings, it is often very difﬁcult to assess the quality of

the resources used relying on these procedures, which

mainly include student surveys and peer review. How-

ever, due to the transition of higher education insti-

tutions to distance learning (Bobyliev and Vihrova,

2021) and in the context of the pandemic generated

https://orcid.org/0000-0002-0136-4936

https://orcid.org/0000-0002-6717-3884

https://orcid.org/0000-0002-8849-9648

https://orcid.org/0000-0002-3145-8802

https://orcid.org/0000-0002-0136-4936

https://orcid.org/0000-0003-0723-4195

https://orcid.org/0000-0003-3099-722X

by COVID-19, the problem of analyzing the quality

and adapting the design models of educational envi-

ronments (Morze et al., 2013; Glazunova and Shyshk-

ina, 2018), according to the types of institutions, ed-

ucational program, available resources, has become

more relevant and other (Edelhauser and Lupu-Dima,

2020).

Looking at e-learning quality indicators from

2000 to 2017, Silva et al. (Silva et al., 2018), based

on an analysis of scientiﬁc publications, identiﬁes

that indicators that have the greatest weight can be

grouped into three categories: e-resources, data, pro-

cesses. In the context of this study, we can look at

the same resource provision with a focus on practical

cases and a combination of quantitative and qualita-

tive assessment.

For example, in distance learning for future health

professionals, digital imaging should be used ex-

tremely closely to real-life practice. Educational

institutions use specialized software, e.g. Clinical

Study Export (TCE) as a platform for extending the

PACS infrastructure by connecting educational func-

444

Glazunova, O., Mokriiev, M., Kuzminska, O., Korolchuk, V., Morze, N., Varchenko-Trotsenko, L. and Zolotukha, R.

Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal.

DOI: 10.5220/0010932700003364

In Proceedings of the 1st Symposium on Advances in Educational Technology (AET 2020) - Volume 2, pages 444-451

ISBN: 978-989-758-558-6

tions (Mildenberger et al., 2011).

Understanding the importance of communication

and cooperation in the implementation of distance

learning, the effectiveness of the use of different ser-

vices is the subject of analysis. For example, Biasutti

(Biasutti, 2017) presents the results of a comparative

analysis of forums and wikis as tools for online col-

laborative learning. However, as each service has to

be seen in the context of student training requirements

and educational goals, more and more researchers

are looking at integrated solutions to support self-

regulated educational strategies (Kraleva et al., 2019).

In this context, among the various learning Man-

agement Systems (LMS) on the market (Basaran and

Mohammed, 2020), LMS Moodle is the most pop-

ular in the implementation of distance and blended

learning in higher education (Oguguo et al., 2021;

Mintii, 2020; Abdula et al., 2020). According to re-

search by scientists from different countries, techno-

logical satisfaction about Moodle in higher education

is quite high – the effect is equal to 0.78 with a 95%

conﬁdence interval from 0.72 to 0.84 (Garc

ıa-Murillo

et al., 2020). The transition to distance learning and

increasing the frequency of interaction between teach-

ers and students through LMS has necessitated the

search for resources to bring online communication

closer to ofﬂine. One such solution is Moodle LMS

integration with Amazon Alexa for creating voice

content (Ochoa-Orihuel et al., 2020).

On the other hand, the success of a distance ed-

ucation program can be assessed, in addition to aca-

demic performance, by the level of student satisfac-

tion. However, there is a correlation between stu-

dent satisfaction with e-courses (e-resources and con-

tent) and readiness for online learning (Deveci Topal,

2016). The latter is usually determined by surveying

students, and is checked – by data of educational an-

alytics which allow to analyze behavior of students

in LMS (Kadoi

c and Ore

ski, 2018). Since the use

of educational analytics or analytics for student suc-

cess, according to Brown et al. (Brown et al., 2020)

is identiﬁed as one of the areas of educational tech-

nology, we consider it appropriate to use educational

analytics to determine the causes of student satisfac-

tion / dissatisfaction, followed by recommendations

for improving resources and methodological support

its application in higher education institutions. To do

this, it is necessary to explore the tools that can be

used to quickly analyze the effectiveness of resource

use, in particular, e-learning courses in disciplines,

and relevant indicators. Such analysis will allow a

rapid response to the problems that users of e-courses

are experiencing, allow for quick resolution of these

problems and make student learning more productive.

Research goal: justify the choice of tools for ed-

ucational analytics on the use of e-learning courses,

in particular to determine user activity, frequency and

duration of use of course resources in order to rec-

ommend to teachers to improve the quality of both

course materials and methods of using relevant re-

sources during distance learning.

Figure 1: Moodle e-course comparison module data model.

2 METHODS AND STUDY

MATERIALS

The study was conducted using data from the train-

ing portal of National University of Life and Envi-

ronmental Sciences of Ukraine (NULES) and Borys

Grinchenko Kyiv University (BGKU). Methods and

technologies of statistical analysis were used for the

research.

The university’s learning portal usually operates

on the basis of CLMS (Content Learning Manage-

ment System) platforms and is designed to support

the learning process with e-resources in the format

of e-books, web pages, lessons and video lessons,

test tasks, laboratory and practical, independent work.

For each discipline in the e-learning course can be

placed the above and other resources. It should be

noted that the use of e-learning courses in the ed-

ucational process should correspond to the working

curriculum of the discipline, and the resources should

contain relevant, popular information. The procedure

for attestation of electronic courses in universities in-

volves the implementation of a number of criteria,

Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal

445

which are usually spelled out in the relevant regula-

tions. In particular, in NULES, these criteria are di-

vided into structural-functional, scientiﬁc-substantive

and methodological. But the use of certiﬁed courses

is not uniform throughout the semester. At the same

time, students actively use part of the resources in the

courses, and some resources are not used at all. In or-

der to identify the reasons for this and to select tools

for quick analysis of course performance data and its

resources, it is necessary to analyze the relevant tools,

which are built-in or complementary.

Let us focus on the statistical and analytical tools

of CLMS Moodle. Course resource efﬁciency indi-

cators can be obtained through the use of the embed-

ded modules: “Course Comparison” and “Statistics”

modules, as well as the optional “Analytics” module.

2.1 Features of the Module

“Comparison of Courses”

The analysis modules in the base Moodle distribution

are not very powerful, but they are present too. One

of the ﬁrst modules that is appropriate to use when

analyzing courses is Course Comparison. This can be

found under Manage – Site Management – Reports.

This module allows you to view four reports:

1. The most active courses (ranks courses by the

number of actions taken by their participants as

a whole).

2. Most active courses (weighted) (calculates the av-

erage performance per user in the course).

3. Participation rate (shows courses in descending

order of user participation rate).

4. Activity ratio (determined by the ratio of involve-

ment and participation of users in the course).

Some of these concepts need clariﬁcation.

The ﬁrst analytical report gives us the opportunity

to see on which course there is active activity. Activ-

ity on a course is the total number of all views and

publications on the course during the period under

study. However, this is of little use as a large num-

ber of enrolled users will visually create more activity

compared to courses with a relatively small number

of participants.

Consequently, a weighted average of each user’s

activity on the course is already more informative and

will show everyone’s participation. Here you can see

how truly active courses with a small number of users

come to the fore. However, if there are few active

users on the course, the mass of enrolled but inactive

users will drag the course down in the rankings.

The following report can clarify this nuance. It

shows how many real active students are on the

course. The participation rate of active users is calcu-

lated as the share of active users to the total enrolment

in the course. Active users are deﬁned as those who

had had activity during the period under study. But

here again the question arises, what are these users

doing on the course? Are they just reading (receiving

information) or are they active?

The fourth report gives us the answer to this ques-

tion. The publication and views activity ratio is calcu-

lated as the share of publications in relation to views.

Where views refer to any user going to another page

and ’reading’ it or downloading a ﬁle resource from

the course to their computer. Publications are deﬁned

as any activity the user performs on a course, for ex-

ample, completing a quiz, completing a task (down-

loading a ﬁle or writing a text response), replying to a

forum post, and so on. That is publishing is not just a

forum post.

2.2 Features of the “Statistics” Module

Another auxiliary module for analyzing course per-

formance is the Statistics module, which operates at

the site level, providing some statistics on the activity

on courses as a whole, as well as in each individual

course. At the site level, the results of this module are

available to administrators and site managers. At the

level of each course, teachers can use it to generate

statistics within that course.

About the Course Comparison module analyzing

activities without dividing into teachers and students,

it is important to say exactly who generates such ac-

tivities – the actual training of students or the active

creation of a teaching course by teachers. The Statis-

tics module brings clariﬁcation. Using this module,

we can see the activities of each individual role. It is

also possible to look separately at views, only publi-

cations, or only introductions. At a site-wide level,

these metrics plot all roles, while at a course level we

can get a separate graph for each role.

2.3 Possibilities of the “Analytics”

Module

Additional analytical reports can be obtained using

third-party modules. One such powerful addition is

the Analytics module. With its help it is possible to

analyze activity of each student both as a whole on a

course, and in each concrete resource of a course. We

are provided with such reports:

• Valuation chart (shows the distribution of valua-

tions using a stock chart)

• Work with content (shows the activities of stu-

AET 2020 - Symposium on Advances in Educational Technology

446

Figure 2: Courses usage statistics.

dents with each resource separately – how much

they worked and how much they ignored)

• Student activity (shows a consolidated distribu-

tion of student activities on the course in terms

of hours per day)

• Execution of tasks (shows a diagram for all Tasks,

which demonstrates compliance with the dead-

lines)

• Passing tests (shows a diagram for all tests, which

demonstrates compliance with the deadlines)

• Distribution of views (shows the schedule of per-

sonal activities on the course of each student)

3 MAIN FINDINGS

Only certiﬁed electronic learning courses (ELC) were

chosen for the study, i.e. courses in which the struc-

ture and set of resources are correctly selected for the

implementation of the educational process. Thus in

NULES the certiﬁed courses operate within 5 years,

and in BGKU – within 1 year. The number of certi-

ﬁed courses at the end of 2020 in NULES was 1644

courses, in BGKU – 768 courses.

The ﬁrst hundred most active courses (with high

average activity per user) can be obtained by using the

Course Comparison module and its “active courses

(weighted)” report (ﬁgure 2).

Based on the results (ﬁgure 2) it is possible to

identify courses with low user activity. The follow-

ing analysis of the content of such courses, didactic

features of the use of course resources will make it

possible to identify relevant problems with their use.

Usage statistics courses and the way they use

within the categories are also using the module

“Statistics”. For example, in BGKU such statistics

can be obtained by categories of departments (ﬁg-

ure 3), and in NULES – by categories of specialties.

A direct query to the database of courses makes it

possible to obtain such data for all certiﬁed courses.

As a result, all courses can be divided into 3 cat-

egories: courses with low efﬁciency, sufﬁcient and

high in terms of “activity per user”. For example,

NULES with a high degree of use has 12% of cer-

tiﬁed ELC, with sufﬁcient – 57%, with low – 31%.

Analyzing ELCs with high efﬁciency, a number of

studies have been conducted on the use of resources

of these ELCs using an analytical module built into

Moodle.

For example, ﬁgure 4 reﬂects the students’ activ-

ity in the course “Computer Technologies and Pro-

gramming” during the last semester. The activity of

students in revising resources and publishing com-

pleted tasks or passing tests is uneven. The peak

falls towards the mid-term examinations and the end

of the semester, which may explain the need to com-

plete the quizzes. But we can conclude that the use

of the course at the beginning of the module is not

Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal

447

Figure 3: Courses usage statistics by category.

active enough and indicates that the laboratory and

self-study assignments are not completed on time and,

consequently, students are not working with the re-

sources.

To ﬁnd out with which resources in e-courses, stu-

dents work actively, the function “Working with con-

tent” of the module “Analytics” is used. For exam-

ple, ﬁgure 5 shows the activity of using theoretical

resources of the course “Information Technology”.

From this diagram we can conclude about the ex-

tremely low activity of students in the use of theo-

retical resources, in contrast to laboratory work.

Often students actively use electronic learning re-

sources in a discipline only because they have to take

a test every day and hand in work to be tested, but they

do not use theoretical resources because they are not

very informative. Another option is for students to ac-

tively use methodological materials in the discipline,

watching video tutorials, and to a lesser extent use re-

sources designed to monitor learning achievements.

An important task for universities is to obtain tools

to quickly assess the quality of electronic resources

by further using their content and methodology in the

teaching process.

Next, you need to analyze the content of edu-

cational material set out in theoretical resources, in

terms of structure, accessibility, relevance, practical

orientation. These are all tasks of scientiﬁc and sub-

stantive examination. Such an electronic course can

be reconsidered by the educational and methodical

commissions of the faculties regarding the possibility

of its use in the educational process.

As a result of using such tools, we have the oppor-

tunity to determine which e-courses contain:

• an excessive number of tasks (exceeding the num-

ber of laboratory, modular and independent tasks)

AET 2020 - Symposium on Advances in Educational Technology

448

Figure 4: Analysis of electronic courses on the subject of general activity of students.

Figure 5: Analysis of e-courses on the use of resources.

that required the work of students with the course;

• little informative, unstructured training materials

• qualitatively presented theoretical materials and

methodical recommendations, which were ac-

tively used by students;

• educational resources that were systematically

used during the semester.

To increase the efﬁciency of the use of ELC in

higher education, a number of steps can be taken to

use statistical and analytical tools Moodle (ﬁgure 6).

The ﬁrst step should be to rank the courses by ac-

tivity per 1 user (weighted indicator). For all courses

that are actively used, the second step is performed –

statistics of general activity in e-courses and analytics

of resource use in e-courses. The third step is to form

Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal

449

Figure 6: Scheme of using statistical and analytical tools Moodle to increase the efﬁciency of ELC.

conclusions and recommendations.

Identifying e-courses that are insufﬁciently used

in the educational process, provides an opportunity to

intensify work with teachers to improve their skills

with information support, to create e-courses, the use

of e-resources in the educational process. The e-

resources found not to be used by students in the

learning process should be reviewed according to sci-

entiﬁc peer review criteria. Built-in and additional

CLMS Moodle tools allow you to analyze the effec-

tiveness of e-courses in general and in terms of differ-

ent types of resources and, based on this analysis, to

form general recommendations for course teachers to

improve the use of e-courses in education.

4 CONCLUSIONS AND THE

RESEARCH PERSPECTIVE

The use of statistical and analytical tools in CLMS

Moodle to determine the effectiveness of the use of

e-courses contributes to the quality of the educational

process, in particular blended and distance learning.

By measuring weighted course user activity, overall

activity within the course and analyzing the use of

course resources, it is possible to identify the reasons

for the inefﬁcient use of e-courses in the educational

process. Since the study was carried out on the ba-

sis of two higher education institutions, we can assert

general trends on the problems of using e-courses in

blended and distance learning. In the future, we see

the need to develop a model that provides automated

determination of levels of effectiveness of e-courses,

e-course resources, identiﬁcation of factors that affect

the effectiveness of the use of courses, and speciﬁc

resources.

REFERENCES

Abdula, A. I., Baluta, H. A., Kozachenko, N. P., and Kas-

sim, D. A. (2020). Peculiarities of using of the Moo-

dle test tools in philosophy teaching. CEUR Workshop

Proceedings, 2643:306–320.

Basaran, S. and Mohammed, R. K. H. (2020). Usability

evaluation of open source learning management sys-

tems. International Journal of Advanced Computer

Science and Applications, 11(6):400–410.

Biasutti, M. (2017). A comparative analysis of forums and

wikis as tools for online collaborative learning. Com-

puters & Education, 111:158–171.

Bobyliev, D. Y. and Vihrova, E. V. (2021). Problems and

prospects of distance learning in teaching fundamental

subjects to future mathematics teachers. Journal of

Physics: Conference Series, 1840(1):012002.

Brown, M., McCormack, M., Reeves, J., Brooks, D. C., ,

Grajek, S., Alexander, B., Bali, M., Bulger, S., Dark,

S., Engelbert, N., Gannon, K., Gauthier, A., Gibson,

D., Gibson, R., Lundin, B., Veletsianos, G., and We-

ber, N. (2020). The 2020 EDUCAUSE Horizon Re-

port: Teaching and Learning Edition. EDUCAUSE,

Louisville, CO. https://library.educause.edu/-/media/

ﬁles/library/2020/3/2020 horizon report pdf.pdf.

Deveci Topal, A. (2016). Examination of university

students’ level of satisfaction and readiness for e-

courses and the relationship between them. Euro-

pean Journal of Contemporary Education, 15(1):7–

23. http://ejournal1.com/journals n/1459666234.pdf.

Edelhauser, E. and Lupu-Dima, L. (2020). Is Romania Pre-

pared for eLearning during the COVID-19 Pandemic?

Sustainability, 12(13). https://www.mdpi.com/2071-

1050/12/13/5438.

Garc

ıa-Murillo, G., Novoa-Hern

andez, P., and Rodr

ıguez,

R. S. (2020). Technological Satisfaction About Moo-

dle in Higher Education—A Meta-Analysis. IEEE Re-

vista Iberoamericana de Tecnologias del Aprendizaje,

15(4):281–290.

Glazunova, O. and Shyshkina, M. (2018). The concept,

principles of design and implementation of the univer-

AET 2020 - Symposium on Advances in Educational Technology

450

sity cloud-based learning and research environment.

CEUR Workshop Proceedings, 2104:332–347.

Kadoi

c, N. and Ore

ski, D. (2018). Analysis of student

behavior and success based on logs in moodle. In

2018 41st International Convention on Information

and Communication Technology, Electronics and Mi-

croelectronics (MIPRO), pages 0654–0659.

Kraleva, R., Sabani, M., and Kralev, V. (2019). An analysis

of some learning management systems. International

Journal on Advanced Science, Engineering and In-

formation Technology, 9(4):1190–1198. http://ijaseit.

insightsociety.org/index.php?option=com content&

view=article&id=9&Itemid=1&article id=9437.

Mildenberger, P., Br

uggemann, K., R

osner, F., Koch, K.,

and Ahlers, C. (2011). PACS infrastructure sup-

porting e-learning. European Journal of Radiology,

78(2):234–238. From PACS to the clouds.

Mintii, I. S. (2020). Using Learning Content Management

System Moodle in Kryvyi Rih State Pedagogical Uni-

versity educational process. CEUR Workshop Pro-

ceedings, 2643:293–305.

Morze, N., Kuzminska, O., and Protsenko, G. (2013). Pub-

lic information environment of a modern university.

CEUR Workshop Proceedings, 1000:264–272.

Ochoa-Orihuel, J., Marticorena-S

anchez, R., and S

aiz-

Manzanares, M. C. (2020). Moodle LMS Integration

with Amazon Alexa: A Practical Experience. Ap-

plied Sciences, 10(19). https://www.mdpi.com/2076-

3417/10/19/6859.

Oguguo, B. C. E., Nannim, F. A., Agah, J. J., Ugwuanyi,

C. S., Ene, C. U., and Nzeadibe, A. C. (2021).

Effect of learning management system on student’s

performance in educational measurement and eval-

uation. Education and Information Technologies,

26(2):1471–1483.

Silva, J. C. S., Zambom, E., Rodrigues, R. L., Ramos,

J. L. C., and da Fonseca de Souza, F. (2018). Ef-

fects of learning analytics on students’ self-regulated

learning in ﬂipped classroom. International Journal

of Information and Communication Technology Ed-

ucation, 14(3). https://www.igi-global.com/gateway/

article/205624.

Vakaliuk, T. A., Spirin, O. M., Lobanchykova, N. M., Mart-

seva, L. A., Novitska, I. V., and Kontsedailo, V. V.

(2021). Features of distance learning of cloud tech-

nologies for the organization educational process in

quarantine. Journal of Physics: Conference Series,

1840(1):012051.

Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal

451