Complex Framework of Digital Learning Quality Assessment in
Covid-19 Context: Survey Study for European and Oriental
Languages Programs
Rusudan Makhachashvili
a
and Ivan Semenist
b
Borys Grinchenko Kyiv University, Bulvarno-Kudryavska-st., 18/2, Kyiv, Ukraine
Keywords: Hybrid Learning, e-Learning, Complex Digital Learning Model, Individual Quality Assessment, Oriental
Languages, European Languages.
Abstract: The global pandemic and subsequent quarantine measures have posed an array of challenges to the complex
comprehensive structures and procedures of higher education workflow, which influenced significantly the
scope of individual experiences, projected outcomes, and estimated quality of higher education in countries
across the world. This study focus is the in-depth assessment of individual perceptions and experiences of
complex models of digital distance and blended learning by students of different tiers (Bachelor’s level,
Master’s level, Graduate school level) in European and Oriental Languages university programs in regions of
Ukraine through the span of educational activities in the time-frame of COVID-19 quarantine measures of
March 2020 to January 2021. The survey is used to assess the parameters and challenges of individual quality
and efficiency of translation of the complex system of real-life Foreign Languages Acquisition practices into
the digital format, involving activation of interdisciplinary skills and cross-sectorial activities, implemented
by ICT tools. The online survey of 14 universities in regional centres of Ukraine provides for disclosure of
student comprehensive assessment of the complex framework of digitalized foreign languages education,
systematized individual experiences, and quality estimation of e-learning and hybrid learning in the
framework of COVID-19 lockdown.
1 INTRODUCTION
Theoretical problems of complex, holistic,
multidimensional modelling of reality and its separate
spheres (one of which within the English-speaking
linguistics of the XX-XXI centuries is the sphere of
development of the latest computer technologies) are
directed by the deterministic interaction of objects,
signs of their reception and interpretation (in the field
of individual and collective consciousness),
embodiment, consolidation and retransmission of the
results of interaction of these systems of features.
In this regard, the fundamental dimension of being
(as synonymous with the latter "reality", "realm" and
"world" can be used) (Aristotle, 2014; Budko, 1990;
Kireev, 2008) is defined as a heterogeneous concept
that can summarize the multidimensional features of
the world order: the world that really exists and is
a
https://orcid.org/0000-0002-4806-6434
b
https://orcid.org/0000-0002-0847-8856
subject to direct perception; a world that is not subject
to direct reception, but exists in reality; a world that
is imaginary, unreal (for example the ideal,
mythological images); reality, which exists
objectively, regardless of human consciousness
(nature, objective physical laws of the world order,
space-time dimensions); the general way of life of
humankind, society, culture, civilization.
Thus, the Computer Being dimension is
understood as a fundamental ontological parameter
that exists outside the scope of human mind and is
involved in the perceptual field regardless of the will
of the subject of cognition.
The intellectualization of modern global culture
determines a qualitatively new approach to
understanding the processes of parallel development
of human activities and cognitive (intellectual)
experiences. That is the origin and methodological
premise of the concept of "Noosphere". The
Makhachashvili, R. and Semenist, I.
Complex Framework of Digital Learning Quality Assessment in Covid-19 Context: Survey Study for European and Oriental Languages Programs.
DOI: 10.5220/0010496701070116
In Proceedings of the 6th International Conference on Complexity, Future Information Systems and Risk (COMPLEXIS 2021), pages 107-116
ISBN: 978-989-758-505-0
Copyright
c
 2021 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
107

Noosphere is defined as the current stage of
development of the biosphere, associated with the
emergence of humanity in it (Kasavin, 1990; Gachev,
1993), and is interpreted as part of the planet and
planet ambient with traces of human activity.
The integral real component of the Noosphere is
identified as the Technosphere – a set of artificial
objects (technologies) created by the humankind, and
natural objects changed as a result of technological
activity of humankind (Heim, 1993; Makhachashvili,
2020). In turn, Computer Being (computer reality,
cyberspace) is a complex, multidimensional,
interdisciplinary sphere of synthesis of reality, human
experience and activity mediated by the latest digital
and information technologies; technogenic reality, a
component of the Technosphere of existence.
In the educational sphere, according to our
estimations, the result of fundamental Technosphere
shift, induced by the COVID-19 pandemic
development and enhanced by subsequent
digitalization measures, was the need to take quick
comprehensive action (Makhachashvili, Semenist,
Bakhtina, 2020) in order to achieve such desirable
results: a) to adapt the existent complex educational
scenarios to digital, remote and blended formats; b) to
activate comprehensive complex skillsets, otherwise
latent or underutilized in the educational process; c)
to boost ICT competence and digital literacy of all
participants of the educational process relocated to
computer being.
The global pandemic of COVID-19 emerged as a
kind of a black swan scenario or a singularity (event
horizon) (Tu, 1994) for interdisciplinary domains of
social and economic life, including education. The
black swan theory is a concept that describes an event
that comes as a surprise, has a major effect on society,
and is often rationalized after the fact (Taleb, 2010).
The structure and procedure of higher education
workflow in 2020-2021 were disrupted by a range of
challenges, spawned by the global pandemic
restriction measures, which influenced significantly
the scope of individual experiences, projected
outcomes and estimated quality of higher education in
countries across the world.
Foreign Languages Acquisition at university level
major programs is a rigorous process (Law of Ukraine,
2019) that involves different a complex arrangement
of stages and a comprehensive regimen of
communicative activities and competences across
interconnected interdisciplinary domains. The
presented study is a parcel of comprehensive
institutional inquiry (TRANSITION: Transformation,
Network, Society, Education) into the toll
digitalization and amplified emergency use of ICT
tools put on different aspects of foreign languages
acquisition efficiency, assessment, programmed
results, soft skills and digital competency formation in
COVID-19 lockdown paradigm.
The accumulated data so far has been seminal for
a range of publications (Makhachashvili, Semenist,
Bakhtina, 2020; Makhachashvili, Semenist, 2021),
covering the issues of ICT tools efficiency rating
regimen for foreign languages programs, isolated and
comparative case studies of Oriental and European
languages acquisition digitization procedures and best
practices, student and faculty satisfaction with
emergency online education measures.
The estimations of digital distant education state-
of the-art, structure and quality, conducted across
various studies in the pre-COVID-19 inquiry
framework have spanned, among others, assessment
of satisfaction with distance learning experience
(Bekele, 2010; Bolliger, 2009); added value of online
learning (Thiessen, Ambrock, 2011); evaluation of
learning outcomes (Costareie, 2011); overall attitudes
to distance learning (Salyers et al, 2014); challenges
of online education (Markova et al, 2017), networking
principles in e-learning (Smyrnova-Trybulska,
Kommers et al, 2019).
These issues require a comprehensive revisit in
terms of the toll the global pandemic took on complex
framework transformation of educational formats to
digital remote mode, subsequently, on the individual
learning experiences and subsequent shift in quality
estimations of linguistic education in universities due
to the abrupt transition to exclusively distant, digital
distant or hybrid learning formats as a mode of formal
education and university degree acquisition (as
opposed to being a mostly auxiliary learning method
or an informal learning format).
This study objective, henceforth, is to estimate
and critically review the quality assessment of
complex, digitally enhanced formats of linguistic
education via in-depth evaluation of individual digital
distance and hybrid learning experiences and best
practices by students of Oriental and European
Languages of university programs in regions of
Ukraine (Eastern Europe) through the span of
educational activities in the time-frame of COVID-19
quarantine measures of March 2020 to January 2021.
The survey and analysis of different ICT tools is used
to assess the comprehensive array of parameters and
challenges of individual quality and efficiency of
translation of the real life Foreign Languages
Acquisition practices into digital blended format,
involving activation of interdisciplinary skills and
cross-sectorial activities, assisted by ICT tools.
COMPLEXIS 2021 - 6th International Conference on Complexity, Future Information Systems and Risk
108

The study premise is based on identification of
complex digital distant learning formats (u(biquitous)-
learning (Crowe, 2007), m(obile)-learning, b(lended)-
learning, hybrid learning) and modes for university
educational programs and projected digital literacy
requirements (European Commission, 2020; DQ,
2019; World Economic Forum, 2020).
2 METHOD AND SAMPLE
OVERVIEW
The study employs the combination of mixed methods
(Almalki, 2016) – a proportional arrangement of
quantitative and qualitative inquiry to assess in-depth
aspects of subjective and individual quality estimation
of digital distant and hybrid learning. The
comprehensive study design methodology included
the following consecutive steps: 1) complex format of
digital distant learning and application modelling; 2)
The online survey method - based on D. Dillman’s
concept of mixed media and mixed mode surveys
(Dillman, 2014) – applied to conduct in-depth
assessment of individual digital distance and hybrid
learning experiences and practices by relevant groups
of stakeholders in Foreign Languages university level
programs in regions of Ukraine; 3) digital content
analysis of individual associations with different
modes of learning by relevant groups of stakeholders
in Foreign Languages university level programs in
regions of Ukraine (via corpus and text-mining tools);
4) digital distance and blended learning models and
procedures adaptation and customization, tailored to
the individual and group needs of stakeholders in
Foreign Language Acquisition on university level.
Based on the activity profile (digital distance and
blended learning in Foreign Language Acquisition) an
online survey was devised to assess in-depth
subjective experiences of e-learning and hybrid
learning in COVID-19 timeframe (March 2020 –
January 2021) for students of Oriental languages
(Mandarin Chinese and Japanese) and European
Languages (English, French, Spanish, Italian) major
programs.
The online questionnaire comprised of 21
questions total (multiple choice, ranking/scoring and
open-end/short answer types), divided into groups,
according to inquiry dimensions: 1) questions on
overall assessment of individual experiences and
quality estimation of e-learning and hybrid learning in
the framework of COVID-19 lockdown and
quarantine measures (March 2020 – January 2021) for
university programs of Oriental and European
languages; 2) questions on assessment of individual
experiences and quality of e-learning and hybrid
learning in the framework of COVID-19 lockdown as
compared to traditional, face-to-face learning formats;
3) questions on individual quality assessment of
learning process design, programmed leaning
outcomes and projected competences for university
programs of European and Oriental languages in the
framework of COVID-19 lockdown and quarantine
measures (March 2020 – January 2021).
A total 507 respondents total across programs of
European and Oriental languages in universities of the
capital city and regional universities of Ukraine took
part in the survey. The following groups of
stakeholders of the digital distant education in the
timespan of COVID-19 pandemic measures were
respondents of the survey overall in regions of
Ukraine: Students of Bachelor’s programs for Foreign
languages (1st, 2nd, 3rd and 4th years of study) –
94,3%; Students of Master’s programs for Oriental
Foreign languages (1st and 2nd years of study) –
3,8%; Students of Post-graduate programs for
Linguistics and Foreign languages (1st, 2nd, 3rd and
4th years of study) – 1,9%.
The survey sample geographic distribution covers
the universities of the capital city of Ukraine (Kyiv) (3
universities) and a representative selection of regional
universities of Ukraine (14 universities), offering
major programs in Oriental (Mandarin Chinese,
Japanese) and European (English, French, Spanish,
Italian) languages on all three tier of educational levels
according to the legislature of Ukraine (Law of
Ukraine, 2019): undergraduate, graduate and post-
graduate.
The regional parameters of the assessed
universities allow to estimate complex quality
assurance through individual experiences of e-
learning and hybrid learning in the framework of
COVID-19 lockdown across foreign languages
programs in 9 regional city-centers of Ukraine
(Kharkiv, Dnipro, Zaporizhzhya, Kropivnitski
(Kirovohrad), Kherson, Odessa, Chernivtsi, Lviv,
Lutsk) and 3 regional cities of Ukraine (Mariupol,
Kryviy Rih, Dneprodzerzhinsk). The geographic
distribution of the sample allows to feature all regions
of Ukraine - North, East, South and West.
For the purposes of this study the survey results
were scaled to feature students of European and
Oriental languages programs of 14 regional
universities of Ukraine (excluding the capital city –
Kyiv) to estimate the parameters and challenges of the
individual quality assessment of complex e-learning
and hybrid learning formats in the framework of
Complex Framework of Digital Learning Quality Assessment in Covid-19 Context: Survey Study for European and Oriental Languages
Programs
109

COVID-19 lockdown, common for higher educational
institutions of the regional scale.
The overarching research project hypothesis is the
observable variation in the individual perception of e-
learning and hybrid learning quality of foreign
languages acquisition by different groups of
stakeholders in different regions of Ukraine and in the
capital city due to an arrangement of educational (core
competences, soft skills proficiency), social and
psychological (information fatigue, stress, community
influence) and technological factors (digital literacy,
digital divide, digital gap, pre-existing level of
educational process digitization and e-learning
experiences).
3 FINDINGS
3.1 Modelling of Complex Digitally
Enhanced Learning Formats
For the purposes of this study digitally enhanced or
electronic learning (e-learning) is estimated as a form
of ubiquitous learning (u-learning) that involves
learning in an environment with full access to digital
devices and services at any given moment (Van’t
Hooft, 2007).
As a parcel of the Technosphere shift in
comprehensive human activity, parameterization
principles of a concept of electronic (or digitally
enhanced) interaction and communication in the
paradigm of the humanities in general
(Makhachashvili, 2020), allow to identify the features
of e-learning as a complex object system pertaining
the following parameters: Ubiquity; Ontocentrism;
Integrativity; Automorphism; Normativity;
Communicative substantiality; Information capacity;
Interactive framework meta status.
Through the fragmented set of qualitative
features, digitally enhanced (electronic) learning
framework is tangent to the concepts of complex
frameworks of cognitive outlook and activity, such
as: 1) Model of the world/worldview (inclusivity,
integrativity, automorphism); Linguistic
worldview/mapping (communicative substantiality;
interactive framework meta status); Noosphere
(ontocentrism, information capacity).
The integrative concept of e-learning stands as a
complex synthesis of these frameworks (Figure 1):
Figure 1: E-learning profiling against complex cognitive
systems.
The framework evolution and innovation of e-
learning as a complex system of human activity in the
digital age, is determined by a range of qualifying
conditions of its emergence, existence and
transformation in the COVID-19 timeframe,
including: 1) exhaustive synchronization of the
object, phenomenological and anthropological planes
of educational activities and procedures of ICT
development; 2) exhaustive output of
parameterization isomorphism of ontological
(substance phenomenological), anthropic and
digitized structures of reality and educational activity;
3) flexibility, adaptability and dynamic potential of
the educational activities enhanced by ICT tools and
digital technology (that is fulfilled, in particular
through info-capacity, hybridization, and evolution of
the basic functional features of the learning process).
COVID-19 quarantine measures, put in place
country-wide in educational institutions Ukraine in the
time spans of March-June 2020, October-December
2020 and January 2021 have demanded the
simultaneous employment and overlap of the
following structurally complex learning formats: e-
learning 1.0 (direct synchronous computer assisted
distant instruction); e-learning 2.0 (asynchronous
computer-assisted collaborative learning) (Trentin,
2010); blended learning (in-person teaching with
asynchronous ICT assisted learning methods) (Voorn,
Kommers, 2013; Dos Reis, 2015; Boyarsky, 2020);
hybrid learning (synchronous instruction of in-
presence and remote students via ICT tools) (Duff,
2020).
Modelling of the comprehensive framework of
educational activities and experiences in foreign
language acquisition transformation into digitally
enhanced format is, therefore, possible through an
interoperable set of parameters (Figure 2): 1) Time; 2)
Space; 3) Communicative distance; 4) Dependence on
ICT tools and infrastructure; 5) Level of complexity.
COMPLEXIS 2021 - 6th International Conference on Complexity, Future Information Systems and Risk
110

Figure 2: Model of Complex Digital Learning Formats.
Based on the level of quarantine alert, adopted in
regions of Ukraine, different level of complexity was
utilized for educational activities in universities: e-
learning 1.0/2.0 (for red and orange zones of
quarantine) and hybrid learning (for yellow zones)
formats were implemented.
3.2 Complex Digitally Enhanced
Learning Formats Individual
Quality Assessment: Survey Results
Profiling and subsequent modelling of complex
digitally enhanced learning formats that permeated
the foreign languages acquisition landscape in the
timeframe of COVID-19 measures informed the
content and structure of the survey, conducted among
Oriental and European languages primary
stakeholders to assess the in-depth individual
experiences among the estimated parameters of
digitally enhanced distant learning models:
Complexity; Subjective efficiency; Information and
communication efficiency; Learning outcomes
efficiency.
Overall assessment of individual experiences and
quality estimation of e-learning and hybrid learning in
the framework of COVID-19 lockdown and
quarantine measures (March 2020 – January 2021) for
university programs of European and Oriental
languages – yielded the following representative
results.
The assessment of overall comprehensive
individual experience (or a take-away) of e-learning
and hybrid learning in the framework of COVID-19
lockdown by different groups of stakeholders was
conducted according to the 5-Point Likert Scale (a
response scale in which responders specify their level
of engagement with a statement or a parameter in five
points: (1) Strongly disagree; (2) Disagree; (3) Neither
agree nor disagree; (4) Agree; (5) Strongly agree
(Handbook, 2010). The extremum points for
evaluation were 1 (maximum negative overall
experience of e-learning and hybrid learning) and 5
(maximum positive overall experience of e-learning
and hybrid learning).
Students of Foreign languages programs of all
levels in regional universities of Ukraine evaluated
their comprehensive individual experience of e-
learning as predominantly 4 – mostly agreeable
(34,3% of respondents), 3 – Average (30,6%), 5 –
most agreeable – 21,3%.
The average estimation of the overall individual
satisfaction with e-learning and hybrid learning
experience on Oriental and European Languages
programs by students of regional universities of
Ukraine is 3>4 – average to mostly agreeable.
Qualitative assessment of the e-learning and
hybrid learning was conducted through the
retrospective evaluation of respondents’ individual
experience in order identify and select the preferred
features and elements of e-learning and hybrid
learning among the following: Ability to work from
home; Ability to customize of the study space (video
conferencing or LMS); Adaptability and flexibility of
the work schedule; Save time for commute; Parallel
performance of several tasks and activities;
Opportunity to technologically diversify educational
activity, educational materials and methods; Ability to
adapt the structure and content of curricula of
disciplines to the conditions of online learning; Ability
to activate and improve different types of soft skills;
Opportunity to improve digital literacy; Ability to
work independently in the learning process; No need
for constant interpersonal communication; Ability to
engage international professionals through digital
means.
Quantitative assessment demonstrated that
students of Foreign languages programs of all levels
have distributed preferred features and elements of e-
learning and hybrid learning and identified the
significantly top ranking ones as follows: 1) Ability to
work from home (75,5% of respondents); 2) Parallel
performance of several tasks and activities (50%); 3)
Adaptability and flexibility of the work schedule
(47,2%); 4) Ability to work independently in the
learning process (38,7%).
Most common preferred features of e-learning and
hybrid learning by all groups of students of Foreign
languages programs of all levels in regional
universities of Ukraine are correspond directly to the
framework complex skills (WEF, 2020), identified as
relevant for the job market of the 2021-2020 timespan
– flexibility, adaptability, active learning, time-
management.
Qualitative evaluation of respondents’ individual
experience through the span of March 2020 to
January 2021 allowed to identify the features and
Complex Framework of Digital Learning Quality Assessment in Covid-19 Context: Survey Study for European and Oriental Languages
Programs
111

elements of complex e-learning and hybrid learning,
considered drawbacks, challenges or a hindrance to
overall quality: Forced need to work at home
(uncomfortable living conditions); Impossibility /
difficulty of adaptation and flexibility of the work
schedule; The need for specialized technical means of
training and online communication / technical
limitations; Lack of interpersonal communication
with students and colleagues on a regular basis;
Emotional burnout; The need to activate and improve
different types of soft skills; The need to improve
digital literacy; Increasing the amount of workload in
preparation for training sessions and ensuring the
learning process; The need to adapt the structure and
content of curricula of disciplines in terms of online
learning; The need to organize and control the
independent work of students; Availability of
mechanisms for control and accounting of the
educational process.
Quantitative assessment distributed drawback
features and elements of e-learning and hybrid
learning as follows (Figure 3):
Figure 3: Drawback features and elements of e-learning and
hybrid learning.
Top features and elements of complex e-learning
and hybrid learning framework, considered
challenges or a hindrance to quality of education by
all respondents are: 1) Increased amount of workload
in preparation for training sessions (63,9%); 2)
Emotional burnout (54,6%); 3) Lack of interpersonal
communication on a regular basis (47,4%); The need
for specialized technical means of training and online
communication / technical limitations (32%).
Qualitative assessment of complex foreign
languages acquisition process by respondents allowed
to select the following indispensable elements of the
educational design: Conducting lectures; Conducting
practical classes; Development of training materials
and materials for assessment; Execution of tasks
(volumes, format, method of presentation);
Organization of group work of students; Organization
of independent work of students; Organization of
creative work of students; Organization of research
work; Communication with students (current);
Communication with students (advisory); Informal
communication outside the learning process /
mentoring and moderation; Assessment (formative,
summative, qualification).
Each element of educational design was subjected
to ranking by all groups of students according to the
5-Point Likert Scale in terms of its efficiency for e-
learning and hybrid learning quality assurance.
Respondents of Oriental and European languages
programs of all levels have identified the following
elements of the educational design and management
as ranking highest (5 – most conductive for quality of
e-learning and hybrid learning in individual
experience): Conducting lectures (52% of
respondents); Organization of group work of students
(42%); Summative assessment (38%); Organization
of independent work of students (29,5%);
Communication with students (current) (29,5%).
Respondents of Foreign languages programs of all
levels selected the e-learning and hybrid learning
educational process elements that are conductive to
quality of complex learning experiences outside of
the COVID-19 framework and are preferable to be
retained and implemented on a regular basis. The
following elements of hybrid and e-learning scored
highest as desirable to be retained in the complex
framework of the educational process past of the
COVID-19 emergency digitization measures: 1)
Electronic tests and assessments online – 70,8% of
respondents; 2) Use of e-textbooks and materials –
55,7%; 3) Online classes through video conferencing
systems – 54,7%; 4) Communication through
electronic resources – (mail, video, chat services,
social networks) – 53,8%.
Qualitative and quantitative comparative
assessment of individual experiences and quality of
e/hybrid learning in the framework of COVID-19
lockdown and face-to-face learning formats yielded
the following representative results.
Respondents of all groups were asked to compare
the individual experiences and estimations of quality
and efficiency of traditional, face-to-face in-presence
learning and hybrid or e-learning in the timeframe of
the COVID-19 lockdown (respondents were
instructed to disregard their individual assessment of
blended learning or online learning prior to 2020).
The comprehensive comparison of individual quality
assessment of traditional learning (TL) and e-learning
(EL) was ranked according to the 5-Point Likert Scale
(5 – maximum quality and efficiency, 1 – minimum
quality and efficiency) – Figure 4:
COMPLEXIS 2021 - 6th International Conference on Complexity, Future Information Systems and Risk
112

Figure 4: Comparison of individual experiences ranking of
traditional learning (TL) and e-learning (EL).
Respondents of foreign languages programs of all
levels have estimated the comparative framework
quality of traditional and e-learning in the following
way: 1) 4 – mostly qualitative and efficient (TL –
48%, EL – 40% of respondents); 2) 5 – most
qualitative and efficient (TL – 28%, EL – 26% of
respondents); 3 – average in quality and efficiency
(TL – 20%, EL – 25% of respondents); 4) 2 – low in
in quality and efficiency (TL – 4,1%, EL – 6% of
respondents); 5) 1 – poor in quality and efficiency
(TL –0%, EL – 4% of respondents). The average
score of framework quality and efficiency
comparison for traditional learning (TL) and e-
learning (EL) by all groups of students in regional
universities of Ukraine is 4 – mostly qualitative and
efficient. The assessment of e-learning ranks higher
in reverse correlation to the estimated quality of
traditional learning formats.
The respondents of all groups were prompted to
contrast their individual assessment e-learning and
hybrid learning against the prior to COVID-19
timeframe paradigm of experience of traditional in-
presence learning. In-presence learning (TL) was
estimated as a conventional 100% of potential quality
of education process and learning outcomes for
European and Oriental languages programs.
Accordingly, the respondents were to scale their
assessment of e-learning and hybrid learning quality
from 0% (no correspondence to TL quality in
individual experience) to 100% (full correspondence
to TL quality in individual experience) in 10%
increments.
According to survey data in regional universities
of Ukraine, e-learning and hybrid learning
satisfaction in the timespan of COVID-19 scores the
highest quality approximation to traditional learning
in such regional centers as: Kharkiv (East) – 85%
comparative score to TL; Lviv (West) – 80%
comparative score to TL; Zaporizhzhya (South) –
80% comparative score to TL; Dnipro (South-East) –
78% comparative score to TL. The accompanying
social, educational and technological factors that
contributed to this results are estimated as follows: 1)
Satisfactory to high level of digitalization in surveyed
universities (prior to quarantine measures); 2) High
level of blended learning implementation in the
classrooms of European and Oriental languages
surveyed; 3) High level of digital literacy of students
(individual and institutional); 4) High level of
international mobility prior to quarantine measures
that informed implementation of e-learning
techniques.
Respondents of all groups were prompted to
evaluate the efficiency of adaptation of each
educational design element, heretofore identified as
conductive to quality of e-learning and hybrid
learning format in the framework of COVID-19. The
scale of 5 (most efficiently adapted for complex e-
learning and hybrid learning format) to 1 (least
efficiently adapted for complex e-learning and hybrid
learning format) was applied. Respondent identified
the following elements of the educational design and
management as ranking highest (5 – most efficiently
adapted for e-learning and hybrid learning format): 1)
Conducting lectures (54% of respondents) 2)
Independent work of students (42% of respondents)
3) Assessment (summative, qualification) (35% of
respondents); 4) Practical classes/seminars (34% of
respondents); Training materials and materials for
assessment (34% of respondents). Of the elements of
educational design, identified heretofore as
instrumental to e/hybrid learning quality,
“Communication with students (current)” didn’t
score as efficiently adapted to the emergency digital
learning format.
Qualitative content analysis of the individual
experiences of foreign languages programs students
through hybrid and e-learning was conducted via
Voyant Engine (https://voyant-tools.org/) – a family
of digital cloud-based corpus and text-mining tools.
Each respondent submitted at least three (3) free
ranging concepts that were individually associated
with hybrid and e-learning through the COVID-19
pandemic timespan. The concepts (words and
phrases) and associations were originally supplied by
respondents in their native languages (Ukrainian and
Russian).
The key notions, thematically foregrounded in the
corpus sample, that designate individual experiences
of students in Oriental and European languages
programs of all levels in regional universities of
Ukraine:
ONLINE (frequency index 0,12); TIME
(frequency index – 0,078);
LACK (frequency index –
0,039);
SCHEDULE (frequency index – 0,039); WORK
(frequency index – 0,039);
CONVENIENCE (frequency
Complex Framework of Digital Learning Quality Assessment in Covid-19 Context: Survey Study for European and Oriental Languages
Programs
113

index – 0,025); COMMUNICATION (frequency index –
0,022);
STRESS (frequency index – 0,022). Identified
foregrounded concepts correspond to the dimensions
of the complex digitally enhanced learning formats
modelling.
The collocation content analysis of foregrounded
notions allows to identify the following predicate
frameworks that shape the subjective quality
assessment of complex hybrid and e-learning
experiences by students of foreign languages
programs in regional universities of Ukraine (Figure
5): 1)
LACK =>(of)=.> communication; 2) LACK
=>(of)=.>convenience; 3)
LACK =>(of)=.>
digital/online connection; 4)
LACK =>(of)=.>
resources; 5)
LACK =>(of)=.> teacher/mentor.
Figure 5: Collocation content analysis of individual e-
learning and hybrid learning experinces.
Preliminary qualitative assessment allowed to
identify and select the following generic challenges
that impaired effective and qualitative adaptation of
complex learning design for foreign languages
programs into hybrid and digital distant format: 1)
Social and psychological; 2) Technical challenges
and digital literacy; 3) Soft skills.
Students of foreign languages programs have
identified the following challenges and limitations of
efficient and qualitative complex transformation of
educational elements into hybrid and e-learning
format in their individual experience: 1) Technical
difficulties (lack of stable Internet connection, lack of
necessary equipment, capacity of household
computer equipment) – 68% of respondents; 2)
Fatigue – 59,6% of respondents; 3) Emotional
burnout – 45,2% of respondents; 4) Stress – 45,2% of
respondents.
Most of the challenges of the ‘digital literacy’ type
specifically feature as relatively insignificant in
individual experiences of students of all levels. Such
distribution of technical challenges testifies to the
phenomenon of digital gap (DQ, 2019; WEF, 2020),
pervasive in various areas of educational activities in
the framework of Covid-19 lockdown. That placed
students as an age group, as mostly representatives of
the “digital native” generation – a term by M. Prensky
(2001) – at a significant advantage in the hybrid and
digital educational process through the
implementation of emergency quarantine measures of
2020 and 2021.
4 CONCLUSIONS
The global pandemic and subsequent quarantine
measures and restrictions have significant influence
on the complex structure and procedures of higher
education workflow, which informed the scope of
individual experiences, projected outcomes and
estimated quality of higher education in the realm of
Foreign languages acquisition in all regions of
Ukraine.
The survey results on the individual experiences
and quality assessment of e-learning and hybrid
learning in the framework of COVID-19 lockdown
yield comprehensive data on the parameters and
challenges of e-learning and hybrid learning complex
transformation of Oriental and European languages
programs, common for students of higher educational
institutions of the regional scale and status across
Ukraine.
The qualitative assessment of the complex e-
learning and hybrid learning framework was
conducted through the retrospective evaluation of
respondents’ individual experience that helped
identify the comparative coordinates of positive and
negative dimensions of the latter. The invariant
positive quality indicators for e-learning and hybrid
learning across all groups of students surveyed are the
opportunity for multitasking, digital skills
improvement and lack of commute expenditure. The
invariant negative quality indicators for e-learning
and hybrid learning across all groups of students
surveyed across the regions of Ukraine are the lack of
interpersonal communication, technical impediments
and psychological states (fatigue, burnout, stress).
The contrastive exposure of individual
experiences and quality of e-learning and hybrid
learning in the framework of COVID-19 lockdown
against the backdrop of traditional, in-presence
learning formats identify e-learning and hybrid
learning as mostly to average comparative in quality
across all regions of Ukraine. Regional distribution of
contrastive assessment of hybrid and e-learning
permits to further disclose social, economic, cultural
and political factors, that inform the overall success
of educational formats adaptation for Oriental and
COMPLEXIS 2021 - 6th International Conference on Complexity, Future Information Systems and Risk
114

European languages programs in different regions
Ukraine.
Assessment of complex learning process design,
programmed leaning outcomes and projected
competences allowed to identify the types of
challenges that impaired effective and qualitative
adaptation of learning process design into hybrid and
digital distant format as predominantly socio-
psychological for students of all groups surveyed in
regions of Ukraine.
The presented study is limited in scope to the
indicative survey results, exemplifying the outlined
quality assurance dimensions of complex e-learning
and hybrid learning implementation for foreign
languages programs in regional universities of
Ukraine, induced by COVID-19 measures. The future
inquiry includes fine-tuned estimation of the in-depth
subjective quality assessment and subsequent
efficiency assessment of hybrid and e-learning in
different regions of Ukraine and Europe; contrastive
studies of Oriental and European languages programs
emergency digital distance format adaptations in the
COVID-19 timespan; efficiency assessment of ICT
tools instrumental to quality assurance of complex
digital and hybrid learning formats.
REFERENCES
Almalki, S. 2016. Integrating Quantitative and Qualitative
Data in Mixed Methods Research – Challenges and
Benefits. In Journal of Education and Learning, 5(3).
Aristotle, 2014. Categories. In The Complete Works of
Aristotle, 2 vols (One-Volume Digital Edition). Transl.
J. L. Ackrill. Princeton: Princeton University Press. p.
2510.
Bekele, T. A., 2010. Motivation and satisfaction in internet-
supported learning environments: A review. In
Educational Technology & Society, 13(2), pp. 116–127.
Bolliger, D. U. & Wasilik O., 2009. Factors influencing
faculty satisfaction with online teaching and learning in
higher education. In Distance Education, 30(1), pp.
103–116.
Boyarsky, K., 2020. What is Hybrid Learning. eThink.
Retrieved from: https://www.owllabs.com/blog/hybrid-
learning
Budko, V., 1990. Adekvatnost' nauchnogo poznanija.
Har'kov : Logos, 154 p.
Costareie, S., 2011. Assuaring quality based on the
assessment of learning outcomes. In 5th International
Technology, Education and Development. Conference
(INTED) Mar. 07-09, 2011, pp. 3235–3236.
Crowe, A. R., 2007. Learning to teach with mobile
technology: A teacher educator’s journey. In
Ubiquitous computing in education. Mahwah, New
Jersey: Lawrence Erlbaum Associates, pp. 127–144.
Dillman, D. A., Smyth, J. D., Christian, Leah Melani, 2014.
Internet, Phone, Mail and Mixed-Mode Surveys: The
Tailored Design Method, 4th edition. John Wiley:
Hoboken, NJ.
Dos Reis, A., 2015. To Be a (Blended) Teacher in the 21st
Century - Some Reflections. In International Journal of
Research in E-learning, 1(1), pp. 11–24.
DQ Global Standards Report, 2019. World’s first global
standard for digital literacy, skills and readiness
launched by the Coalition for Digital Intelligence.
Retrieved from: https://www.dqinstitute.org/ (accessed
January 2021).
Duff C., 2020. Everything you need to know about
education, technology and distance learning. eThink,
Retrieved from: https://www.owllabs.com/blog/hybrid-
learning
European Commission, 2020. European E-Competence
Framework Guideline. Retrieved from:
https://www.ecompetences.eu/ (accessed January 2021).
Gachev G., 1993. Gumanitarnyj kommentarij k
estestvoznaniju. In Voprosy Literatury. Issue 11, pp.
71–78.
Handbook of Disease Burdens and Quality of Life
Measures, 2010. Springer Science+Business Media,
pp. 4288–4288.
Heim, M., 1993. The Metaphysics of Virtual Reality. LA:
Westport Publishers.
Kasavin, I., 1990. Postigaja mnogoobrazie razuma.
Zabluzhdajushhijsja razum? In Mnogoobrazie
Vnenauchnogo Znanija. Moscow, pp. 67–79.
Kireev, G., 2008. O kartine mira. In Kosmologicheskoe
esse. Lebed', Issue 534, pp. 25–29.
Law of Ukraine, 2019. On Higher Education. Retrieved
from: https://zakon.rada.gov.ua/laws/show/1556-
18#Text (accessed January 2021).
Makhachashvili, R., 2020. Models and Digital Diagnostics
Tools for the Innovative Polylingual Logosphere of
Computer Being Dynamics. In Italian-Ukrainian
Contrastive Studies: Linguistics, Literature,
Translation. Monograph. Peter Lang GmbH
Internationaler Verlag der Wissenschaften, Berlin, pp.
99–124.
Makhachashvili, R., Semenist, I., Bakhtina, A., 2020.
Digital Skills Development And Ict Tools For Final
Qualification Assessment: Survey Study For Students
And Staff Of European And Oriental Philology
Programs. In Electronic Scientific Professional Journal
“Open Educational E-Environment Of Modern
University, (9), pp. 54–68. https://doi.org/10.28925/
2414-0325.2020.9.5
Makhachashvili, R., Semenist, I., 2021. Student
Satisfaction With Digital Hybrid Learning In European
And Oriental Languages Programs: Survey Study Of
Regional Universities Of Ukraine. In Proceedings Of
The 19
th
International Conference E-society, pp. 133–
143.
Markova, T. et al, E. 2017. Quality Issues of Online
Distance Learning. In Social and Behavioral Sciences,
Issue 237, pp. 685–691.
Complex Framework of Digital Learning Quality Assessment in Covid-19 Context: Survey Study for European and Oriental Languages
Programs
115

Prensky, M. 2001. Digital Natives, Digital Immigrants. In
On the Horizon, Vol. 9, No. 5.
Salyers, V., Carter, L., Carter, A., Myers, S., & Barrett, P.,
2014. The search for meaningful e-learning at Canadian
universities: A multi-institutional research study. In The
International Review of Research in Open and
Distributed Learning, Issue 15(6), pp. 313–347.
Smyrnova-Trybulska, E., Kommers P. et al (Eds), 2019.
Universities in the Networked Society. Springer.
Taleb, N., 2010. The Black Swan: The Impact Of The
Highly Improbable (2nd ed.). London: Penguin. ISBN
978-0-14103459-1.
Thiessen, J., Ambrock, V., 2011. Value Added – The Editor
in Design and Development of Online Courses. In The
Theory and Practice of Online Learning, AU Press,
Athabasca University. Retrieved from:
https://read.aupress.ca/read/the-theory-and-practice-of-
online-learning/section/7b34c0dd-89ad-4280-9c95-
137c1e2aa471#ch11
Trentin, G., 2010. Networked Collaborative Learning:
Social Interaction and Active Learning. Springer.
Tu, P. N. V., 1994. Dynamical systems : an introduction
with applications in economics and biology. Berlin
New York: Springer.
Van’t Hooft, M., Swan, K., Lin, Y-M., & Cook, D., 2007.
What is ubiquitous computing? In Ubiquitous
Computing in Education, pp. 3–17.
Voorn, R. J., & Kommers, P. A., 2013. Social media and
higher education: introversion and collaborative
learning from the student's perspective. In International
Journal of Social Media and Interactive Learning
Environments, Issue 1(1), pp. 59–73.
World Economic Forum, 2020. The Future of Jobs Report.
Retrieved from: http://www3.weforum.org/docs/
WEF_Future_of_Jobs_2020.pdf (accessed November
2020).
COMPLEXIS 2021 - 6th International Conference on Complexity, Future Information Systems and Risk
116