The Formation Level Assessment of Foreign Language

Communicative Competence in Multi-level Groups

Nadezhda A. Zabelina

, Yulia I. Semenova

and Svetlana S. Sotnikova

Kursk State University, Kursk, Russia

Keywords: Assessment, Communication Competence, Higher Education, Conversation, Portfolio, Feedback, Testing.

Abstract: The article deals with the foreign language communicative competence formation level assessment

organization among the language faculty students at the university. The relevance of the problem is clear in

the conditions of training the specialists who are ready to solve professional and life tasks. The article analyzes

the possibilities of using the alternative means for evaluating the learning in higher education, and examines

the experience of using this complex at the Language Theory Department and Methods of Teaching Foreign

Languages Department in Kursk State University. The described complex includes writing an essay, keeping

a reflective diary, talking, etc. The article examines the results of a students survey in 2018 and 2020,

demonstrating the positive impact of the proposed set of alternative monitoring and evaluation tools on the

students attitude to learning a foreign language.

1 INTRODUCTION

One of the key areas of the modern Russian economy

is the education and training of specialists who are

ready to meet professional challenges of various

complexity (the program "Digital Economy in the

Russian Federation" (approved by Government

Decree No. 1632 of 28/07/2017)). Therefore, the task

of improving the foreign language education quality

in the context of higher education is more urgent than

ever (Bolotov, 2015).

The growing number of multi-level groups and

the current and intermediate control orientation

provided for in the curriculum and program to the

"average" student make it difficult to assess each

individual student results in the context of an

individual educational trajectory, which has led to a

decrease in the students motivation to learn foreign

languages over the past few years.

Obviously, there is a need to find and implement

non-traditional assessment forms that are designed to

demonstrate the real foreign language communicative

competence formation level among students, to track

their individual progress, as well as to create an

educational process comprehensive control, while

https://orcid.org/0000-0002-0573-8384

https://orcid.org/0000-0002-1486-5772

https://orcid.org/0000-0003-1402-7354

contributing to the motivation growth (Abramova,

2017).

Taking into account these requirements, the

Language Theory and Methods of Teaching Foreign

Languages Department of Kursk State University

(KSU) professors have created a set of monitoring

and evaluation tools. The priority task is to maintain

and stimulate students' interest in learning English at

various levels of mastery, taking into account the

students diversity.

A prerequisite for the using this complex can be

considered the current and intermediate rather low

student learning outcomes analysis, which, in our

opinion, is due to the factor of multilevelness within

one group, including citizens of near and far abroad,

as well as the generally accepted system for

monitoring and evaluation, which does not track

progress, difficulties and individual achievements.

2 RESEARCH METHODOLOGY

During the implementation, both general scientific

methods and empirical research methods were used,

406

Zabelina, N., Semenova, Y. and Sotnikova, S.

The Formation Level Assessment of Foreign Language Communicative Competence in Multi-level Groups.

DOI: 10.5220/0010669400003223

In Proceedings of the 1st International Scientiﬁc Forum on Sustainable Development of Socio-economic Systems (WFSDS 2021), pages 406-412

ISBN: 978-989-758-597-5

i.e. questionnaires, observation and experimental

training. The experimental training (2018-2020)

involved 360 students of the Foreign Languages

Faculty of KSU aged from 17 to 22 years and 8

teachers. 100 students participated in the experiment

to identify the impact of alternative tools for

monitoring and evaluating the the foreign language

competence development results and on the students

motivation to learn English. The experiment included

several stages.

3 RESEARCH RESULTS

At the first stage (2018), a questionnaire was

developed and conducted to determine the intended

areas of a foreign language use in future professional

activities, as well as to identify problems/barriers that

prevent the individual results improvement and, as a

result, reduce the motivation to master a foreign

language. 100 students of the Foreign Languages

Faculty of KSU (еру Teacher Education field) took

part in the survey. The results showed that 65% of

students study a foreign language in order to use it for

professional activities in a school or linguistic center

as a foreign language teacher, while 35% are going to

continue their studies abroad after graduation, do

research work, 25% plan to use a foreign language as

an additional skill to their main job (lawyer,

economist, manager). The question "Do you agree

that the test papers and the test/exam fully

demonstrate your individual results of mastering the

discipline?" was answered "affirmative" by only 25%

of students, and the question "Do you think that it is

necessary to use additional assessment tools

(reflection, portfolio, etc.)" was answered

"affirmative" by 75% of students. The survey results

analysis led to the conclusion that the majority of

students are not motivated to improve their English

proficiency level, primarily due to the lack of a clear

picture on individual results, the difficulties and gaps

in knowledge causes, as well as a clear understanding

of the future professional competencies application,

which prompted a change, including the approach to

monitor and evaluate the learning outcomes, to make

it more individual, which would increase the students

interest (Abramova, 2018; Bachman, 2004).

The second stage included the development and

testing of a set of tools for monitoring and evaluating

the foreign language communicative competence

results assessment.

In the unified control and evaluation system, the

monitoring, rating system, pedagogical tests,

portfolio, information test technologies, reflexive

diary, writing articles, projects, and the game method

were used (Kirianova, 2016; Semenova, 2020). Let's

look at some of them:

1. Monitoring.

Monitoring as a form of level of competence

formation control is understood as monitoring the

educational process state for the purpose of

assessment, control or prediction. Such monitoring

should be carried out systematically (Common

European Framework 2011).

In our case, this is a result didactic, statistical

monitoring, i.e. we need to track the knowledge, skills

and abilities acquisition in a particular discipline, the

level of compliance with the Federal State

Educational Standard requirements and statistically

show what was done by each student from the volume

of material planned by the teacher and what was

obtained as a result.

Monitoring is implemented in several stages. At

the initial stage, a cross-section of students residual

knowledge in the disciplines preceding the course

study is carried out. Such a diagnostic test allows to

get a complete picture of the necessary competencies

formation. Based on the test results, some

recommendations are developed for each student, and

the educational process is planned for a specific

discipline. A diagnostic test can be used to determine

how well students are prepared to learn a new

discipline, and what measures should be taken to fill

the gaps (Popham, 2009).

Then, at the next monitoring stage, students are

given intermediate tasks of various types, which test

the acquired knowledge and skills application. The

intermediate tasks results are also written in the table

for the formation of cumulative points at the end of

the discipline. This control allows to identify an

educational material assimilation objective picture of

each individual student and the group as a whole.

Based on the test results analysis, a professor

differentiates the educational process and develops

individual training routes. At the same time, the most

effective, purposeful adjustment of students

knowledge is carried out.

At the final stage, it is very important to conduct

a final exam test that allows to get an objective picture

of the students discipline assimilation as a whole and,

if possible, identifies the reasons why students did not

learn some program material topics.

The professor then prepares individual reports for

each student and a summary report for the group as a

whole. Such data allows to determine how well

students are prepared to learn the next discipline or

course. All statistics are written by a professor into

the matrix (Up-to-date Approaches 2019).

The Formation Level Assessment of Foreign Language Communicative Competence in Multi-level Groups

407

4 DISCUSSION OF RESULTS

Let's consider how monitoring is carried out on the

example of the discipline "Lexical and grammatical

aspects of foreign language communication", which

is studied by students of the 2-3 courses of the

Foreign Languages Faculty of KSU, the education

field is Pedagogical education. This discipline is

divided into two courses (2-3) and two semesters (4-

5). Here is an example of the students foreign

language communicative competence formation level

monitoring in the 5th semester of the 3rd year.

First, the students write a diagnostic test for the

residual knowledge they received in the 4th semester

of the 2nd year. Then, in the course of further

discipline study, they perform control tasks. All

results are written in the matrix as a percentage of

completed tasks or as points. The evaluation criteria

are developed for each type of task. For example, the

presentation of a dialogue on a topic under study can

be evaluated according to the following criteria:

pronunciation, vocabulary, grammar, communication

skills and compliance with a given topic. For each

criterion, the points ranking is given (see Table 1).

Table 1: Assessment criteria of level of foreign language competence for the discipline “Lexico-grammatical aspects of

foreign language communication”.

Poin

Pronunciation Vocabulary Grammar

Communication skills

skills

Discursive skills

5 Sounds and words

are pronounced

correctly,

intonation, pauses

and phrasal stress

correspond to the

norms, no more

than 1 phonetic

error is allowed

Vocabulary is at

high complexity

level with a

variety of

synonyms,

antonyms, the

ability to

paraphrase, 1-2

lexical errors are

allowe

Grammatical

forms are used

correctly, 1-2

grammatical

errors are

allowed in

complex

constructions

Supports and

develops

communication at a

high level, reacts to

the interlocutor's

remarks in accordance

with the topic

High level of ability

to logically construct

a statement, organize

a coherent text,

express own point of

view, 1 logical error

or 1 error in the

means of

communication is

allowe

4 Sounds and words

are pronounced

correctly in most

cases, intonation,

pauses and

phrasal stress

practically

correspond to the

norms, and no

more than 2-3

phonetic errors

are allowe

Vocabulary is at

advanced

complexity level

with a variety of

synonyms,

antonyms, the

ability to

paraphrase, 3-4

lexical errors are

allowed

Grammatical

forms are

mostly used

correctly, 3-4

grammatical

errors in

complex

constructions

are allowed

Supports and

develops

communication at a

good level, mainly

reacts to the

interlocutor's remarks

in accordance with the

topic

A good level of

ability to logically

construct a statement,

organize a coherent

text, express own

point of view, 2-3

logical errors or 2-3

errors in

communication tools

are allowed

3 Sounds and words

are mostly

pronounced

correctly,

intonation, pauses

and phrasal stress

do not always

correspond to the

norms, no more

than 4-5 phonetic

errors are allowed

Vocabulary is at

medium

complexity level

with the presence

of synonyms,

antonyms, the

ability to

paraphrase is not

very well

developed, 5-6

lexical errors are

allowe

Grammatical

forms are

mostly used

correctly, 5-6

grammatical

errors are

allowed in

complex

constructions

Supports and

develops

communication at an

average level, does

not always respond to

the interlocutor's

remarks in accordance

with the topic

Average level of

ability to logically

construct a statement,

organize a coherent

text, express own

point of view, 4-5

logical errors or 4-5

errors in

communication tools

are allowed

2 Sounds and words

are often

pronounced

incorrectly,

intonation, pauses

and phrasal stress

do not alwa

Vocabulary is at a

low complexity

level with a rare

use of synonyms,

antonyms, the

ability to

ara

hrase is not

Grammatical

forms are often

used incorrectly,

7-8 grammatical

errors are

allowed in

com

lex

Supports and

develops

communication at a

low level, does not

react to the

interlocutor's remarks

Low level of ability to

logically construct a

statement, organize a

coherent text, express

own point of view, 6-

7 logical errors or 6-7

errors in

WFSDS 2021 - INTERNATIONAL SCIENTIFIC FORUM ON SUSTAINABLE DEVELOPMENT OF SOCIO-ECONOMIC SYSTEMS

408

correspond to the

norms, no more

than 6-7 phonetic

errors are allowe

developed, 7-8

lexical errors are

allowed

constructions or

5-6 errors in

simple

constructions

in accordance with the

topic

communication tools

are allowed

1 Sounds and words

are often

pronounced

incorrectly,

intonation,

pauses, and

phrasal stress do

not correspond to

the norms, no

more than 8-9

phonetic errors

are allowe

Vocabulary is at

low complexity

level, synonyms,

antonyms are

absent, the ability

to paraphrase is

not developed, 9-

10 lexical errors

are allowed

Grammatical

forms are used

incorrectly, 9-10

grammatical

errors are

allowed in

complex

constructions or

7-8 errors in

simple

constructions

Does not know how

to maintain and

develop

communication, does

not respond to the

interlocutor's remarks

in accordance with the

topic

Does not know how

to logically construct

a statement, organize

a coherent text,

express own point of

view, 8-9 logical

errors or 8-9 errors in

the means of

communication are

allowed

0 Speech is

unintelligible.

Sounds and words

are pronounced

incorrectly,

intonation, pauses

and phrasal stress

do not correspond

to the norms, no

more than 10-11

phonetic errors

are allowe

Vocabulary is at

the basic

complexity level,

synonyms,

antonyms are

absent, the ability

to paraphrase is

not developed,

more than 11

lexical errors are

allowed

Grammatical

forms are used

incorrectly,

more than 11

grammatical

errors are

allowed in

complex

constructions or

more than 9

errors in simple

constructions

Does not know how

to maintain and

develop

communication, does

not respond to the

interlocutor's remarks

in accordance with the

topic, does not

understand the

interlocutor's speech

Does not know how

to logically construct

a statement, organize

a coherent text,

express own point of

view, more than 10

logical errors or more

than 10 errors in the

means of

communication are

allowed

The points for each criterion are summed up and the

arithmetic mean score for the entire task is output: 5

(pronunciation) + 4 (vocabulary) + 4 (grammar) + 5

(communication skills) + 5 (discursive skills) = 23

points should be divided by 5= 4.6. The result is

written in the matrix (see Table 2).

Table 2: Record of results of foreign language competence formation for the discipline “Lexico-grammatical aspects of

foreign language communication”.

№

Student's full name

Tasks for students

Diagnostic test

Listening skills 1

Dialogue on topic 1

Summary 1

Text retelling 1

Monologue on topic

Essay

Text analysis 1

Intermediate result

…..

Grammar Test

Final result

1 student 1

5 4.9 5 4.8 5 4.5 4.7 5 4.8

2 student 2

4.8 5 5 4.5 5 4.3 4.2 4.9

4.7

3 student 3

3.6 4.1 4 3.8 4 3.9 3.5 4.2 3.8

4 student 4

2.8 3.4 4 3 3 3.1 2.5 3.6 3.1

5 student 5

4.5 4.8 4.2 4 5 4.6 4.7 4

4.4

6 student 6

3.2 3.2 3.6 3.5 3 3.4 3.7 3.3 3.3

At the end of the topic or section study, an

intermediate result is summed up. Similarly, the work

on the next topic or section is organized. Then the

final result is calculated.

The Formation Level Assessment of Foreign Language Communicative Competence in Multi-level Groups

409

The intermediate result is necessarily discussed in

an individual conversation between the professor and

each student. During the conversation, the problems

faced by the student are identified, the causes of these

problems are analyzed, recommendations are

formulated and solutions are outlined.

2. Rating assessment system.

The rating system is designed to identify the

student studying level at each stage of the educational

process, as well as to differentiate the grades received

by students for performing various types of work

(current and final control, home work, independent

work, etc.) (Belozertseva, 2015).

The rating system involves the development of a

number of modules that will form an assessment of

the level of competence formation among students. In

the case of studying a particular discipline, for

example, "Lexical and grammatical aspects of foreign

language communication", the following modules

can be distinguished: academic, motivational,

Olympic, leadership.

For each module, the highest score is set at 20 and

the significance coefficient, depending on the degree

of the module importance in the overall assessment

system. The final score for the discipline is formed on

the basis of the results for each module, expressed as

a percentage and then transferred to points on a 5-

point scale.

The summary table of students achievements may

look as follows (see Table 3).

Table 3: Summary of students’ achievements for the discipline “Lexico-grammatical aspects of foreign language

communication”.

Student's

full name

Academic module

20 points

the coefficient is 1.5

Motivational module

20 points

the coefficient is 0.5

Olympic module

20 points

the coefficient is 0.5 –

3.0

Leadership module

20 points

the coefficient is 1.0

Final score

100 points

The final score for

topic 1

(10)

The final score for

topic 2

(10)

Attendance rate

(5)

Activity in the

classroom

(10)

Preparation for

classes

(5)

Participation in

English language

Olympiads and

competitions.

(10)

Participation in

scientific

conferences

(

)

Teaching English

to schoolchildren

(10)

The participation

in the faculty

career guidance

activities (10)

Student 1 10 10 5 10 5 10 10 10 10 100

Student 2 10 8 5 8 4 - - 10 5 75

For the academic module, the student can get 20

points maximum. These points are made up from the

final grades for each topic. Next, the module

significance factor is applied for the discipline

development. In this case, the coefficient is set to 1.5,

i.e. the points received are multiplied by the

coefficient. A student who gets 20 points will

eventually have 30 points. For the motivational

module, the coefficient is 0.5, since attendance,

preparation for classes and activity are undoubtedly

important, but can not overtake academic knowledge.

A student who scores 20 points in this module will

eventually have 10 points.

In the Olympic module, a high coefficient was set,

since here it is assumed that the student knows the

discipline very well and is able to compete with other

students at the regional, federal or international level.

In this case, the coefficient is different: for the 1st

place it is 3.0, for the 2nd place it is 2.0, for the 3rd

place it is 1.0. For participation in the event the

coefficient is 0.5. Some extra points can be added for

the event level: + 1 point for the regional level, + 2

points for the federal level, and + 3 points for the

international level. The leadership module involves

the participation in the English language activities

with students. Therefore, the coefficient here is

average. As a result, a student can score 100 or more

points.

The points received are transformed on a 5-point

scale into an assessment that corresponds to the

"credited" rating when setting the score on a test or

can be set as an exam score for the studied discipline.

3. Keeping a reflective diary.

A means of developing students' constructive

skills is a reflexive diary, which contributes to the

students predictive, reflexive, and self-assessment

skills formation and development. These skills can be

tracked in the following sections: "My learning

WFSDS 2021 - INTERNATIONAL SCIENTIFIC FORUM ON SUSTAINABLE DEVELOPMENT OF SOCIO-ECONOMIC SYSTEMS

410

opportunities and forecasting" (fixed at the beginning

of the month); "My learning opportunities and the

learning activities result" (at the end of the observed

period); "Final self-assessment of the opportunities

level in academic subjects", etc. The self-analysis and

self-assessment skills development dynamics are

recorded in the sections: "Classes reflexive analysis",

"Achievements for the week", "The educational

results self-analysis". Working with a reflexive diary

is expected for a week daily in all disciplines

(Baranova, 2015; Alderson, 2006) (see Table 4).

Table 4: Reflection diary for the discipline “Lexico-grammatical aspects of foreign language communication”.

Date Lesson to

ic Questions M

answe

Recommendations to m

self

17/09/

2019

The modal

verb "can"

1. What functions of

the modal verb do I find

difficult?

2. What functions of

the modal verb do I

consider simple and can

I use them in speech?

3. I know what

infinitive forms to use

after a modal verb.

4. What did I not

understand?

5. How did I work?

1. Strong doubt, astonishment.

2. Mental, physical, circumstantial

ability, permission, prohibition.

3. Yes, a simple infinitive for

mental, physical, circumstantial

ability, permission, prohibition,

a simple or perfect infinitive after

can/could in the strong doubt,

astonishment function.

4. It is difficult to construct a

sentence with an expression of

doubt in a negative form.

5. I have understood the theory, but

I need more

ractical tasks.

1. To create a table with

the forms of the modal verb

in different functions with

examples.

2. To perform 5-10

exercises of different types

on this material.

3. To apply the modal verb

in different functions in

speech in other English

language speech practice

classes.

Answering the questions, students control their

grammatical material assimilation, determine the

points that are difficult for them and require

improvement, and determine the further movement

vector.

"Non-marking" (qualitative) assessment:

1. Game method (multimedia educational game,

board game).

The game method of assessing the students'

competencies formation has proven itself well. When

studying the discipline "Lexical and grammatical

aspects of foreign language communication",

students independently developed various

multimedia and board games. Especially popular

among students is the multimedia platform Kahoot,

using which they create various quizzes or games.

First, the form is selected: with or without a

playing field. Games with a game board like "Snakes

and Ladders" allow to exercise and test both lexical

and grammatical skills. The playing field is created

by the students themselves, then tasks and steps are

developed. So, for example, the game field called

"Planes and trains" looks like this. Tasks for the game

included questions on English modal verbs

knowledge repetition and control (Romanova, 2017).

Players make moves and perform various kinds of

grammar tasks presented on the cards. The color of

the circle on the map corresponds to the color of the

card. Cards contain different types of tasks: make

sentences..., ask a question to the sentence, put the

word in the plural form, etc.

Figure 1: The game field "Planes and Trains"

The game process revealed all the "gaps" in the

students knowledge on the topic, allowed them to

understand the complex cases of using modal verbs,

and to acquire new knowledge in a relaxed way

(Cockey 2014; McCaslin, 2009).

Another variant of the board game is grammar

cards for modal verbs. On one map there is a picture

and the beginning of the sentence, on the other there

is a possible variant of the end of the sentence. For

example:

The Formation Level Assessment of Foreign Language Communicative Competence in Multi-level Groups

411

Figure 2: Example of a grammar map for modal verbs.

Obviously, the students not only have a stake in

the result of creating the game, but also control each

other's knowledge.

2. Participation in various events (conferences,

grants, etc.).

At the third stage of the study (2020), a survey and

questionnaire were conducted in order to identify the

effectiveness of the used set of additional tools for

monitoring and evaluating the foreign language

competence formation level among students. 100

students of the faculty took part in the survey and

questionnaire containing questions of open and

closed types. It should be noted that in comparison

with the results of the survey and observation in 2018,

there was a positive trend. 65% of respondents stated

the need to improve their foreign language

proficiency for academic and professional purposes,

to continue their master's studies at foreign

universities, etc. 70% of respondents noted reflexive

diary and monitoring as effective ways to evaluate

individual results. 80% answered positively to the

question "Do you think the rating system is effective

in learning a foreign language?".

5 CONCLUSIONS

The results of the study indicate that the proposed set

of additional tools for monitoring and evaluating the

foreign language competence formation level gives a

clearer picture of the learning outcomes within the

students individual educational trajectory. The

students achievements are their personal acquisitions,

their individual progress in mastering a foreign

language, the personal and professional competencies

development, a clear understanding of knowledge

gaps, their causes and ways to eliminate them. The

use of various assessment formats that track personal

development and individual progress, the necessary

competencies formation level increases the degree of

assessment individuality, which, without a doubt,

stimulates the growth of interest in the subject, as well

as motivates more in-depth and purposeful students

work. The proposed formats for evaluating the

foreign language communicative competence

formation results in other areas of training are the

prospect of further research in the field of effective

methods in higher education control.

REFERENCES

Abramova, I.E., Ananjina A.V., Sherekhova O.M.,

Shishmolina E.P., Yusupova L.N. (2017). Alternative

Ways of Assessing Results of Teaching English to

Students of Non-Linguistic Specialties. Petrozavodsk.

102.

Abramova, I.E., Shishmolina, E.P. (2018). Alternative

Assessment and Evaluation. Baltic Humanitarian

Journal, Т. 7, 3(24).

Alderson, J.Ch. (2006). Language Test Construction and

Evaluation. CUP, 245.

Bachman, L.F. (2004). Statistical Analyses for Language

Assessment. CUP, 364.

Baranova, O.I. (2015). Reflexive Diary of a Student of

Pedagogical Institution in terms of Acmeological

Approach. Modern Scientific Technologies, 12 (3):

471-475.

Belozertseva, N.V. (2015). Specifics of Control in

Teaching Foreign Language and Culture to Students in

terms of Competence Approach. Competencies and

Education: Models, Methods, Technologies. Moscow:

Centr nauchnoj misli, 7–40.

Bolotov, V.A. (2015). Russian Experience of Quality

Assessment of Education in International Context.

Cockey, S.W. (Ed.). (2014). Teaching World Languages: a

Practical Guide. Washington: NCLRC. 2014.

Common European Framework of Reference for

Languages: Learning, Teaching, Assessment. (2011).

Council of Europe, CUP.

Kirianova, E.A. (2016). Portfolio: Development of

Authentic Assessment Way. Elekronnii zhurnal

“Koncept”, 13: 31-37.

McCaslin, M. (2009). Co-regulation of Student Motivation

and Emergent Identity. Educational Psychologist, 44

(2): 137-146.

Popham, W.J. (2009). Assessment Literacy for Teachers:

Faddish or Fundamental? Theory into Practice, 48: 4–

11.

Romanova, A.V. (2017). Practically-oriented Teaching

Technology of Board Games Design in Foreign

Languages. Scientific Research of Young Researchers.

Electronic Journal, 1 (5).

Semenova, Yu.I. (2020). Basics of Designing Electronic

Educational Resources in Teaching Foreign Languages.

Scientific Notes. Electronic Scientific Journal of Kursk

State University. 1 (53).

Sorokovikh, G.V., Zharkova, T.I., Gerasimova, A.M. and

others (2019). Up-to-date Approaches to Foreign

Language Competence Assessment. Theory and

Practice, Moscow: Perspektiva, 279.

WFSDS 2021 - INTERNATIONAL SCIENTIFIC FORUM ON SUSTAINABLE DEVELOPMENT OF SOCIO-ECONOMIC SYSTEMS

412