different tests. Presenting different tests to adjacent
students is important, since it is much easier for
students to cheat with MCQ tests.
The tests, implemented with Moodle, were
answered by the students as homeworks. This e-
assessment component was both of a formative
nature and of a summative nature. The summative
nature is present because the tests have a weight in
the final grades obtained by the students. The
formative nature is present because the tests were to
be answered by the students 7 to 10 days before the
formal tests, to help them ascertain their awareness.
These formal tests were the other component of the
assessment. It consisted of 3 MCQ tests answered by
the students in written format, at the same time for
all the students, in a date previously scheduled by
the school. It is relevant to say that these tests can
also be considered as a type of e-assessment, since
the marks were obtained and stored with electronic
support. The electronic support consisted of an excel
file with adequate formulas. The students answers
were converted to electronic format, grades were
automatically generated and stored in this format,
and statistics were calculated. It is important to say
that presenting the students different tests is a
problem also in these MCQ tests in written format.
At least 8 different versions were necessary for
each test.
The implementation was gradually done, in order
to carefully test the system. The first step consisted
in the development of a bank of MCQ. This bank of
MCQ was carefully planned and implemented to
allow that the tests were randomly generated by
Moodle, allowing that each student is presented with
a different test maintaining, even so, uniformity. The
construction of the MCQ bank is discussed in the
next section.
Next, it was decided to implement 3 tests as
homeworks, during specified periods previously
defined and communicated to the students. It was
intended to do the tests during the lessons, but there
were no technical conditions to do it. Due to these
circumstances, these MCQ tests implemented with
Moodle had initially a weight of 10%, being that the
3 tests MCQ tests in written format had the
remaining 90%. The decision for the weights was
considered good, since this was the first time that the
bank of MCQ was used and some uncontrollable
situations could happen. In addition, the tests were
optional to the students, and answered out of the
lessons environment. As stated before, its purpose
was mainly to serve as formative assessment.
This format was maintained during 3 academic
years, but with slight changes in format and in the
MCQ tests’ weights.
4 CONSTRUCTING THE
MULTIPLE-CHOICE
QUESTIONS BANK
The most important task of the e-assessment was the
construction of the MCQ bank. Three aspects were
considered namely, defining categories for the
questions, building the questions and the tests, and
the revision process. These aspects are explained
following.
4.1 Defining Categories for the
Questions
Moodle can generate tests randomly by selecting a
fixed number of questions from selected categories,
which render a different test for each student. This
poses two important questions:
how to guarantee that the tests assess the same
topics?
how to guarantee that the tests are uniform in
difficulty for all the students?
It was defined that the guarantee that the same
topics are assessed could be achieved with the
definition of categories in which to classify each of
the elaborated questions, each category
corresponding to a learning outcome. The learning
outcomes were carefully defined by the group of
teachers based in the learning outcomes of the
courses being taught. These were defined at the
beginning of each semester by the group of teachers,
based on the students’ necessities, but are somehow
uniform along the years. It was detected that if the
tests include more than one question from a
category, Moodle can select the same question two
times (at least), which is somehow common in
Moodle randomly generated tests. Thus, to avoid
this issue, the tests randomly generated by Moodle
presented to each student, includes only one
question per category, in order to avoid that the
same may question appear more than once in the
test. The categories to include in each test, and
consequently the learning outcomes that are meant
to be assessed, are defined by the teachers for each
test.
As for the guarantee that the tests are uniform in
difficulty for all the students, it was decided that the
teachers should develop questions with low to
medium difficulty level. The questions should also
be uniform in format: for instance it is not
acceptable to have a question with 3 options, and
another question with 7 options, since it is more
difficult for the students to analyze the late case.
e-AssessmentinMathematicsCourseswithMultiple-choiceQuestionsTests
263