Teacher’s Perspective on e-Assessment: A Case Study from Germany

Bastian K

uppers

1,2 a

and Ulrik Schroeder

1 b

Learning Technologies Research Group, RWTH Aachen University, Ahornstraße 55, Aachen, Germany

IT Center, RWTH Aachen University, Seffenter Weg 23, Aachen, Germany

Keywords:

Electronic Examinations, Electronic Assessment, Computer Based Assessment, Bring Your Own Device.

Abstract:

In order to verify common ﬁndings in the literature regarding teacher’s (pre)conception of e-Assessment in

general and e-Assessment on students’ devices, we carried out a survey among teachers of two institutes of

higher education in Germany. From the achieved results, it can be concluded that teachers seem to be open-

minded regarding e-Assessment in general. However, major concerns were mentioned regarding the fairness

and security of e-Assessment on students’ devices. These concerns have to be tackled and clearer in order to

successfully establish e-Assessment as an integral part of the examination landscape in higher education.

1 INTRODUCTION

The attitude of the examiners towards electronic as-

sessment (EA) is rather cautious according to avail-

able literature (Rolim and Isaias, 2018). A main rea-

son for this is the possibility of fraud, which, as Mel-

lar et al. have found, is likely to increase, especially

in bring your own device (BYOD) scenario:

Teachers in all four contexts thought that there

would be an increase in cheating, with the

teachers in the distance education context be-

ing the most concerned. However, teachers

also described a number of ways in which

cheating might be reduced through assess-

ment design, and the opportunities that e-

Assessment offered for increased control of

the assessment process.

(Mellar et al., 2018, p. 19)

The last segment of the quote is in line with

another study by Peytcheva-Forsyth et al., which

comes to the conclusion that “the role of technolo-

gies for prevention (rather than occurrence) of cheat-

ing and plagiarism in the assessment is emphasized”

(Peytcheva-Forsyth et al., 2018). In general, there

is little information on the examiner’s view on e-

Assessment or digital teaching in general as is pointed

out by Thoring et al.:

https://orcid.org/0000-0002-5882-2125

https://orcid.org/0000-0002-5178-8497

The number of papers and studies which fo-

cus on the digitalization in higher education

has increased recently. Most of them focus

on the situation of students [. . . ]. As Cope

and Ward [. . . ] point out, not just the stu-

dents’ perspective is important but also the

perspective of the teaching person. While a

good overview of the status quo of digital-

ization in the US is provided by the annual

ECAR studies by EDUCAUSE – quantitative

surveys conducted with approximately 50,000

students [. . . ] and 13,000 lecturers [. . . ] –,

comparable statistics from Europe are miss-

ing. In Germany, the discourse still is driven

mainly by politics, not science, and in conse-

quence publications are often working papers

with a normative character or guidelines based

on experts’ opinions [. . . ].

(Thoring et al., 2018, pp. 295-296)

Hence, a survey was carried out to understand

the examiner’s view on e-Assessment and BYOD and

which factors inﬂuence it. This papers describes this

research and is organized as follows in the second sec-

tion, we give a brief overview of the ﬁndings already

presented in the literature. In the third section, we

discuss the setup of our survey, followed by a discus-

sion of the achieved results in the fourth section. The

paper closes with a summary and an outlook.

Küppers, B. and Schroeder, U.

Teacher’s Perspective on e-Assessment: A Case Study from Germany.

DOI: 10.5220/0009578105030509

In Proceedings of the 12th International Conference on Computer Supported Education (CSEDU 2020) - Volume 1, pages 503-509

ISBN: 978-989-758-417-6

503

2 RELATED RESEARCH

As already pointed out, there is only little related re-

search available. Thoring et. al. come to the conclu-

sion that “E-assessment is of little importance in most

disciplines, but has become a standard at the Depart-

ment of Medicine [of M

unster University] where ex-

ams are usually multiple-choice. Since there is a lot

of criticism of multiple-choice test, e-assessment pro-

cedures are currently reﬁned to be able to test practi-

cal knowledge (e.g. by using a digital microscope).”

(Thoring et al., 2018). While this conclusion is drawn

from a qualitative study, it still hints to why there is

so little research available: If the teachers do not rec-

ognize e-Assessment as a valuable tool, they do not

bother to deal with their own perspective on it.

Jamil et. al. conducted a survey to be able to

compare computer-based (CB) and paper-based (PB)

examinations. Their ﬁndings draw a rather positive

picture on the teacher’s view on EA. They state that

“CB examinations saves time and also facilitate the

students to improve their understanding which ulti-

mately improve their GPA therefore a country-wide

policy should be prepared at university level regard-

ing CB examinations”(Jamil et al., 2012). They found

that all of the personal attributes they took into account

do inﬂuence the view on e-Assessment in a certain

way. However, the ﬁeld of expertise (“Department”),

the qualiﬁcation and the experience with technology

(“Computer Training Certiﬁcate”) seem to have the

biggest inﬂuence. This is in line with the ﬁndings

by Imtiaz and Maarop who state that “[t]he lectur-

ers have a positive intention towards e-assessment use

and would use it in the future” (Imtiaz and Maarop,

2014).

On a different note, Kuikka et. al. come to the

conclusion that “Learning systems are not necessar-

ily created based on teachers’ needs but based on the

creativity of developers. An approach where systems

are developed starting from pedagogical and teach-

ing perspectives - how to teach new skills- should

be promoted. This eases teachers’ entry into using

the systems, and creates systems with better usabil-

ity” (Kuikka et al., 2014). Due to their investigation,

they come to the conclusion that “Timesaving is cru-

cial for teachers. Features such as automatic evalua-

tion and sharing of exercises help teachers save time.

In order to utilise this fully, teachers can no longer

work by keeping their questions private but instead co-

operation between teachers is necessary. Moreover, in

order to get teachers to use e-exam more widely,it is vi-

tal to provide support and to reserve enough time for

them for the introduction of e-examination.” (Kuikka

et al., 2014). This is a ﬁrst hint at things that teachers

consider important.

3 DESIGN OF THE SURVEY

We constructed a survey to answer our research

question:

Q: Which factors inﬂuence the teacher’s perspec-

tive on e-Assessment?

We anticipated that the perspective on e-Assessment

is inﬂuence by the following attributes:

• Gender [G]

• Age [A]

• Technology Afﬁnity [TA]

• Field of Expertise (e.g. STEM) [FE]

• Teaching Experience [TE]

• Institution (University vs. University of Applied

Sciences) [I]

These factors were adapted from our study on the

students’ perspective on e-Assessment (K

uppers and

Schroeder, 2019). Since we expected that the results

would also be inﬂuenced by the general afﬁnity of the

teachers for technology, we needed to be able to dis-

tinguish between teachers with an afﬁnity for tech-

nology and those without. To measure the technol-

ogy afﬁnity, the TA-EG questionnaire by Karrer et

al. (Karrer et al., 2009) was used. The items of the

TA-EG questionnaire have been rearranged to elimi-

nate effects that might be caused by the clustered an-

swers of the original questionnaire. Furthermore, we

wanted to conduct the survey at several IHEs and for

different study programmes, in contrast to the exist-

ing surveys. This resulted in the, originally German,

survey which is available in the appendix. The survey

was carried out amongst teachers at RWTH Aachen

University and FH Aachen University of Applied Sci-

ences. Due to the wide range of teachers at two insti-

tutes of higher education, no assumptions about the

characteristics of this group can be made. For exam-

ple, there is no information available on how informed

on e-Assessment the teachers were prior to the survey.

4 ANALYSIS OF THE RESULTS

In total, 110 teachers responded to the survey with de-

mographics (A, G) as shown in Table 1. About three

quarters of the participating teachers are male, a lit-

tle less than one ﬁfth is female and about ﬁve percent

consider themselves none of the former. Regarding

the age distribution, about ﬁve percent are younger

than 30, two ﬁfths are between 30 and 50 years old

and the rest is older than 50 years. The teaching areas

CSEDU 2020 - 12th International Conference on Computer Supported Education

504

Table 1: Demographics of the participating teachers.

Male Female Diverse Σ

< 30 0.9 % 2.73 % 0 % 3.63 %

30–50 26.36 % 11.82 % 1.82 % 40 %

> 50 50 % 4.55 % 1.82 % 56.37 %

Σ 77.26 % 19.1 % 3.64 % 100 %

Figure 1: Distribution of the most important FE.

of the participants span a variety of ﬁelds, as Figure

1 shows. However, the majority of the participants

teaches a STEM topic (68.17 %) and has been teach-

ing for over 20 semesters (66.39 %), as shown in Ta-

ble 2.

The TA-EG questionnaire covers a suitable set of

features for performing a cluster analysis on. Figures

2 and 3 depict comparisons between the two clusters

that have been revealed by the cluster analysis us-

ing a k-medoids algorithm (Kaufmann, 1987; Jin and

Han, 2011). Figure 2 shows a comparison between

the minimum and maximum values of both clusters.

Figure 3 shows a comparison of the median of both

clusters.

Figure 2: TA-EG: Clustering 1.

Additionally, the data set was split into subsets to

determine the inﬂuence of gender, age, ﬁeld of ex-

pertise, teaching experience and institution. These

Figure 3: TA-EG: Clustering 2.

subsets were then analyzed for signiﬁcant differences

with a χ

Test for r × c Tables (Sheskin, 2003, pp.

493-572). The resulting p-values for the Likert-scaled

items can be found in Table 3.

Given these p-values, conclusions on the inﬂu-

ence of gender, age, technology afﬁnity, ﬁeld of ex-

pertise, teaching experience and institution can be

drawn to a certain extent. As it turns out, gender and

age have a statistically signiﬁcant inﬂuence The data

indicate that male teachers are more convinced that

a BYOD approach can be beneﬁcial for assessment

than other groups. The age of the examiners inﬂu-

ences their view on question E3. The data suggests

that with increasing age, the preconceptions regard-

ing e-Assessment rise as well. For question E3, the

distribution of the age leads to the conclusion that ex-

aminers of ages above 50 do not see e-Assessment

as a suitable replacement for paper-based examina-

tions, whereas younger teachers have a rather posi-

tive view on e-Assessment replacing paper-based ex-

aminations. As already mentioned, the cluster anal-

ysis revealed two clusters that were derived using a

k-medoids clustering algorithm (Kaufmann, 1987; Jin

and Han, 2011). These clusters represent one group

that clearly has high afﬁnity to technology (Cluster

0) and another, more reserved group (Cluster 1). For

these clusters, statistically signiﬁcant differences be-

tween the clusters were found for question E1. The

participants in Cluster 0 have a more positive atti-

tude towards e-Assessment than the participants in

Cluster 1. The ﬁeld of expertise has a signiﬁcant

inﬂuence on question E3. Teachers from the ﬁelds

of medicine and social Sciences see e-Assessment as

a suitable replacement for paper-based examinations.

Teachers from other ﬁelds (humanities, engineering,

sciences) are more reluctant regarding e-Assessment

Teacher’s Perspective on e-Assessment: A Case Study from Germany

505

Table 2: TE (in semesters) and FE of the participating teachers.

1 − 5 S. 6 − 10 S. 11 − 20 S. > 20 S. [NA] Σ

Engineering 4.55 % 7.23 % 4.55 % 19.1 % 0.91 % 36.34 %

Medicine 0.91 % 0 % 0.91 % 10 % 0 % 11.82 %

Sciences 0 % 0.91 % 7.27 % 23.65 % 0 % 31.83 %

Social Sciences 0 % 0.91 % 4.55 % 4.55 % 0 % 10.01 %

Other 0 % 0 % 0 % 2.73 % 0 % 2.73 %

Humanities 0 % 0 % 0 % 6.36 % 0 % 6.36 %

[NA] 0 % 0 % 0.91 % 0 % 0 % 0.91 %

Σ 5.46 % 9.05 % 18.19 % 66.39 % 0.91 % 100 %

Table 3: p for the χ

Test.

G A TA FE TE I

E1 0.98 0.17 0.002 [< α

0.01

] 0.11 0.49 0.73

E2 0.32 0.52 0.23 0.42 0.84 0.66

E3 0.17 0.014 [< α

0.05

] 0.69 0.024 [< α

0.05

] 0.44 0.59

B1 0.005 [< α

0.01

] 0.15 0.06 0.14 0.51 0.16

C1 0.75 0.22 0.19 0.28 0.72 0.73

C2 0.59 0.33 0.46 0.38 0.63 0.21

replacing paper-based examinations. The reason for

this might well be the present examination policies in

these ﬁelds. For example, as “[m]ultiple-choice ques-

tions [. . . ] are still used in high stakes exams world-

wide to assess the knowledge of medical students”

(Freiwald et al., 2014, p. 1), it is easy to see how

teachers in medicine using this mode of examinations

perceive it as suitably replaceable by EA. The exam-

iners generally agree that e-Assessment is a good sup-

plement to paper-based examinations, while their es-

timate on the opportunities of cheating does not show

a clear tendency. However, there seems to be a con-

sensus that it is easier to cheat in e-Assessment than

in a paper-based examinations.

Perceived Advantages and Disadvantages of e-

Assessment. From the teachers’s perspective, the

main advantage of e-Assessment is faster correction

(91.8%), while aspects like more realistic examina-

tions (15.5%) and more diverse examination tasks

(30%) are not as relevant. It turned out to be use-

ful to cluster and analyze the free text comments to

get a clearer picture of the examiners’ opinion on EA.

The examiners submitted 71 comments in total, in-

cluding 25 comments regarding the advantages and

46 comments concerning the disadvantages. The re-

sults of clustering the comments appear in Figures 4

and 5. The clusters of the advantages are intercon-

nected, i.e. they affect each other. For example, bet-

ter readability of exam answers leads to simpler cor-

rection. The same holds for the clusters of the dis-

advantages, where for example inappropriate assign-

ments can lead to stress. It is important to note that

the number of clusters that were created for the dis-

advantages exceeds the number of clusters found for

Advantages

Readability

Exam Management

Learning - &

Assessmentprocess

Environmental

Factors

Archival

Innovative

Assignments

Correction

Figure 4: EA: Free Text Comments about Advantages.

Disadvantages

Accessibility

Cheating

Organizational

Matters

Stress /

Concentration

Writing

Technical Problems /

Hardware

(Automated)

Correction

Assignment / Exams

Inappropriate

Figure 5: EA: Free Text Comments about Disadvantages.

the advantages. This could be caused by the positive-

negative asymmetry (Pietri et al., 2013; Baumeister

et al., 2001; Mittal et al., 1998), yet this can not be

safely concluded from the data.

CSEDU 2020 - 12th International Conference on Computer Supported Education

506

Disadvantages

Compulsory Device

Responsibility

Cheating

Miscellaneous

Differences

Figure 6: BYOD: Free Text Comments about Disadvan-

tages.

Figure 7: BYOD: Shares of Clusters of Disadvantages.

Perceived Advantages and Disadvantages of

BYOD. According to the data collected, teachers

have a fairly clear view of e-Assessment in a BYOD

scenario: they do not support it. In total, only eleven

comments on the beneﬁts of BYOD were made in

the survey. However, four of these comments stated

things like no beneﬁts (no beneﬁts) or no beneﬁts

(none), leaving only seven positive comments. In

contrast, a total of 36 comments were made on the

disadvantages of BYOD. The resulting clustering and

the proportions of each cluster for the disadvantages

are shown in the ﬁgures 6 and 7.

5 CONCLUSION

The data collected lead to several aspects that the

examiners consider to be critical. The examiners

see several positive aspects of EA, such as better

exam management, including student administration

and statistics, and innovative tasks, possibly includ-

ing multimedia. On the other hand, examiners are

concerned about certain tasks or even entire exams

that are not suitable for an electronic environment,

or about problems with organizational matters. Over-

all, the positive and negative aspects of e-Assessment

balance each other out, but when it comes to e-

Assessment in a BYOD setting, examiners clearly re-

ject this idea. Almost no positive aspects were men-

tioned by the examiners, and most negative aspects

concern the possibility of fraud. It is therefore ex-

tremely important to address these issues in the con-

cept of a framework to convince the examiners that

e-Assessment can succeed in a BYOD setting without

opening the doors to fraud. From the data collected,

the following list of important aspects to be consid-

ered was derived:

1. An e-Assessment software has to . . .

(a) . . . offer a way to carry out management com-

fortably (register students, create assignments,

correct, archive, ...).

(b) . . . offer a software interface to implement and

adapt types of assignments to the examiners’

needs.

2. Measures have to be taken to prevent . . .

(a) . . . unfair advantages for particular students.

(b) . . . cheating during the EA.

(d) . . . data loss during the exam, e.g. due to a sys-

tem crash.

3. The whole e-Assessment process has to be trans-

parent to the examiners to clear up doubts.

6 SUMMARY AND OUTLOOK

This paper describes a survey on e-Assessment that

was carried out amongst teachers at RWTH Aachen

University and FH Aachen University of Applied Sci-

ences. The analysis of the results revealed that a

teacher’s view on e-Assessment is inﬂuenced by per-

sonal attributes such as age or teaching experience.

Generally, it seems that teachers are cautious regard-

ing e-Assessment, because they see a high risk of

cheating, especially if students are allowed to use

their own devices. Nevertheless, the teachers also rec-

ognize advantage of e-Assessment, for example the

possibility of a faster correction. From our point of

view, the most important result of the analysis is the

need for transparency when utilizing e-Assessment.

Only by implementing a transparent process for e-

Assessment, it will be possible to convince the teach-

ers to accept e-Assessment as an integral part of the

examination landscape in higher education.

REFERENCES

Baumeister, R. F., Bratslavsky, E., Finkenauer, C., and

Vohs, K. D. (2001). Bad is stronger than good. Review

of general psychology, 5(4):323–370.

Teacher’s Perspective on e-Assessment: A Case Study from Germany

507

Freiwald, T., Salimi, M., Khaljani, E., and Harendza, S.

(2014). Pattern recognition as a concept for multiple-

choice questions in a national licensing exam. BMC

Medical Education, 14(1).

Imtiaz, M. A. and Maarop, N. (2014). Feasibility Study Of

Lecturers’ Acceptance Of E-Assessment. In Proceed-

ings of the 25th Australasian Conference on Informa-

tion Systems.

Jamil, M., Tariq, R. H., and Shami, P. A. (2012).

COMPUTER-BASED VS PAPER-BASED EXAM-

INATIONS: PERCEPTIONS OF UNIVERSITY

TEACHERS. Turkish Online Journal of Educational

Technology-TOJET, 11(4):371–381.

Jin, X. and Han, J. (2011). K-medoids clustering. In

Encyclopedia of Machine Learning, pages 564–565.

Springer US.

Karrer, K., Glaser, C., Clemens, C., and Bruder, C.

(2009). Technikafﬁnit

at erfassen – der Fragebo-

gen TA-EG. In Lichtenstein, A., St

oßel, C., and

Clemens, C., editors, Der Mensch im Mittelpunkt

technischer Systeme. 8. Berliner Werkstatt Mensch-

Maschine-Systeme, volume 22 of ZMMS Spektrum,

pages 196–201, D

usseldorf, Germany: VDI Gerlag

GmbH”.

Kaufmann, L. (1987). Clustering by means of medoids. In

Proc. Statistical Data Analysis Based on the L1 Norm

Conference, Neuchatel, 1987, pages 405–416.

Kuikka, M., Kitola, M., and Laakso, M.-J. (2014). Chal-

lenges when introducing electronic exam. Research

in Learning Technology, 22.

uppers, B. and Schroeder, U. (2019). Students’ percep-

tions of e-assessment. In Passey, D., Bottino, R.,

Lewin, C., and Sanchez, E., editors, IFIP Advances

in Information and Communication Technology, vol-

ume 524 of IFIP Advances in Information and Com-

munication Technology, pages 275–284, Cham. IFIP

TC 3 Open Conference on Computers in Education

(OCCE) 2018, Linz (Austria), 24. Jun 2018 - 28. Jun

2018, Springer International Publishing.

Mellar, H., Peytcheva-Forsyth, R., Kocdar, S., Karadeniz,

A., and Yovkova, B. (2018). Addressing cheating in

e-assessment using student authentication and author-

ship checking systems: teachers’ perspectives. Inter-

national Journal for Educational Integrity, 14(1):2.

Mittal, V., Ross, W. T., and Baldasare, P. M. (1998). The

asymmetric impact of negative and positive attribute-

level performance on overall satisfaction and repur-

chase intentions. Journal of Marketing, 62(1):33.

Peytcheva-Forsyth, R., Aleksieva, L., and Yovkova, B.

(2018). The impact of technology on cheating and

plagiarism in the assessment – the teachers’ and stu-

dents’ perspectives. volume 2048, pages 020037:1–

020037:10.

Pietri, E. S., Fazio, R. H., and Shook, N. J. (2013). Weight-

ing Positive Versus Negative: The Fundamental Na-

ture of Valence Asymmetry. Journal of Personality,

81(2):196–208.

Rolim, C. and Isaias, P. (2018). Examining the use of

e-assessment in higher education: teachers and stu-

dents’ viewpoints. British Journal of Educational

Technology, (4):1785–1800.

Sheskin, D. J. (2003). Handbook of Parametric and

Nonparametric Statistical Procedures: Third Edition.

CRC Press.

Thoring, A., Rudolph, D., and Vogl, R. (2018). The dig-

ital transformation of teaching in higher education

from an academic’s point of view: An explorative

study. In Learning and Collaboration Technologies.

Design, Development and Technological Innovation,

pages 294–309. Springer International Publishing.

APPENDIX

The survey is shown in Figure 8. Particular options

for the tagged items are:

1. < 30 Years; 30–50 Years; > 50 Years

2. Humanities

• Archaeology, Ethics, History, Cultural Studies, Lit-

erature Studies, Philosophy, Theology, Linguistics,

Other Humanities

Engineering

• Civil Engineering, Biotechnology, Electrical Engi-

neering, Information Technology, Mechanical Engi-

neering, Medical Engineering, Environmental Engi-

neering, Process Engineering, Materials Engineer-

ing, Other Engineering Science

Medical Sciences

• Health Sciences, Human Medicine, Pharmaceutics,

Veterinary Medicine, Dental Medicine, Other Medi-

cal Science

Natural Sciences

• Biology, Chemistry, Geoscience, Computer Sci-

ences, Mathematics, Physics, Other Natural Science

Social Sciences

• Comparative Education, Human Geography, Com-

munication Studies, Media Studies, Political Sci-

ences, Psychology, Laws, Sociology, Economics,

Other Social Science

3. Diverse, Female, Male

4. 1 − 5 Semesters; 6 − 10 Semesters; 10 − 20

Semesters; > 20 Semesters;

5. Faster Correction, More Realistic Examinations,

More Diverse Examination Tasks, Other (free

text)

6. Security, Usability, Fairness, Uncertain Legal Sit-

uation, Other (free text)

7. Familiar Device, Location-independent Examina-

tions, Cost Reduction for the IHE, Other (free

text)

8. Security, Differences Between Devices, Other

(free text)

CSEDU 2020 - 12th International Conference on Computer Supported Education

508

Figure 8: Survey (translated to English).

Teacher’s Perspective on e-Assessment: A Case Study from Germany

509