e-Assessment Behind the Scenes

Common Perception of e-Assessment and How We See It Nowadays

Bastian K

uppers

1,2

, Thomas Eifert

, Marius Politze

and Ulrik Schroeder

IT Center, RWTH Aachen University, Germany

Learning Technologies Research Group, RWTH Aachen University, Germany

Keywords:

Electronic Examinations, Electronic Assessment, Computer Aided Assessment, Computer based Assessment.

Abstract:

There exist common preconceptions about e-Assessment that, in the view of many, prohibit e-Assessment to

come into operation. In this paper we examine the most commonly found preconceptions and derive how e-

Assessment should be implemented from our point of view to overcome these obstacles, considering program-

ming assessment as an example. While deriving an e-Assessment scenario, we concentrate on the integration

of e-Assessment into the process workﬂow of institute of higher education, software for e-Assessment and

suitable operational scenarios for e-Assessment.

1 INTRODUCTION

When talking to colleagues who do not deal with e-

Assessment in their daily working life, it seems that

some common preconceptions about e-Assessment

exist, especially when it comes to summative as-

sessment. These concern mainly how to imple-

ment e-Assessment and in which situation to use e-

Assessment. Altogether, these preconceptions and

concerns often lead to a very reluctant position with

respect to e-Assessment. The problem with this is,

that even the deciders in institutes of higher edu-

cation sometimes share these positions, which can

potentially hinder the successful dissemination of e-

Assessment. In this paper we will discuss the precon-

ceptions most commonly expressed towards us in the

past and state why we consider these as not justiﬁed

any more (see section 2). Afterwards, we will discuss

what in our point of view are strong and weak points

of e-Assessment (see section 3). From the discussed

points we will derive how e-Assessment should be

used and implemented in our opinion (see section 4),

using programming assessment as a use case (see sec-

tion 5). The paper closes with a summary and an out-

look (see section 6).

2 e-ASSESSMENT IS...

Either by colleagues from our own university, which

is a public university in Germany, or by other re-

searchers met at conferences and similar occasions,

some beliefs about e-Assessment have been expressed

towards us quite frequently - and apparently we are

not the only ones experiencing this (Jeremias et al.,

2015). In this section we examine the most fre-

quently expressed preconceptions and state why we

think these are not true nowadays.

2.1 ... Just About Multiple Choice

Admittedly, it is true that multiple choice questions

(MCQ) are used intensively in e-Assessment nowa-

days (Oldﬁeld et al., 2012) and if done right, there

is nothing wrong with MCQ in particular (Nicol,

2007). One reasons for the prevalence of MCQ in e-

Assessment is certainly the ease of correcting these

type of questions, as it can be done automatically.

This spares the examiner of a manual correction and

furthermore provides the students with their result im-

mediately. MCQ are, however, by far not the end of

the road for e-Assessment (Crisp, 2009). From our

point of view, e-Assessment can only unfold its full

potential, if for a particular lecture a ﬁtting type of

assignment is developed, if applicable. Otherwise, it

could be better to relinquish e-Assessment in favour

of another type of assessment (see also 2.2).

Küppers, B., Eifert, T., Politze, M. and Schroeder, U.

e-Assessment Behind the Scenes.

DOI: 10.5220/0006788402850291

In Proceedings of the 10th International Conference on Computer Supported Education (CSEDU 2018), pages 285-291

ISBN: 978-989-758-291-2

285

2.2 ... a Replacement for Traditional

Exams

Some people tend to lose interest in e-Assessment, if

they think that e-Assessment is only about MCQ, as

discussed in the previous section. In contrast to that,

other people are over-enthusiastic when it comes to

e-Assessment and want to replace every Traditional

Examination with e-Assessment (Curtis, 2009). From

our point of view, that is also not the right way, since

there are clearly cases where e-Assessment is not the

right tool (K

uppers and Schroeder, 2017). Consider

for example examinations in a maths lecture, where

students have to draw a function graph. This works

a lot better in a paper-based examination than in e-

Assessment, even if it could be made possible in e-

Assessment if a graphics tablet is provided. There-

fore, we think that e-Assessment is a valuable tool,

which should be used complementary to already es-

tablished types of examinations.

2.3 ... Less Secure than Traditional

Exams

This is a point, which is regularly brought up by ob-

jectors of e-Assessment. In fact, it is true that making

the scenario of the examination more complex also in-

troduces more potential security threats and surely we

can not postulate absolute security for e-Assessment,

since this cannot be postulated for any complex sys-

tem (Cook, 1998). However, that does not necessarily

mean that e-Assessment is more vulnerable to cheat-

ing than paper-based examinations. First of all, it has

to be distinguished between on-campus examinations

and off-campus examinations. For the on-campus ex-

aminations is has to be further distinguished between

e-Assessment using devices provided by the institute

of higher education, e.g. in a lab, and e-Assessment

using the students’ own devices (BYOD). Even for

off-campus examinations, solutions to retain integrity

of the examination exist (Frank, 2010), however, in

this paper we will concentrate on on-campus exam-

inations. Hence, e-Assessment does not mean that

the general setting of an examination has to change.

Therefore, there are still invigilators in the examina-

tion room, who keep on eye on the students. Thus,

uninterrupted cheating during the examination is still

not possible, even if the e-Assessment is conducted

on the students’ devices and not on devices provided

by the institute of higher education. Additionally, e-

Assessment does not only introduce new ways of pos-

sible cheats, but also introduces new ways to prevent

those (K

uppers et al., 2017a). We think that, if im-

plemented correctly, e-Assessment is even less likely

to be cheated in than paper-based examinations. The

reason for our opinion is simply, that it is a lot eas-

ier to cheat in a traditional way, than to ﬁnd and ex-

ploit security ﬂaws in an e-Assessment system while

being watched by invigilators. Therefore, even for e-

Assessment we believe cribs and smartphones are the

cheating tools that will be used rather than security

ﬂaws in the the e-Assessment system, which require

a certain effort to be exploited.

2.4 ... a Simple Add-On to Assessment

Even if people are convinced that e-Assessment is

worth implementing, they seem to underestimate the

necessary effort to do so successfully. If it shall be

done correctly, e-Assessment does not work as an

Add-On. That means, that necessary processes need

to be developed and integrated into the process land-

scape of an institute of higher education. At the least,

students have to be identiﬁable in a digital workﬂow

(Apampa et al., 2009) and results have to be stored

reliably (K

uppers et al., 2017b). This does not work,

however, if these requirements are not met by the ba-

sic IT infrastructure of an institute of higher educa-

tion. Therefore, it may be necessary to redesign ex-

isting processes and digital workﬂows in order to pro-

vide a valid basis for e-Assessment.

3 STRONG AND WEAK POINTS

OF e-ASSESSMENT

In the last section, we discussed preconceptions about

e-Assessment and why we think, these are not true

nowadays. In this section we will discuss the strong

and weak points of e-Assessment from our point of

view to further illustrate the .

3.1 Strong Points

3.1.1 Life Cycle

During the life cycle of an assessment, which starts

with creating the assignments and ends with archiv-

ing the sheets of paper that the students handed in,

for a paper-based examination, several switches be-

tween analogue and digital occur: The assignments

are created on a computer (digital), then are printed to

sheets of paper and the students write their answers to

that paper (analogue). Afterwards, the sheets of pa-

per are corrected by pen (analogue) and the resulting

marks are stored on a computer (digital), for example

in an excel sheet. In the end the sheets of paper have

be stored somewhere (analogue). For e-Assessment,

CSEDU 2018 - 10th International Conference on Computer Supported Education

286

these switches can be removed from the life cycle of

the assessment, since every step can be accomplished

digitally.

3.1.2 Domain-speciﬁc Tools

In most lectures, accompanying practicals and tuto-

rials introduce domain-speciﬁc tools. For example,

programming courses introduce integrated develop-

ment environments, like Eclipse

or NetBeans

. This

way, students get already used to important tools dur-

ing their studies, which may be important for later

working life. In a paper-based assessment, these tools

are obviously not available, which leads to a media

disruption between the practised scenario and the as-

sessment. This media disruption can be removed by

switching to e-Assessment. Additionally, the students

then have to show competency in using important

domain-speciﬁc tools.

In addition, the niveau of the assessment can

be raised. Considering Krathwohl’s revised ver-

sion of Bloom’s Taxonomy of Educational Objectives

(Krathwohl, 2002) (see Fig. 1), assessing the higher

levels of the taxonomy, like Evaluate and Create, can

be achieved in a more realistic fashion (Baumann

et al., 2009). The tools available to the students can

take care of the lower levels of the taxonomy, allow-

ing the students to focus on the higher levels. Again,

in programming courses, the integrated development

environment for example provides auto completion,

therefore the students do not need to remember every

keyword of a programming language.

Figure 1: Revised Taxonomy of Educational Objectives by

Krathwohl.

3.1.3 Simpliﬁed Correction

The same domain-speciﬁc tools that can be used by

students during the assessment, can also be used by

http://www.eclipse.org

http://netbeans.org

the correctors. Sticking with the example of a pro-

gramming course, this means the correctors can easily

use a debugger to comprehend the internals of a piece

of source code, instead of having to tediously work

through handwritten code, which may in the worst

case be crabbed.

3.1.4 Innovative Assessments

In a paper-based examination, every assignment has

to be answerable only having pen and paper at hand.

Additionally, the assignment can not include me-

dia, like audio ﬁles or video clips. Having digital

tools available during the assessment, this is possi-

ble, therefore innovative methods of assessment are

within reach (Baumann et al., 2009).

3.1.5 Simpliﬁed Archiving

In case of paper-based assessment, the archiving de-

mands a lot of effort regarding personal and storage

resources. Enough space has to be available where

the paper sheets of the assessment can be kept. Ad-

ditionally, staff members have to set up an archiving

system which allows to retrieve a set of paper sheets

later on. Last, but not least, the sheets of paper have to

brought to the storage location and possibly have to be

retrieved somewhen. In case of e-Assessment, archiv-

ing can be made a lot easier. Since the students results

are already digitally available, archiving can even be

automated. Additionally, a lot less space needs to be

available and retrieving an archived exam can be done

via a computer program.

3.2 Weak Points

3.2.1 Dependence on Infrastructure

To successfully conduct e-Assessment, a functioning

infrastructure has to be available. Especially power

supply and most often also network access have to be

provided. In case mobile devices are used, a loss of

power supply may be bridgeable for a short period of

time due to the batteries in the devices. This is, how-

ever, not a reliable backup strategy. However, in a

modern university almost everything relies on work-

ing IT-Infrastructure, from administrative tasks across

all ﬁelds of research and education. From this per-

spective, an e-Assessment is ”just another” aspect that

relies on a 100% reliability, besides all other essential

processes that keep a university running.

e-Assessment Behind the Scenes

287

3.2.2 Effort

Depending on the particular set up of the assess-

ment, the switch from paper-based assessment to e-

Assessment can demand a lot of effort. If, for exam-

ple, individualized MCQ shall be utilized, a question

pool has to be built up ﬁrst. This question pool has to

be peer reviewed to ensure that subsets of questions

are of the same difﬁculty. This is important to be treat

all students equally (see 4.1) and therefore to be able

to give meaningful marks.

4 HOW TO DO e-ASSESSMENT

Based upon the previously discussed preconceptions,

in this section we derive how e-Assessment should

be implemented from our point of view in order to

overcome the widely seen obstacles.

4.1 Basic Requirements to e-Assessment

There are some basic requirements to summative as-

sessment, which do not change for summative e-

Assessment. These requirements can be directly de-

duced from the purpose of summative assessment:

Check whether a particular student reached the learn-

ing goals of a course. Therefore, the assessment has to

be reliable in multiple manners, which has some im-

plications. First, the writings produced by every stu-

dent during the assessment have to be stored in a reli-

able way. That means especially, that the writings of

the student cannot be altered once they were handed

in, neither by the student nor by the examiner. Also,

the method for storing the writings allows for correc-

tion and a later review of the correction. Second, the

completion of the examination has to be reliable in the

sense that it has to be ensured that the handed-in writ-

ings are indeed the work the particular student created

by himself during the examination. This means in par-

ticular that cheating has to be prevented. The exact

circumstances may differ, depending on the speciﬁc

mode of the assessment, which may differ in several

points, e.g. the location (on-campus or off-campus)

or the allowed aids (closed book or open book).

Additionally, it is important that all students are

treated equally during the exam. This is ethically im-

portant and, of course, required by law. In Germany,

for example, Article 3 of the Basic Law of the Federal

Republic of Germany demands an equality of treat-

ment for all people (Bundestag, 2014). Therefore, it

has to be ensured that all students have an equally dif-

ﬁcult exam, though it has not to be exactly the same

(Forg

o et al., 2016). Additionally, also the circum-

stances for all students have to be similar enough to

not handicap a particular student, since it is not pos-

sible to provide all students with the exact same cir-

cumstances. For e-Assessment, this applies also to the

computing power of the used computer. Therefore, a

student must not be handicapped by limited comput-

ing power of the used device.

4.2 Integration of e-Assessment Into

Existing Processes

As previously discussed, authorship and integrity are

important features of e-Assessment. In general, cer-

tiﬁcates are the current state-of-the art approach to en-

sure authorship and integrity of digital data by using

these certiﬁcates in a digital signature (Kaur and Kaur,

2012). The problem that everyone is in principle able

to generate a certiﬁcate - equivalent to a signature

one can not compare with a known-good sample - is

overcome by widely trusted Certiﬁcation Authorities

(CAs). For e-Assessment within an institute of higher

education, there is no global communication, which

could potentially include a wide variety of people and

hence more than one CA, but only communication

for processes within the institute of higher education

itself. Therefore, exactly one CA has to be trusted,

which ensures integrity for all certiﬁcates used in this

setting. If such a CA is available, for example the

DFN-PKI (DFG, 2016) for RWTH Aachen Univer-

sity, it should be used. If such a CA is not available,

however, the institute of higher education can act as a

CA itself. That would imply, of course, that the stu-

dents have to trust the institute of higher education,

but this seems to be the case since students already

conﬁde their personal data to the educating institutes

and also trust them to handle examinations results and

the like in an appropriate manner.

In order to use certiﬁcates for e-Assessment in an

institute of higher education, it is crucial that not only

valid certiﬁcates are used, but these also have to be

unambiguously relatable to a person. Therefore, in-

formation about the issued certiﬁcates has to be stored

in the Identity Management (IdM) of the institute of

higher education (Eifert, Th. and Bunsen, G., 2013),

for example the corresponding public key. Only if

this link between an identity and a certiﬁcate is es-

tablished, authorship and integrity of the results of an

e-Assessment can be ensured. This would be an im-

provement over the current situation in paper-based

examinations. Nowadays the identiﬁcation is nor-

mally done by looking at the students’ identity cards

and letting them sign an attendance list as well as the

examination itself sometimes. The student card at this

CSEDU 2018 - 10th International Conference on Computer Supported Education

288

point veriﬁes the identity of the student, because it

contains a picture of the particular student as well as

the matriculation number, which links to an identity

in the IdM. However, the signature on the examina-

tion may ensure authorship of the students’ answers

in an examination, but does not ensure integrity of the

documents, as described before. For example it is not

uncommon that additional sheets of paper get lost.

There is, however, a crucial point that has to be

tackled: If the previously described certiﬁcates are

only used for e-Assessment, this may tempt students

to give their private key to other students, for exam-

ple for letting someone else take the examination who

can do it better than oneself. Therefore, the certiﬁ-

cate which is linked to the student has to be of high

value for the student by being used not only for e-

Assessment, but for all or many processes of a stu-

dent life cycle in order to prevent students from shar-

ing their certiﬁcates. For example, viewing grades,

enrolling for examinations and other high valued ad-

ministrative processes can be secured on the basis of

the students’ certiﬁcates. Therefore, everyone pos-

sessing the certiﬁcate could carry out these processes

in place of the certiﬁcate’s true owner. Additionally,

securing digital processes by the usage of certiﬁcates

could improve the privacy of the communication be-

tween a student and the institute of higher education a

lot in difference to unsigned and unencrypted emails.

4.3 (Security) Software for

e-Assessment

When deciding about the software that is used for e-

Assessment, it is important to take into account the

hardware setup ﬁrst. If devices can be provided by

the institute of higher education, e.g. in a lab scenario,

the use of so-called lockdown programs, for example

the Safe Exam Browser developed at ETH Z

urich as

open source project

, is from our point of view the

best way to go. Since these devices can be precon-

ﬁgured and the students have only access to these de-

vices during the examinations without administrative

privileges, lockdown programs can be considered as

being secure for this scenario. However, not all insti-

tutes of higher education can afford a centrally man-

aged IT-infrastructure that provides enough capacity

to carry out e-Assessment properly, since such an in-

frastructure is costly in terms of building up and main-

taining it (Biella et al., 2009)(B

ucking, 2010). This

issue can be resolved by letting the students use their

own devices, since most of them already posses de-

vices that are suitable for e-Assessment (Dahlstrom

https://safeexambrowser.org/

et al., 2015)(Poll, 2015)(Willige, 2016). Addition-

ally, suitable rooms are usually available, since these

would also be needed for regular examinations. If

a BYOD setting is targeted, a lockdown program is

not the best way to go, since there are doubts about

the security of this approach on untrusted platforms

(Søgaard, 2016) and thus about the applicability of

these programs in a BYOD setting. Therefore, we

introduced an alternative approach (K

uppers et al.,

2017a), which employs monitoring and logging of the

students’ actions during the examination, instead of

locking the operating system. For this approach, only

the e-Assessment software itself has to be considered

instead of the whole operating system. Therefore, this

approach can be easily implemented platform inde-

pendent, which is important in a BYOD setting. Ad-

ditionally, this reduces the complexity of the security

task, since only one software, which is well known,

has to be taken into security considerations instead of

a whole operating system. This results is a consid-

erably sparer threat model, which is less likely to be

error-prone and therefore better to handle.

If and which security measures are in fact nec-

essary depends on how the assignments of the e-

Assessment are designed. We discuss further details

about suitable operational scenarios of e-Assessment

in the next section.

4.4 Operational Scenarios for

e-Assessment

As already stated, we do not think that e-Assessment

is a suitable replacement for every paper-based exam-

ination. We rather think that there are paper-based

examinations for which the assessment can be drasti-

cally enhanced when being switched to e-Assessment.

From our point of view, these are the examinations

where e-Assessment can help to conduct the exami-

nation in a more realistic fashion. Basically speak-

ing, these are the assessments where computers are

nowadays part of the workﬂows related to the compe-

tences that shall be assessed. For example, program-

ming assessment is obviously such a case, but there

are also other examples like CAD design for archi-

tecture and mechanical engineering. How exactly the

e-Assessment has to be set up in order to be consid-

ered properly set up, depends from our point of view

heavily on the particular use case. In section 5 we

discuss programming assessment as a representative

case study on how to set up e-Assessment.

e-Assessment Behind the Scenes

289

5 CASE STUDY: PROGRAMMING

ASSESSMENT

Programming is a topic that is obviously suitable for

e-Assessment, since the workﬂow of programming it-

self utilizes a computer. Therefore, e-Assessment can

be used to design an assessment that is more realis-

tic and thus better suited to assess important com-

petences. For programming, it is not only impor-

tant to know all the keywords and constructs of a

programming language, but a successful programmer

also needs to know how to combine those keywords

and constructs into an algorithm that solves a partic-

ular problem. But that is still not sufﬁcient, because

on the way from a problem to a programmed solution,

several tools are used, for example a debugger. Gen-

erally speaking, a successful programmer not only has

to be ability to write working program code, but also

needs to master the usage of all the tools that are nec-

essarily part of a software development workﬂow. Es-

pecially this latter part of the software development

process can not easily be assessed in a traditional ex-

amination, i.e. an examination on paper. When do-

ing e-Assessment, however, it is very easy to assess

these competences since the programmer’s workﬂow

and the e-Assessment-environment can be integrated

with each other so that the complete workﬂow can

get subject of the assessment. That means, that e-

Assessment is not about using the computer for the

same things that were carried out on paper before, but

it is about providing a different workﬂow, which in-

cludes all necessary tools. For programming assess-

ment, this means that an editor with syntax highlight-

ing and probably auto-completion, a debugger and

potentially a version control system not only have to

be available but have to be part of the examination.

This ensures that the outcome of the assessment tells

something about a students’ ability to write working

software, and not only about her comprehension of a

programming language.

Which editor is available and how this editor is in-

tegrated into a security concept does not matter that

much, as long as the whole concept is consistent and

fulﬁls all requirements regarding security and relia-

bility. We think, however, that it is important that the

setup in the examination does not differ from a setup

that was used during practicals and tutorials, which

potentially are held alongside the lecture. Otherwise,

we observe that the students undergo separate train-

ings, for the exam and for the job they are trained for.

Additionally, tackling assessment like described

above leads to a lot more meaningful marks. Consid-

ering a situation where an employer is looking for em-

ployees, a mark in a programming course that was ob-

tained with the presented approach to e-Assessment

tells a lot more about the skills of a potential employee

and thus is way better suited to select potential appro-

priate applicants.

6 SUMMARY AND OUTLOOK

In this paper we discussed preconceptions regarding

e-Assessment and why we think that these are nowa-

days not justiﬁed any more. From the discussion of

the preconceptions, we derived how we think that e-

Assessment should be applied to institutes of higher

education, focusing on integration, security and suit-

able operational scenarios of e-Assessment. Based on

the presented ideas, we discussed programming as-

sessment as a case study.

Looking at the directions that e-Assessment al-

ready has taken in the last years, we are convinced

that it will ﬁnd its place as an important part of the

examination system at institutes of higher education

in the future. We do not know yet how e-Assessment

will arrive there and what will probably change until

then, but projecting some ten years into the future, we

cannot imagine how e-Assessment could not be a part

of the future of assessment - especially considering all

the advantages that it offers.

REFERENCES

Apampa, K. M., Wills, G., and Argles, D. (2009). Towards

security goals in summative e-assessment security. In

2009 International Conference for Internet Technol-

ogy and Secured Transactions, (ICITST), pages 1–5.

Baumann, M., Steinmetzer, J., Karami, M., and Sch

afer,

G. (2009). Innovative electronic exams with voice in-

and output questions in medical terminology on a high

taxonomic level. Medical Teacher, 31(10):e460–e463.

Biella, D., Engert, S., and Huth, D. (2009). Design and De-

livery of an E-assessment Solution at the University of

Duisburg-Essen. In Proceedings EUNIS 2009, EUNIS

Proceedings.

ucking, J. (2010). eKlausuren im Testcen-

ter der Universit

at Bremen: Ein Praxis-

bericht. https://www.campussource.de/events/

e1010tudortmund/docs/Buecking.pdf. Accessed

2017-12-06.

Bundestag, G. (2014). Basic law for the fed-

eral republic of germany. https://www.btg-

bestellservice.de/pdf/80201000.pdf.

Cook, R. I. (1998). How complex systems fail. Cogni-

tive Technologies Laboratory, University of Chicago.

Chicago IL.

Crisp, G. (2009). Interactive e-assessment: mov-

ing beyond multiple- choice questions.

CSEDU 2018 - 10th International Conference on Computer Supported Education

290

https://www.researchgate.net/proﬁle/Geoffrey Crisp/

publication/228460929 Interactive e-

Assessment moving beyond multiple-choice

questions/links/004635249f36897ffc000000.pdf.

Accessed 2017-11-21.

Curtis, P. (2009). Computerised testing likely to re-

place traditional exams, says head of board.

https://www.theguardian.com/education/2009/jul/12/

written-exams-computerised-testing. Accessed

2017-11-21.

Dahlstrom, E., Brooks, C., Grajek, S., and Reeves, J.

(2015). Ecar study of students and information

technology, 2015. https://library.educause.edu/ /me-

dia/ﬁles/library/2015/8/ers1510ss.pdf?la=en. Ac-

cessed 2017-12-06.

DFG (2016). Certiﬁcation policy of the DFN-

PKI. https://www.pki.dfn.de/ﬁleadmin/PKI/Policy-

Archiv/DFN-PKI CP-EN.pdf. Accessed 2017-11-22.

Eifert, Th. and Bunsen, G. (2013). Grundlagen und En-

twicklung von Identity Management an der RWTH

Aachen. PIK - Praxis der Informationsverarbeitung

und Kommunikation.

Forg

o, N., Graupe, S., and Pfeiffenbring, J. (2016).

Rechtliche Aspekte von E-Assessments an

Hochschulen.

Frank, A. J. (2010). Dependable distributed testing: Can the

online proctor be reliably computerized? In 2010 In-

ternational Conference on e-Business (ICE-B), pages

1–10.

Jeremias, X. V., Sellmer, B., and Tippe, U. (2015). E-

assessments in der hochschullehre: Chancen und

herausforderungen. https://opus4.kobv.de/opus4-

th-wildau/ﬁles/361/11

E-Assessments-in-der-

Hochschullehre.pdf. Accessed 2017-12-06.

Kaur, R. and Kaur, A. (2012). Digital signature. In 2012 In-

ternational Conference on Computing Sciences, pages

295–301.

Krathwohl, D. R. (2002). A Revision of Bloom’s Taxon-

omy: An Overview. Theory Into Practice, 41(4):212–

218.

uppers, B., Kerber, F., Meyer, U., and Schroeder, U.

(2017a). Beyond lockdown: Towards reliable e-

assessment. In GI-Edition - Lecture Notes in Infor-

matics, volume 273, pages 191–196.

uppers, B., Politze, M., and Schroeder, U. (2017b). Reli-

able e-assessment with git - practical considerations

and implementation. In EUNIS 2017 Shaping the

Digital Future of Universities, pages 253–262.

uppers, B. and Schroeder, U. (2017). Vergleich von pa-

pierklausuren und elektronischen pr

ufungen. In Eibl,

M. and Gaedke, M., editors, INFORMATIK 2017,

pages 307–318. Gesellschaft f

ur Informatik, Bonn.

Nicol, D. (2007). E-assessment by design: using multiple-

choice tests to good effect. Journal of Further and

Higher Education, 31(1):53–64.

Oldﬁeld, A., Broadfoot, P., Sutherland, R., and Tim-

mis, S. (2012). Assessment in a digital age:

A research review. http://www.bristol.ac.uk/media-

library/sites/education/documents/researchreview.pdf.

Accessed 2017-11-21.

Poll, H. (2015). Student Mobile Device Sur-

vey 2015: National Report: College Stu-

dents. https://www.pearsoned.com/wp-

content/uploads/2015-Pearson-Student-Mobile-

Device-Survey-College.pdf. Accessed 2017-12-06.

Søgaard, T. M. (2016). Mitigation of Cheating Threats in

Digital BYOD exams. Master’s thesis, Norwegian

University of Science and Technology, Trondheim,

Norway.

Willige, J. (2016). Auslandsmobilit

at und

digitale Medien: Arbeitspapier Nr. 23.

https://hochschulforumdigitalisierung.de/sites/

default/ﬁles/dateien/HFD AP Nr23 Digitale

Medien und Mobilitaet.pdf. Accessed 2017-12-

06.

e-Assessment Behind the Scenes

291