ADAPTIVE E-ASSESSMENT APPLICATION SCENARIO

A Framework and Concept for the Integration of askme! in Higher Education

Andr

e Schulz

1,2

and Christian Saul

Erfurt University of Applied Sciences, Department of Applied Computer Science, Altonaer Str. 25, 99085 Erfurt, Germany

Fraunhofer IDMT, Children’s Media Department, KinderMedienZentrum, Erich-K

astner-Str. 1a, 99094 Erfurt, Germany

Fraunhofer IDMT, Business Area Data Representation and Interfaces, Ehrenbergstr. 31, 98693 Ilmenau, Germany

Keywords:

Adaptivity, E-Learning, Higher Education, Assessment, Theoretical Computer Science, Moodle.

Abstract:

To provide the expertise to apply learned knowledge in practice is one main challenge of higher education.

Especially, the engineering graduates need to solve complex problems in their profession and apply speciﬁc

expertise in various contexts. These demands encompass less recalling of factual knowledge, but more com-

prehensive key competencies like problem-solving expertise. In this paper, the Adaptive Assessment System

askme! is used to address this challenge. The authors discuss the general conditions of the education of a

course in theoretical computer science offered for students of a Bachelor degree program. Furthermore, this

paper provides an application scenario in which the applied learning platform Moodle will be extended by

adaptive methods of assessment.

1 INTRODUCTION

For educational processes it is crucial to obtain evi-

dence about knowledge and skills that were learned.

Hence, the measurement of learning outcomes is an

integral part of developing successful learning mate-

rials and a critical catalyst for student learning. This

challenge is addressed through assessment.

The assessment in Higher Education (HE) has tra-

ditionally been focused on ”[...] retention and appli-

cation of knowledge in limited contexts as measured

by paper and pencil tests and academic assignments

[...]” (Reeves, 2000). But, as ICT-based applications

become more prevalent in HE, assessment is going

to be of increasing importance, because ”[...] assess-

ment drives learning [...]” (Reeves, 2006) and ”[...] if

something is not assessed in HE, then it is not learned

[...]” (Bain, 2004). The majority of assessment strate-

gies used in HE tend to focus on what is easy to mea-

sure rather than what is important. This is one of the

”[...] strongest deﬁcits of both traditional and recent

course design and implementation in HE [...]” (Ship-

man et al., 2003). But, assessment is crucial to help

students to learn and not just to rate and rank their ef-

forts. Entering the world of employment makes new

demands on today’s graduates. These demands en-

compass less recalling of factual knowledge, but more

comprehensive key competencies (Sippel, 2009).

Therefore, a main objective of HE is to empower stu-

dents to discover ﬁndings and to develop knowledge

by themselves – so called problem-solving expertise.

This is reﬂected by a case study, which is developed

in this paper. Speciﬁcally, the case study concerns

the course ”Theoretical Computer Science (THI)” at

Erfurt University of Applied Sciences, which is com-

posed of two sub-modules: THI 1 and THI 2. The lec-

tures and seminars offered pursue the following three

main objectives: (1) provide formal and algorithmic

skills, (2) develope analytical skills, design expertise,

construction expertise and project management skills,

and (3) promote methodological competence, social

competence and self-competence.

In this paper, the Adaptive Assessment System

(AAS) askme!, which is being developed at the Fraun-

hofer Institute for Digital Media Technology (IDMT),

is used to address the challenges currently posed by

THI 1 and 2. E-assessment systems must evolve

to further enter the ﬁeld of HE for engineering sci-

ences. What is needed are personalized assessment

solutions, which are able to cover all levels of think-

ing skills. The AAS askme! addresses this challenge

by taking into account students’ individual context,

prior knowledge and preferences in order to person-

alize the assessment. In order to face the challenge

of didactical interactivity, askme! enables integrating

Interactive Content Objects (ICOs).

132

Schulz A. and Saul C..

ADAPTIVE E-ASSESSMENT APPLICATION SCENARIO - A Framework and Concept for the Integration of askme! in Higher Education.

DOI: 10.5220/0003923601320137

In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 132-137

ISBN: 978-989-8565-07-5

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

2 ASSESSMENT IN MOODLE

Assessment can be deﬁned as follows: A systematic

method comprising the process of identifying, gath-

ering, analyzing and interpreting information about

peoples knowledge, skills, attitudes and other char-

acteristics aiming at drawing inferences about their

achievements and progresses as well as improving

their learning and development performance.

Such methods are exams, tests, quizzes and sur-

veys. Although the terms are often used interchange-

ably, they differ signiﬁcantly in terms of the purpose

of measurement and the scope of content covered.

2.1 Opportunities

Moodle

is an open source web-based and SCORM-

compliant LCMS and LMS that can be used in many

types of environments such as in education, train-

ing and development as well as in business settings.

For creating assessments, Moodle provides several

assessment options like set up a time limit, conﬁg-

ure grading method or control the access. Two main

forms of assessment namely assignments and quizzes

can be used. Assignments allows teachers to collect

work from students, review it and provide feedback

including grades, whereas quizzes allows teachers to

design and build assessments consisting of a set of

questions. Furthermore, Moodle allows creating a

variety of different question types to set up assess-

ments. In general, it can be distinguished between

standard (e.g., multiple choice, true/false, cloze, etc.)

and third-party question types (e.g., drag and drop,

opaque, regular expression short answers, etc.).

For interacting with the questions in the assess-

ment, Moodle supports different question behavior

modes (e.g., deferred feedback, manually graded, im-

mediate feedback, etc.). Feedback can be displayed

at different times during the assessment and can not

only include text, but also images, multimedia ﬁles,

equations, etc. Moodle distinguishes between gen-

eral, overall, speciﬁc and combined feedback.

Questions created in Moodle can be exported ac-

cording to the IMS QTI 2.0 speciﬁcation - in addi-

tion to the export in a Moodle-speciﬁc text or XML

format. For importing questions, Moodle has a va-

riety of ﬁle formats that can be used. This in-

cludes some LCMS-speciﬁc question formats (e.g.,

the Blackboard V6+ format, the Examview format,

the WebCT format, etc.) as well as the Moodle-

speciﬁc text and XML format. At the moment of

writing, importing questions in IMS QTI format is not

possible.

http://www.moodle.org

2.2 Reality

In the everyday practice, it is not surprising that only

a few of these features are used. LCMS such as Moo-

dle are mostly used only to provide educational con-

tent, whereas the assessment possibilities of Moodle

are sparsely used. Even the THI courses have only

used basic possibilities of the assessments so far. Fo-

cussing the lectures, Moodle was only used to pro-

vide additional learning material. Thereby, Moodle

resources like labels, pages or ﬁles were used to pro-

vide information or to download presentation slides.

In addition, Moodle activities like forums were used

to exchange experiences and problems.

For assessment purposes, both of the aforemen-

tioned methods (assignments and quizzes) were used.

In terms of quizzes, tests were provided in addition

to the exercises delivered during the presence semi-

nars. The results of the students were included into

the grading of the course, but they are also offered

for self-studying purposes. But, the range of question

types used were very limited. Mostly multiple-choice,

embedded answers (cloze) and true/false questions

were used to assess the students’ knowledge. Miss-

ing question types were tried to cover by assignments.

However, assignments are associated with an huge ef-

fort for correcting.

The use of e-learning systems in general and e-

assessment in particular points to some deﬁciencies

in practice.

2.3 Deﬁciencies and Problems

Especially theoretical computer science deals with

some very complex issues, which can not be covered

by the standard question types provided by Moodle.

The problem is not asking the questions, but rather

giving the answer and its automatic grading. As an

example, let us consider a task in which a pushdown

automaton is to be developed, which should recognize

a given language L(G) =



i+ j

|i, j > 0



. If we

want to combine different batched results and ﬁnd-

ings to create a complex scenario, the limits of the

standard assessment features in Moodle are already

far exceeded.

The learnability of theoretical computer science is

generally not very easy. This is due to the mappabil-

ity of many serious issues in a practical and complex

relationship. In fact, these are not ideal conditions

for the creation of scenarios in which learners fully

can immerse and to experience it. But the authors of

the present paper are convinced that this state must be

changed.

ADAPTIVEE-ASSESSMENTAPPLICATIONSCENARIO-AFrameworkandConceptfortheIntegrationofaskmein

HigherEducation

133

3 FRAMEWORK

Increasing the effectiveness and efﬁciency of HE can

not be guaranteed solely by the use of e-learning sys-

tems. In the following, several requirements are dis-

cussed, and matched against the askme! functionality.

3.1 Advanced Question Types

Theoretical computer science deals with very com-

plex problems. In some rare cases, they can be cov-

ered by conventional question types. Questions such

as (i) create a context-free grammar that generates

the language L(G) or (ii) create a pushdown automa-

ton that recognizes the language L(G) far exceeds the

core functions of classical LCMS like Moodle.

Facing the challenge, askme! enables to create ad-

vanced question types by integrating ICOs like Web-

bles (Arnold et al., 2012), PATTI (Klein et al., 2010),

RemoteLab (Wuttke and Henke, 2008) or WLABEL

(Loureiro and Depover, 2005). Each of the ICOs

allows setting input parameters, interacting with the

tool, constructing hypotheses based on prior knowl-

edge and their validation by observing the effect of

changes.

3.2 Integrability of Third-party Tools

For some fundamental problems in theoretical com-

puter science, tools have been developed, that offer

very high potential as an illustration and for practical

exercise. However, the offer is limited mostly to sin-

gle special cases. The only option to integrate such

tools into a learning resource is to insert a link to the

external tool. A real integration into individual prac-

tice and teaching offer is desirable, but requires well-

deﬁned interfaces.

And again, askme! faces this challenge by inte-

grating these tools as ICOs. It enables the assess-

ment of the upper levels of Bloom’s cognitive tax-

onomy (HOTS) by encouraging the students to apply

their knowledge in new situations to solve unexpected

problems, by that way creating new knowledge. This

complements the system’s ability to assess LOTS in

a holistic adaptive assessment process addressing all

levels of thinking skills.

3.3 Automation of the Grading

Direct feedback when editing tasks promotes the

recognition of errors. However, this can rarely be im-

plemented in a regular curriculum. The large number

of students and the limited staff capacities play a con-

siderable role in this case. A key objective is to give

each student the opportunity to review the own knowl-

edge and quickly comprehend errors in the context of

problem solving. E-Learning systems do offer the po-

tential to guarantee this, but for complex exercises,

they quickly reach their limits.

askme! faces this challenge by taking sophisti-

cated feedback techniques and methods into account,

which results in providing feedback that is appropri-

ate for the students’ context, knowledge level, in-

dividual characteristics and preferences (Saul et al.,

2010). Feedback can not only consists of textual in-

formation, but also graphics, audio and video. In ad-

dition, feedback is classiﬁed according to the levels of

veriﬁcation and elaboration incorporated. This results

in ﬁve feedback categorizations, which supports their

use in adaptation rules.

3.4 Practicability of the Authoring

For the acceptance and practical use of software sys-

tems, their practicability is of crucial importance. In

general, the LMS and the e-assessment in particular,

this concerns mainly the creation of content.

askme! faces this challenge by providing ques-

tion and test authoring in an accurate way based on

a well-designed user interface. It requires autho-

rization and provides secure use according to user

rights. Moreover, the sustainability of the questions

created by askme! is guaranteed by the conformance

to the IMS QTI v2.1 speciﬁcation. The integration

of askme! with established LCMS (e.g., EDMedia,

Moodle, ILIAS) is realized through a generic inter-

face (Saul et al., 2011).

3.5 Personalization of the Tests

Although online-assessments were the ﬁrst imple-

mented and are one of the most interactive compo-

nents in LCMS (Brusilovsky and Miller, 1999), there

are some problems to be addressed. This includes

more technical ones such as user identity veriﬁcation

and security issues and more general ones such as per-

sonalization aspects (Vasilyeva et al., 2007). Person-

alization in online-assessments has to consider indi-

vidual and social aspects to avoid treating all students

in the same manner and to prevent the feeling of get-

ting lost in the masses (Geister and Rastetter, 2009).

askme! faces this challenge by selecting questions

dynamically. Based on rules and the last response of

the student, appropriate questions are dynamically se-

lected at runtime. This allows authors of tests express-

ing their didactical philosophy and methods through

the creation of appropriate rules.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

134

4 APPLICATION SCENARIO

Only the practical application shows the meaning of

theoretical concepts. In order to realize this a appli-

cation scenario for the complex course of theoretical

computer science in the Bachelor program of applied

computer science was developed.

4.1 Didactic Concept

The objectives of the course are (i) to provide formal,

algorithmic skills, (ii) to develop analytical skills,

design expertise, construction expertise and project

management skills, and (iii) to promote method-

ological competence, social competence and self-

competence.

The didactic concept of the whole event is based

on the blended learning approach. Besides the ba-

sic content (taught in weekly lectures), accompanying

exercises are offered. In addition to the face-to-face

lectures, also additional digital content and learning

opportunities (e.g., lecture slides, especially the lec-

ture exercises, tests and additional interactive learning

tools) are offered through Moodle.

The course concludes with a complex test where

the content of the two sub-modules is tested. It is cru-

cial that the knowledge imparted is not reproduced,

but also comprehended and applied in this situations.

However, it is important to design the learning pro-

cess for students so that the training will promote the

problem-solving skills optimally.

4.2 Didactic Implementation

An extension of e-Learning opportunities offered by

the adaptive e-Assessment system askme! is intended

to be a ﬁrst step towards the improvement of teach-

ing. In the current Moodle course the core features

of Moodle are used. Moodle resources like labels,

pages or ﬁles and Moodle activities like forums, as-

signments and quizzes are used.

There is no visually apparent difference. The ob-

jective of the present paper is to evaluate askme! in

practice. To ensure this, only the core functionality of

Moodle can be used.

On the one hand, the practicability of the au-

thoring process and the resource management perfor-

mance must be ensured. On the other hand, the com-

parison to e-learning using traditional assessments

must be ensured. One obvious option is to leave the

didactic structure of the previous course. Only the

assessment elements are implemented internally by

askme! and appear as Moodle assessment activities.

4.3 Adaptivity

The user-driven customisation of system-parameters

is understood as adaptability, while an automatic,

system-controlled adaptation to changing conditions

is known as adaptivity (Leutner, 1995; Blank, 1996).

According to the remarks on the basic functional-

ity of adaptive software systems the essentially three

phases of the adaption must be considered in the im-

plementation of askme! (Brusilovsky, 1996). Or, as

Marcus Specht (1998) formulated it: ”To increase the

quality of technology anhanced learning it is impor-

tant to distinguish what should be adapted, to what

features should it be adapted and how should it be

adapted”.

The objectives of adaptation are: (i) avoid treating

all students in the same manner, (ii) Identifying and

rectifying individual shortcomings and deﬁcits, and

(iii) Encouraging students to actively thinking. The

means of adaptation are: (i) Sequencing of Questions

and (ii) Presentation of Feedback. The information

for the adaptation are: (i) Performance of the student,

and (iii) the Student Proﬁle. The process of adaptation

is (i) based on IF <Condition> THEN <Action>

rules and (ii) Forward chaining.

4.4 Technical Implementation

The integration of the AAS askme! with the LCMS

Moodle was done according to the approach pro-

posed in (Saul et al., 2011) by using the OPAQUE

(Open Protocol for Accessing QUestion Engines)

is based on SOAP and allows LCMS to delegate the

presentation of questions, the scoring of responses

and the generation of feedback to a remote question

engine.

In the following, an arbitrary testing procedure is

described (see Figure 1): (1) Question is presented,

(2) Student takes the questions and ﬁnally submits an

answer, (3) askme! evaluates the answers, (4) askme!

examines the condition for the rule, and (5) askme!

updates the student proﬁle with the results of the test.

Student

User Profile

Adaptive Test

(Set of Questions and

associated Rules)

askme!

(Adaptation-Engine)

(2)

(4)

(3)

(5)

(1)

Figure 1: Technical Implementation.

https://github.com/timhunt/moodle-qtype opaque

ADAPTIVEE-ASSESSMENTAPPLICATIONSCENARIO-AFrameworkandConceptfortheIntegrationofaskmein

HigherEducation

135

5 EVALUATION

Besides the practical realisability of a system, it is

critical that the end users perceive the system as a sup-

porting system and accept it. Based on a formative

evaluation experiences for the further development as

well as strengths and weaknesses of askme! will be

determined.

The ﬁrst deployment of askme! should take place

within the framework of the application scenario de-

scribed in Chapter 4. In the following discussion, the

actual experimental conditions are described.

5.1 Objectives

The integration of askme! in a real teaching and learn-

ing scenario is the basis to obtain important experi-

ence of the technical feasibility and practicability of

the system. In further considerations, the acceptance

of teachers and students, the effort for teachers and the

beneﬁts for learners should be considered in detail. In

this context they are regarded as long-term goals for

the following evaluations, but should remain in mind.

The major objective of the evaluation is to esti-

mate the beneﬁts for learning. This essentially relates

to the investigation of the efﬁciency and effectiveness

of learning.

5.2 Deﬁnitions

The efﬁciency is generally deﬁned as the ratio of ef-

fort to do something, and the beneﬁts achieved. Fol-

lowing this understanding, the efﬁciency of learning

describes the relationship between learning time and

learning beneﬁts. According to the authors view an

intensive investigation of the beneﬁts and success of

learning isn’t useful at this initial evaluation. There-

fore, the present evaluation is limited merely to the

investigation of the learning effort in terms of time

needed for it. The learning beneﬁts or learning suc-

cess will in simpliﬁed form equated with the grading.

The effectiveness can generally deﬁned as the

value of the outcome of an action. In the considered

action ”learning”, the grading or scoring can be seen

simply as the value of the result.

5.3 Question, Hypotheses, and Criteria

The main problem of this evaluation can be sum-

marised by the following research question:

Is there an increasing of the efﬁciency and ef-

fectiveness of learning through personalised

E-assessment?

Based on this research question, the following hy-

potheses can be form:

1. The more personalised e-assessments are de-

ployed, the better the grades in the ﬁnal test.

2. The more the assessments are adapted to the indi-

vidual characteristics, the less time is needed for

proper execution.

3. The more personalised e-assessments are used in

the course, the more rapid decreases the frequency

of errors.

Out of the question and the hypotheses three research

criteria were extracted: (i) learning time, (ii) number

of errors and (iii) grades.

5.4 Research Design

The key to successful evaluation with authentic and

useful result is the choice of research methods. Essen-

tially quantitative and qualitative research approaches

can be distinguished (Flick, 2002; Lamnek, 1995).

To evaluate the efﬁciency and effectiveness, quanti-

tative methods are particularly suitable. But to get

an impression of the usability, the practicality, and

the didactic approaches using qualitative methods are

reasonable, too. Therefore, a methodological mix

of qualitative and quantitative research approaches is

chosen.

The population of the study is represented by the

participants and the tutors of the THI 2 course. A to-

tal of about 85 students in the second Semester of the

Bachelor degree program AI participating in the lec-

ture. The period of evaluation is limited to one full

semester. The sample includes all students participat-

ing the course THI 2.

Regarding the large number of different data col-

lection methods (Schnell et al., 2008) the authors

chose especially the log-ﬁle analysis, in addition to

the questionnaire methods - for economic reasons,

and the handling. The non-reactively data collection

by log ﬁles (log ﬁle analysis) is of high practicability.

Moodle provides comprehensive data. Aspects for ex-

ample the frequency of usage or the elapsed time for

processing a task, and their evaluation are logged. In

addition to that, a survey of quantitative data will be

used. Besides general topics e.g., to the usability, spe-

cial issues of askme! are requested.

The purely quantitative questionnaires provide

only little information about which personal reasons

have led to the statements.Qualitative aspects are

requested to complement the quantitative question

items. With the help of free text ﬁelds, the subjects

should specify, inter alia, the reasons for their deci-

sions.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

136

6 CONCLUSIONS

Adaptive e-Assessments are able to support students

to develop methodological competencies. That is one

of the motivating hopes, which underlying the present

paper.

Discussing motivational aspects of higher educa-

tion and emphasizing the requirements of the case

study shows that e-Assessments could be a solution.

Focussing teaching of theoretical computer science

showed the precise requirements could be fulﬁlled by

askme!. The developed application scenario will now

serve as the basis for new insights and cause debate

potential in the ﬁeld of educational research. The au-

thors would like to share this work by the research

community, to take part in this process.

ACKNOWLEDGEMENTS

The authors gratefully acknowledge the cooperation

with a larger number of scientists and engineers who

have contributed to the development.

This work has been supported by the Thuringian

Ministry for Education, Science, and Culture within

the project iCycle under contract PE-004-2-1.

REFERENCES

Arnold, O., Fujima, J., Jantke, K. P., and Tanaka, Y. (2012).

Exploring and understanding the abstract by direct

manipulation of the concrete. In CSEDU 2012 - Pro-

ceedings of the 4rd International Conference on Com-

puter Supported Education, Porto, Portugal. to be

published.

Bain, K. (2004). What the Best College Teachers Do. Har-

vard University Press.

Blank, K. (1996). Benutzermodellierung f

ur adaptive in-

teraktive Systeme: Architektur, Methoden, Werkzeuge

und Anwendungen. PhD thesis, Universit

at Stuttgart,

Stuttgart.

Brusilovsky, P. (1996). Methods and techniques of adap-

tive hypermedia. In User Modeling and User Adapted

Interaction, pages 87–129.

Brusilovsky, P. and Miller, P. (1999). Web-based Testing

for Distance Education. In Proceedings of WebNet,

volume 99, pages 24–30.

Flick, U. (2002). Qualitative Sozialforschung – Eine

Einf

uhrung. Rowohlt, Reinbek.

Geister, S. and Rastetter, D. (2009). Aktueller Stand zum

Thema Online-Tests. In Online-Assessment, pages 3–

16. Springer Berlin Heidelberg.

Klein, F., Kruspe, A., Pﬁtzer, M., Werner, S., and Branden-

burg, K. (2010). Interaktiver audiovisueller Demon-

strator psychoakustischer Ph

anomene. In Proceed-

ings of the Deutsche Jahrestagung f

ur Akustik (DAGA)

2010, pages 183–184.

Lamnek, S. (1995). Qualitative Sozialforschung – Band 2:

Methoden und Techniken, volume 2. Beltz, Weinheim.

Leutner, D. (1995). Adaptivit

at und Adaptierbarkeit mul-

timedialer Lehr- und Informationssysteme. In Issing,

L. J. and Klimsa, P., editors, Information und Lernen

mit Multimedia, pages 139–149. Beltz Psychologie-

Verl.-Union, Weinheim.

Loureiro, M. and Depover, C. (2005). Impact of WLA-

BEL exploitation on electricity learning: analysis of

the students’ conceptual evolution. Interactive Edu-

cational Multimedia, 11:190–203.

Reeves, T. (2000). Alternative Assessment Approaches

for Online Learning Environments in Higher Educa-

tion. Journal of Educational Computing Research,

23(1):101–111.

Reeves, T. (2006). How do you know they are learning?

The importance of alignment in higher education. In-

ternational Journal of Learning Technology, 2(4):294.

Saul, C., D

urrwald, F., and Wuttke, H.-D. (2011). Towards a

Generic Integration of Adaptive Assessment Systems

with Learning Environments. In Proceedings of the

3rd International Conference on Computer Supported

Education (CSEDU 2011), pages 117–122, Noordwi-

jkerhout.

Saul, C., Runardotter, M., and Wuttke, H.-D. (2010). To-

wards Feedback Personalisation in Adaptive Assess-

ment. In Book of Abstracts & Workshop Programme

Sixth EDEN Research Workshop, pages 142–143, Bu-

dapest. European Distance and E-Learning Network.

Schnell, R., Hill, P. B., and Esser, E. (2008). Methoden

der empirischen Sozialforschung. Oldenbourg Wis-

senschaftsverlag, M

unchen, Wien.

Shipman, D., Aloi, S., and Jones, E. (2003). Addressing

Key Challenges of Higher Education Assessment. The

Journal of General Education, 52(4):335–346.

Sippel, S. (2009). Zur Relevanz von Assessment-Feedback

in der Hochschule. ZFHE, 4(1):1–22.

Specht, M. (1998). Adaptive Methoden in computer-

basierten Lehr/Lernsystemen. PhD thesis, University

of Trier, Sankt Augustin.

Vasilyeva, E., Puuronen, S., Pechenizkiy, M., and Rasa-

nen, P. (2007). Feedback Adaptation in web-based

Learning Systems. International Journal of Contin-

uing Engineering Education and Life-Long Learning,

17(4/5):337–357.

Wuttke, H.-D. and Henke, K. (2008). LMS-Coupled Simu-

lations and Assessments in a Digital Systems Course.

In Proceedings of the 10th IEEE International Sympo-

sium on Multimedia, pages 726–731.

ADAPTIVEE-ASSESSMENTAPPLICATIONSCENARIO-AFrameworkandConceptfortheIntegrationofaskmein

HigherEducation

137