Ontology based UX Personalization for Gamiﬁed Education

Zsigmond Imre

Faculty of Mathematics and Computer Science, Babes

- Bolyai University, 1 Mihail Kog

alniceanu,

RO-400084 Cluj - Napoca, Romania

Keywords:

Ontology, Gamiﬁcation, Education, Ontology based Reasoning.

Abstract:

Gamiﬁcation techniques are increasingly used in education, both in private and public sectors. These game

design elements need to be carefully tailored to the students, considering a variety of factors if we want positive

results. The key hindrance is the lack of systematic basic research in mapping out the connections between

the metrics of the student and the game mechanics used. In this paper we present a conceptual framework

for gamiﬁcation implementation improvement. We show how ontologies and ontology-based reasoning can

improve the basic research and the application of gamiﬁcation in education.

1 INTRODUCTION

In the last decade there has been increased research

into the gamiﬁcation of learning. It is an educational

approach to motivate students to learn by using video

game design and game elements in learning environ-

ments (Shatz, 2015). Some of these elements in their

turn come from psychology, entertainment (Skinner,

1935). The goal is to maximize enjoyment and en-

gagement through capturing the interest of learners

and inspiring them to continue learning. It must be

clariﬁed that gamiﬁcation is not “Edutainment” and

maximizing enjoyment should be to the point where

it still aids in learning.

While most gamiﬁcation techniques focus on us-

ing a game mechanic, for example points, for every-

body in the audience, few have tried tailoring the ex-

perience on the individual. The two main hindrances

have been the implied necessary effort and the lack

of useful information on how to do it properly. There

are mentions of tailoring content to the learner’s skill

level, but usually little details are shared or it is a man-

ual process (Kiesler et al., 2011).

To get the same beneﬁts as games the process of

customizing gamiﬁcation elements and study materi-

als needs to be automated. For automation to work

and to be adjustable based on new data gathered from

the ﬁeld, ontologies and ontologies-based reasoning

is a natural choice.

At the current technological state, certain ﬁelds

lend themselves for automated checking and grading

better than others. Automated grading in computer

science is reasonably achievable, it would however be

practically impossible for modern art. We chose com-

puter science because of this technical reason. Check-

ing and grading computer science assignments have

several problems associated with it (Cheang et al.,

2003). The institution needs experienced/trained per-

sonnel to do the job. It is highly labor intensive to

check a classroom full of students’ work, while also

helping them with their individual problems.

The work entails:

• checking if the solution is working as speciﬁed

• checking whether study material speciﬁc techni-

cal details have been met

• checking coding style

• checking for plagiarism

These by themselves can be hard problems to solve. It

is also a problem for educators to properly teach while

accomplishing the above in the short time available.

Automation is a great tool, but it can be used for more

when combined with gamiﬁcation techniques.

The aim is to create a solution that allows for the

easy use of gamiﬁcation, is the most applicable in ed-

ucation and facilitates basic research in the ﬁeld. Per-

sonalizing and automating part of the learning expe-

rience using ontologies allows for both. While there

are several good studies recently that use ontologies

with gamiﬁcation the aim tends to add gamiﬁcation

features to an existing environment instead of per-

sonalizing them (Garcia et al., 2017). Testing a new

hypothesis requires only deﬁning it and learners who

met the criteria will experience the automatically ad-

Imre, Z.

Ontology based UX Personalization for Gamiﬁed Education.

DOI: 10.5220/0009414804150422

In Proceedings of the 15th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2020), pages 415-422

ISBN: 978-989-758-421-3

415

justed mechanics and user interface. After trial the

data needs analysis. In the meantime, changing the

ontology to reﬂect the best current research will yield

the learning experience best supported by science.

When tailoring structural and semantic elements

on the individual level we need to base these on the

learner’s measurements. We consider 3 types of data

into our model, but the model can be expanded to use

more or to ignore some. The 3 measurements consid-

ered are:

• Narrative preferences, see Section 3.1

• Personality traits (Big 5), see Section 3.3

• Player types (Bartle), see Section 3.4

This paper is organized as follows: Related work on

gamiﬁcation of education in Section 2, description of

system and model components in Section 3, model

and implementation details in Section 4, conclusion

and further work in Section 5, and ﬁnally acknowl-

edgements and references.

2 BACKGROUND

A literature review of empirical studies shows that

gamiﬁcation works (Hamari et al., 2014). It also

shows that it is not a silver bullet, and it needs to be

tailored. We can conclude that context is of high im-

portance when applying these techniques. If the con-

text is ignored, then gamiﬁcation techniques do not

always work. Adding achievements and social com-

parison to a purely utilitarian service had no effect on

the great majority of users and small effect for the rest

(Hamari, 2013). The authors point out that hedonistic

or mixed context favor gamiﬁcation.

Gamiﬁcation elements in education have been

around for a long time, but gamiﬁcation as a concept

and its systematic application have only been used for

a decade or so. A systematic review of gamiﬁcation in

education has shown that there is an increasing trend

to use these elements (Dicheva et al., 2015). Most

applications, according to the study, focus on a small

number of game mechanics. Context matters in the

application, but it is not always considered.

Gamiﬁcation in the industry has a wider scope

(Raftopoulos et al., 2015). The primary purpose ac-

cording to the study are: customer loyalty, marketing,

education, recruitment, innovation, community devel-

opment, and motivation. The target audience: inter-

nal staff, customers, suppliers, community. With the

wider scope, compared to the education of students,

one gets a greater variety in techniques.

The effect of extrinsic rewards on schoolchildren can

have detrimental effects so care must be taken, less so

on university students (Deci et al., 2001). If care is

taken, extrinsic motivators can increase intrinsic mo-

tivators. The optimum approach, according to the

meta-analysis, is to provide more interesting activi-

ties, more choice, and ensure tasks are optimally chal-

lenging. Reward types and contingencies should be

informational rather than controlling.

In the case of JFDI Academy the authors used

several predeﬁned gamiﬁcation elements applied at

group level. They taught introductory programming

as an online game (Leong et al., 2011). They used the

narrative of a Sci-Fi world, had achievements, mis-

sions, a 24h grading feedback loop. The results were

positive, with marginally larger grades for a similar

quantity of work, compared to previous semesters.

3 SYSTEM COMPONENTS

3.1 Gamiﬁcation Elements

The most accepted deﬁnition of Gamiﬁcation is: “The

use of design elements characteristic for games in

non-game contexts” (Deterding et al., 2011). This

deﬁnition leaves us with a wide array of tools, game

mechanics, and design choices of how and what to

implement for our learners. In the industry and pri-

vate sector many game mechanics have been tried in

various contexts (Raftopoulos et al., 2015). And it

should be mentioned that only a subset of all mechan-

ics should be used in a system. To reduce the scope

of the model, only the most common elements have

been integrated into our ontology.

Instant feedback is one of the basic gamiﬁcation

tools. It shortens the work-reward cycle and it is

doable on computers. In the case of computation-

ally intensive checks, small artiﬁcial delays can be

added in the animation to give the impression of in-

stant feedback. The classical example is a loading

screen, which is displayed while the application starts

up, giving the impression of something happening in

the mean-time.

Points of many shapes and forms represent a re-

ward for desired behavior, a form of currency, it may

be the primary measurement of progression or status.

Certain systems apply multiple point types to balance

objectives. Most often they are called points. In edu-

cation, these usually take the form of grades, in a role-

playing game(RPG) setting they are usually called ex-

perience.

Achievements or badges are speciﬁc rewards for

a prespeciﬁed behavior separate from points and usu-

ally unique. Real world examples are medals in the

military or merit badges in the scouts. The obtainment

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

416

of badges represents extra goals, and some player

types go out of their way to obtain them, mainly the

achiever. By tailoring achievements to player types

(Bartle, 1996) on a structural or semantic level greater

interest can be achieved.

Missions and quests represent objectives to strive

for. They may be required or optional. In this context

they must be clear and veriﬁable.

Status, success, recognition is usually imple-

mented in the form of leaderboards. These are lists of

some or all participants, ordered usually by the num-

ber of points gathered. The primary role of leader-

boards is to show status and skill, it can be a great

motivator or demotivator depending on the context.

The concept of leveling is a progression mechanic,

it originates from tabletop RPGs (Gygax and Arne-

son, 1974). After acquiring several points (usually

referred to as experience) a milestone is reached. By

tying bonuses to a level, it becomes a good motiva-

tional tool and a measure of progress. Leveling is dif-

ferent from levels in games, the former is essentially a

set of milestones optionally accompanied by rewards,

the latter is the stage where the game is played.

Narrative is one of the strong suits of games. A

story aids in immersion and gives a sense of purpose

to solving exercises. The two pitfalls of a narrative

in a gamiﬁcation setting are quality and speciﬁcity. A

poorly written narrative can detract from the experi-

ence more than it can add to it, in these situations no

narrative is the better option. On the aspect of speci-

ﬁcity, if all other elements of the experience being the

same, people react differently to different types of sto-

ries. Personalized story genre delivery is possible us-

ing software, even if it’s labor intensive to prepare.

3.2 Automated Grading

This paper is a continuation of our work on (Zsig-

mond, 2019). To achieve instant feedback, it is neces-

sary to automatically check the learner’s solution to a

task. The problem of automatic analysis and grading

ranges from the complicated to the impossible. For

the study of computer science, it usually boils down

to: correctness check, coding style check, plagiarism

check and various speciﬁc technical details check.

Correctness check means to verify that the program

runs according to speciﬁcation. Speciﬁcation in this

context means that the program having received cer-

tain input under certain preconditions produces an ex-

pected output that meets the postconditions. The most

widely used method to check this is to run the pro-

gram with a set of provided inputs and expect a set

of outputs. This can be automated if the program that

is tested is monitored by another program written for

this purpose.

Code style check is a subset of static code anal-

ysis, that veriﬁes that the code is in accordance with

readability, structure and documentation guidelines.

In academia and business this is usually done by an

experienced programmer aided by static code anal-

ysis. Many tools have been created to automate as

much of this process as possible (Emanuelsson and

Nilsson, 2008). These tools can be used to automate

sufﬁcient amount of the process to aid gamiﬁcation

efforts.

Plagiarism check is verifying if the students try

to cheat by copying their peer’s work. Plagiarism

check is one of the necessary functionalities that can-

not be completely automated. (Hage et al., 2010)

Even though it cannot be fully automated this func-

tionality needs not be fast. Main issues are project

ﬁles that are generated by various integrated devel-

opment environments that should not be checked and

that sometimes it leads to false positives which would

be detrimental to learning. There are many special

cases and situations in practice that would lead to

false positives. Our solution was to take the reports

generated by Stanford’s Measure Of Software Simi-

larity tool (MOSS) (Schleimer et al., 2003) and check

the ones with high similarity manually.

By various speciﬁc technical details check we

mean tests that are speciﬁc to any given course. These

need to be customized to the course and can be any-

thing from object inheritance to SQL queries. The

main issue here is that these custom checks need to

be integrated in the system in a dynamic way.

Solutions that aim to do this can be found that are

smaller in scale (Cheang et al., 2003), and as such the

ﬁeld to cover is vast. With that in mind this section

will only focus on gamiﬁcation of learning.

3.3 Psychological Traits

In psychology, traits can be deﬁned as relatively

long-term patterns of behavior, thought, and emotion,

which differ on an individual basis. We choose to

use in our model the Big Five personality traits tax-

onomy, which is widely used in psychology (Gold-

berg, 1993). This theory uses descriptors of common

language and association between words to categorize

individuals and suggests ﬁve broad dimensions com-

monly used to describe the human personality. The

ﬁve factors have been deﬁned as openness to experi-

ence, conscientiousness, extroversion, agreeableness,

and neuroticism. The validity of these traits in educa-

tion has been experimentally veriﬁed (Poropat, 2009).

Each trait consists of facets, or dimensions which are

used to measure it. In the following short descriptions

Ontology based UX Personalization for Gamiﬁed Education

417

of the different traits and their facets we list only those

on the high end of the spectrum. If a person would

score low on a spectrum they would be considered the

opposite of the facet listed below. For example, high

dutifulness would follow the rules, while low dutiful-

ness would break the rules.

Openness to experience includes having active

imagination, aesthetic sensitivity, wide interests, and

being imaginative and insightful behavior. The asso-

ciated facets are: Fantasy, Aesthetics, Feelings, Ac-

tions, Ideas, Values.

Conscientiousness includes having scrupulous,

meticulous, principled behavior guided or conform-

ing to one’s own conscience. The associated facets

are: Competence, Order, Dutifulness, Achievement-

striving, Self-discipline, Deliberation.

Extroversion includes having outgoing, talkative,

energetic behavior, projecting one’s personality out-

ward. The associated facets are: Warmth, Gregari-

ousness, Assertiveness, Activity, Excitement-seeking,

Positive Emotions.

Agreeableness includes having kind, sympathetic,

cooperative, warm, and considerate behavior. The

associated facets are: Trust, Compliance, Altruism,

Straightforwardness, Modesty, Tender-mindedness.

Neuroticism includes having anxious, depressed,

self-conscious, impulsive, vulnerable behavior and

display angry hostility. The associated facets are:

Anxiety, Hostility, Depression, Self-consciousness,

Impulsiveness, Vulnerability.

Our ontology uses the aforementioned taxonomy,

it does not require it. With some adjustments our pro-

posed system can be used to the same effect with an

alternate theory if and only if it is a valid measurement

of the learner.

3.4 Player Types

In Bartle taxonomy of player types, 4 player types are

deﬁned: Killers Achievers, Socializers, and Explor-

ers (Bartle, 1996). These 4 characters represent quad-

rants along the preference for interacting with other

players vs. exploring the world axis and preference

for interaction vs. unilateral action axis. The different

types have different preferred actions during game-

play. Each player falls on the spectrum of these scales.

Killers types thrive on competition with other

players instead of competition with the world itself.

In our hypothesis the mechanics that suit the killer

proﬁle are leader-boards to show status, battle to di-

rectly engage with other people, and displays of mas-

tery.

Achievers prefer tangible rewards in the world,

these may be points, badges, loot, prestige etc. They

tend to strive to “beat” the game or having 100% com-

pletion. In our hypothesis the mechanics that suit the

achiever are points, achievements, and progression.

Socializers play games for the social aspects

rather than the game itself. The game being no more

than a tool to interact with people. In our hypothe-

sis the mechanics that suit the socializers are group

quests and missions.

Explorers prefer discovering details of the world,

hidden features, puzzles, bugs, and “Easter eggs”

(hidden references). In our hypothesis the mechanics

that suit the explorer is narrative, optional missions,

and achievements.

Since an engaging experience for each player type

differs we need to ensure that in the context of edu-

cation where learners join from the entire spectrum

we do not ignore any player type. In the context of

personalization this means that certain features need

to be exaggerated or dropped completely based on

player needs. Our taxonomies and decision making

must take this aspect into account.

Figure 1: Ontology excerpt of Game Mechanics and

Learner Proﬁle.

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

418

3.5 Ontology and Ontology-based

Reasoning

The classical deﬁnition of an ontology as often used

is “explicit speciﬁcation of a conceptualization” (Gru-

ber, 1995). In a more practical, computer science per-

spective it boils down to modeling a domain of knowl-

edge with the use of classes, attributes and relations

between classes.

Ontologies are used in most ﬁelds to organize data

into knowledge, usually in the form of taxonomies.

The power of these formalisms is in being an adap-

tor between ﬁelds by experts agreeing on the relation

between their terms.

In the ﬁeld of education Ciuciu (Ciuciu and Tang,

2010) mentions DOGMA paradigm Developing On-

tology Grounded Methodology and Applications. In

the paper they use DOGMA to create personalized

testing and suggestion for further study of medicine,

here they use ODMF (Tang et al., 2010) for matching.

Yu (Yu et al., 2008) recommends content provision-

ing for context-aware ubiquitous learning, Baloian’s

(Baloian et al., 2004) approach considers the learner’s

preferences and available resources to recommend

learning material, while Schmidt (Schmidt and Win-

terhalter, 2004) tries modeling the learner’s environ-

ment to capture the required context for e-Learning

material delivery.

In our work we focused on mapping on the one

hand psychological traits, player types, and narrative

preferences measured for individuals to gamiﬁcation

elements. While presenting the ontology is not the

focus of this paper, we included excerpts to illustrate

the proposed solution, see Figure 1.

4 PROPOSED SYSTEM

4.1 Metrics and Formulae

When constructing the ontology, the problem of de-

ciding what game mechanic to use, based on the myr-

iad of metrics collected on the learner arose. Care

must be taken because, for a learner with high anxi-

ety levels, it is detrimental to expose them to a highly

competitive setting. The opposite is also true.

For each game mechanic there are several ways

to show them and several experiences that they yield.

Different metrics need to be associated to different

types, within one game mechanic. In Figure 2 we

illustrate the ﬁve different sub-types for the leader-

board mechanic and the metric inﬂuencers. The sub-

types are: no leaderboard, showing only the next

two students the rest being hidden, all students in

the group/team, everybody in the year/currently in the

company, and everybody since there are metrics. To

this game mechanic, all player types contribute, but

only Neuroticism, and Extraversion on the traits side.

Figure 2: Leaderboard mechanic subtypes and metric inﬂu-

encers.

The solution we propose is to deﬁne ranges for the

different metrics and use Formula 1 and 2 to make

decisions on it. Formula 1 guaranties that minimum

values are respected across all learner metrics for a

given mechanic sub-type while at least one maximum

value is also respected. In the formula M1 . . . Mn

refers to each relevant learner metric for example Ex-

traversion or Achiever. Formulas V(x) yields the mea-

sured value of a proﬁle in a [0 100] interval, for ex-

ample, Extroversion value of 35. Vmin(x) yields the

minimum threshold of the learner metric for the cur-

rent gamiﬁcation mechanic, while Vmax(x) yields the

maximum threshold. The formula assumes that the

values used are increasing and disjunct. The mathe-

matical formula is:

(∀x ∈ {M

...M

},V (x) > V

min

(x))∧

(∃x ∈ {M

...M

},V (x) < V

max

(x))

For an example of values used for the Leaderboard

mechanic see Table. 1. To see an implementation of

the formula for NextTwo variant of the Leaderboard

mechanic in prot

e see Fig. 3. For consistency the

values for neuroticism values are inverted using the

100-NeuroticismValue formula.

While Formula 1 gives the expected results for any

single gamiﬁcation mechanic, a general case should

be presented for non-prot

e implementation. For-

mula 2 extends Formula 1 by considering all gamiﬁ-

cation mechanics in the form of G1 . . . Gm. Func-

tions Vmin(x, y) yields the minimum threshold for

the metric and mechanic, while Vmax(x, y) yields the

maximum threshold.

∃x ∈ {G

...G

}(∀y ∈ {M

...M

},V (y) > V

min

(x, y))∧

(∃y ∈ {M

...M

},V (y) < V

max

(x, y))

The ontology can be queried using SPARQL to get the

structural and semantic gamiﬁcation elements for a

Ontology based UX Personalization for Gamiﬁed Education

419

Table 1: Proposed values for Leaderboard mechanic.

M1 M2 M3 M4 M5

Neuroticism Extroversion Killer Achiever Socializer

Min Max Min Max Min Max Min Max Min Max

G1 NoLeaderboard 0 20 0 20 0 5 0 5 0 10

G2 NextTwo 20 40 20 40 5 20 5 20 10 30

G3 Group 40 60 40 60 20 50 20 50 30 50

G4 Year 60 80 60 80 50 60 50 60 50 70

G5 AllTime 80 100 80 100 60 100 60 100 70 100

Figure 3: NextTwo variant of the Leaderboard mechanic implementation in prot

given learner. Example of a structural element would

be a Leaderboard sub-type, while semantic would be

Story Preference sub-type.

4.2 GamifyCS Project

The proposed system (Zsigmond, 2019) is par-

tially implemented as a functional website written in

ASP.NET MVC. The system is being used for aca-

demic research and evaluation purposes. It is be-

ing further developed to serve as a gamiﬁcation test-

bench and scientiﬁc tool. The user interface was de-

signed to be minimalist and responsive. The goal was

to remove distractions and appeal to the current gen-

eration’s standards.

For successful usage of the ontology based UX

tailoring, as well as data gathering for the psycholog-

ical thresholds, we needed a platform to support the

various gamiﬁcation mechanics mentioned. Each me-

chanic on it’s own requires certain technical solutions.

These are the mechanics, and the way they were either

implemented or prototyped in our system. An archi-

tectural view of the project with currently used and

prototyped subsystems can be found in Figure 4.

The described ontology was created and tested in

prot

e. Integration with the GamifyCS project was

prototyped and full integration awaits further gami-

ﬁcation mechanic implementations and a validating

experiment. The measurement of Big 5 personality

traits and determination of Player types is beyond the

scope of this project and should be measured by stan-

dardized tools. Once they are made and imported, the

values must be converted to the 0-100 scale which the

ontology uses.

Instant feedback: The most notorious from a tech-

nical point of view, we had to implement an auto-

mated correctness checking mechanism. Since the

tool had to support a wide range of programming lan-

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

420

Figure 4: Architectecture of GamifyCS project.

guages we opted for an external compiler / interpreter.

In our latest study the students uploaded their solu-

tions, we compiled / interpreted their code, redirected

the standard input/output then ran a predeﬁned se-

quence of provided inputs and expected outputs. Then

displayed the results in under a second.

Narrative and Missions: In our latest study stu-

dents could choose among 4 different story styles.

We created the website such that students could be

assigned any number of exercises, which in turn had

4 different texts associated to them for each narrative

style. While doing the same exercise different stu-

dents would see read about their assignments as if part

of that world.

Achievements: We prototyped awarding achieve-

ments based on static code analysis and completed

assignments. A predeﬁned logic together with a pic-

ture and description was created for each of the 5 pro-

totyped achievements and the infrastructure to award

them. This remains the subject of our next study.

Points and Leaderboards: At the moment points

are used only in the form of grades. The awarding

of grades at the successful completion of all tests for

a given exercise or mission was created but proved

to cumbersome to use in practice. A rewriting of the

feature is future work. Leaderboards were not proto-

typed at all.

Avatars: Avatars would be represented by picture

and name provided by the students with text substitu-

tion in the narrative. Currently this is not supported

or prototyped.

Status, Leveling and Mastery: Status related fea-

tures would involve the possibility to view each

other’s avatars / proﬁles to see points and achieve-

ments or a leaderboard, which remain to be proto-

typed once those features are done. Leveling and

Mastery will be prototyped together with fully inte-

grated achievement system.

5 CONCLUDING REMARKS

The focus of the paper was to create a solution that

allows for the easy use of gamiﬁcation, is the most

applicable in education and facilitates basic research

in the ﬁeld. We feel that personalizing the learning

experience using ontologies achieves both. Testing a

new hypothesis requires only deﬁning it and learners

who met the criteria will experience the automatically

adjusted mechanics and user interface. In the mean-

time, changing the ontology to reﬂect the best current

research will yield the learning experience best sup-

ported by science.

The system developed has great potential for ex-

perimentation. It is also an opportunity for educa-

tion. Since it allows for features to be switched off via

ontology and can be freely extended various exper-

iments can designed around the tool. Since we store

various code and usage data, data mining methods can

Ontology based UX Personalization for Gamiﬁed Education

421

be used to improve the tool and the experience in the

future.

The further features that should be added to the

system are: continually reﬁne the ontology, use full

blown sandboxes to run the executables in, the sup-

port for running queues to balance load and improve

measurements, the support for more programming

languages and UI languages.

Since the ﬁeld of gamiﬁcation is vast and so

far under-researched, there are many techniques that

should be added and tried, considerations for pacing

might be added, for example, as well as for mastery

to fuse in solutions that facilitate the study of ﬁne

grained computer knowledge.

REFERENCES

Baloian, N., Galdames, P., Collazos, C. A., and Guerrero,

L. A. (2004). A model for a collaborative recom-

mender system for multimedia learning material. In

International Conference on Collaboration and Tech-

nology, pages 281–288. Springer.

Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players

who suit muds. Journal of MUD research, 1(1):19.

Cheang, B., Kurnia, A., Lim, A., and Oon, W.-C. (2003).

On automated grading of programming assignments

in an academic institution. Computers & Education,

41(2):121–131.

Ciuciu, I. and Tang, Y. (2010). A personalized and col-

laborative elearning materials recommendation sce-

nario using ontology-based data matching strategies.

In OTM Confederated International Conferences” On

the Move to Meaningful Internet Systems”, pages

575–584. Springer.

Deci, E. L., Koestner, R., and Ryan, R. M. (2001). Extrinsic

rewards and intrinsic motivation in education: Recon-

sidered once again. Review of educational research,

71(1):1–27.

Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011).

From game design elements to gamefulness: deﬁn-

ing gamiﬁcation. In Proceedings of the 15th inter-

national academic MindTrek conference: Envisioning

future media environments, pages 9–15. ACM.

Dicheva, D., Dichev, C., Agre, G., Angelova, G., et al.

(2015). Gamiﬁcation in education: A systematic

mapping study. Educational Technology & Society,

18(3):75–88.

Emanuelsson, P. and Nilsson, U. (2008). A comparative

study of industrial static analysis tools. Electronic

notes in theoretical computer science, 217:5–21.

Garcia, F., Pedreira, O., Piattini, M., Cerdeira-Pena, A., and

Penabad, M. (2017). A framework for gamiﬁcation

in software engineering. Journal of Systems and Soft-

ware, 132:21–40.

Goldberg, L. R. (1993). The structure of phenotypic per-

sonality traits. American psychologist, 48(1):26.

Gruber, T. R. (1995). Toward principles for the design of

ontologies used for knowledge sharing? International

journal of human-computer studies, 43(5-6):907–928.

Gygax, G. and Arneson, D. (1974). Dungeons and dragons.

Tactical Studies Rules Lake Geneva, WI.

Hage, J., Rademaker, P., and van Vugt, N. (2010). A com-

parison of plagiarism detection tools. Utrecht Univer-

sity. Utrecht, The Netherlands, 28:1.

Hamari, J. (2013). Transforming homo economicus into

homo ludens: A ﬁeld experiment on gamiﬁcation in

a utilitarian peer-to-peer trading service. Electronic

commerce research and applications, 12(4):236–245.

Hamari, J., Koivisto, J., Sarsa, H., et al. (2014). Does gami-

ﬁcation work?-a literature review of empirical studies

on gamiﬁcation. In HICSS, volume 14, pages 3025–

3034.

Kiesler, S., Kraut, R. E., Koedinger, K. R., Aleven, V., and

Mclaren, B. M. (2011). Gamiﬁcation in education:

What, how, why bother? Academic exchange quar-

terly, 15(2):1–5.

Leong, B., Koh, Z. H., and Razeen, A. (2011). Teaching

introductory programming as an online game. De-

partment of Computer Science, National University of

Singapore.

Poropat, A. E. (2009). A meta-analysis of the ﬁve-factor

model of personality and academic performance. Psy-

chological bulletin, 135(2):322.

Raftopoulos, M., Walz, S., and Greuter, S. (2015). How

enterprises play: Towards a taxonomy for enterprise

gamiﬁcation. In Conference: Diversity of Play:

Games–Cultures-Identities. DiGRA. Recuperado de

https://goo. gl/3PD4f9.

Schleimer, S., Wilkerson, D. S., and Aiken, A. (2003). Win-

nowing: local algorithms for document ﬁngerprinting.

In Proceedings of the 2003 ACM SIGMOD interna-

tional conference on Management of data, pages 76–

85. ACM.

Schmidt, A. and Winterhalter, C. (2004). User context

aware delivery of e-learning material: Approach and

architecture. Journal of Universal Computer Science,

10(1):28–36.

Shatz, I. (2015). Using gamiﬁcation and gaming in order to

promote risk taking in the language learning process.

In Proceedings of the 13th Annual MEITAL National

Conference, Haifa, Israel, pages 227–232.

Skinner, B. F. (1935). Two types of conditioned reﬂex and

a pseudo type. The Journal of General Psychology,

12(1):66–77.

Tang, Y., Meersman, R., Ciuciu, I.-G., Leenarts, E., and

Pudney, K. (2010). Towards evaluating ontology

based data matching strategies matching strategies,

evaluation methodology and results. Lecture Notes in

Computer Science.

Yu, Z., Nakamura, Y., Zhang, D., Kajita, S., and Mase, K.

(2008). Content provisioning for ubiquitous learning.

IEEE Pervasive Computing, 7(4):62–70.

Zsigmond, I. (2019). Automation and gamiﬁcation of com-

puter science study. Studia Universitatis Babes

-Bolyai

Informatica, 64(2):96–105.

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

422