Gamification and Evaluation of the Use the Agile Tests in Software

Quality Subjects: The Application of Case Studies

Isaac Souza Elgrably and Sandro Ronaldo Bezerra Oliveira

Graduate Program in Computer Science, Institute of Exact and Natural Sciences,

Federal University of Pará, Belém, Pará, Brazil

Keywords: Gamification, Learning, Teaching, Education, TDD, Agile Tests.

Abstract: With the greater immersion of software development teams to agile methods and practices, it became

necessary for students to have earlier contact with Agile Testing practices. Thus, this study aims to use the

gamification concepts to stimulate the support to teach and engage the motivation of students in the

Software Quality subject taught in postgraduate and undergraduate courses in computer science. For this,

classes were set up to teach agile tests that used games elements as motivation for students. Therefore, this

research resulted in an enrichment of the knowledge of these students in testing practices. This work aims to

contribute to the teaching of agile test practices for students, aiming at a better preparation for the software

development market. It was also verified that the use of gamification elements for the teaching of agile tests

was efficient, because the participating students dedicated themselves more to the tasks and were

participative in all the different types of classes.

1 INTRODUCTION

Software Testing is one of the main activities that

are part of software quality assurance. Additionally,

studies show the increasing adoption of agile

methodologies in the software industry, where until

the 2018 three out of four Information Technology

companies in the world will be using the agile

methodologies in their work, prioritizing the

delivery of value, efficiency, security and

assertiveness (Gartner, 2015). As one of the agile

techniques, we have the Agile Tests, where,

according to Nadalete and Kryszczun (2013), agile

testing requires a strong adaptation in the routine

and dynamics of the team in relation to the

development process adopted.

However, the teaching of agile tests still has

several limitations, since this topic requires that the

student has a certain knowledge of programming and

techniques of software quality, which may not

happen the adequate absorption of these contents by

the students when teaching is restricted to theoretical

classes. Thus, the phenomenon of gamification

(Werbach and Hunter, 2012), which consists in the

use of the games elements outside their context, is

used to mobilize the subjects to act, to help solve

problems and to promote learning (Kapp, 2012).

Additionally, Kapp (2012) defends the proposal to

use the games elements to involve the students to

stimulate their learning. Perhaps one of the great

advantages of using gamification in education is to

provide a system in which students can visualize the

effect of their actions and learning as the activity

progress happens, making it easier to understand the

relationship of the parts with the everything, as in

games (Fardo, 2013).

In this context, we used a set of programming

techniques adopted by agile development teams,

such as DOJO (Luz and Neto, 2012) and LAB

(Programming Laboratory), with the objective of

providing a practical teaching of agile tests, giving a

greater emphasis on practice of TDD (Test Driven

Development). A technique developed by Beck

(2010), which describes an evolutionary approach to

development, where it must write a test before

writing a sufficient production code to perform this

test and perform its refactoring, thus doing that code

is born tested. In parallel with these classes, students

participated in a game, as in (Freitas et al., 2016), in

which they got achievements in each of the classes.

Thus, the objective of this paper is to report and

discuss the results obtained in the application of

these techniques for the teaching of Agile Testing in

416

Elgrably, I. and Oliveira, S.

Gamiﬁcation and Evaluation of the Use the Agile Tests in Software Quality Subjects: The Application of Case Studies.

DOI: 10.5220/0006800304160423

In Proceedings of the 13th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2018), pages 416-423

ISBN: 978-989-758-300-1

a Software Quality subject, which was taught in

postgraduate and undergraduate courses in computer

science. The study presented in this paper has the

purpose of analyzing and discussing the results

obtained by the classes with the teaching practices

mentioned, considering the gamification as a

motivating point. Thus, this research aims to

contribute to the teaching of agile test practices for

students, aiming at a better preparation for the

software development market, and to discuss how

the elements of gamification can contribute as a

motivating teaching technique and trying to show

that the gamification is much more than a set of

points, medals and progress bars, as well as

criticized (Chou, 2015).

One of the reasons for the concern with the

teaching of Agile Testing with a focus on the

practice of TDD is that in the research conducted by

Elgrably and Oliveira (2017), it was verified that the

TDD practice is one of the main agile test techniques

used in the software development of Brazilian

industry and most of the MPT-Br (Brazilian Test

Improvement Model) consultants find it difficult to

implement the technique, where one of the main

reasons is the little or no prior knowledge of the

technique by the developers.

2 RELATED WORKS

In order to identify works that propose the use of

gamification as support to teaching, not necessarily

to the teaching of Agile Tests, a search was made

in the specialized literature with emphasis on

published papers. No work found focuses on the

teaching of Agile Testing, using gamification as

a motivation to aid teaching. This already

underscores the importance of this study,

demonstrating the relevance and originality of the

case studies carried out.

There are studies that propose the use of

gamification as teaching support, without the use of

software or applications, only the elements of

gamification to support students. Thus, authors of

the (Freitas et al., 2016) work created a gamified

environment for the subject of Fundamentals of

Computer Architecture. The central element of the

game was the duel of knowledge among students

enrolled in the subject, either individually or in a

group, where the purpose of the game is to

accumulate points that were the final mark of the

student in the subject. The work described in this

paper has the differential of application of

techniques used by agile development teams, such as

DOJO and Programming Laboratory; to be more real

challenges for students, because it motivates them to

learn in depth techniques to be able to meet

challenges of programming that were developed.

Finally, in the work of Matsubara and Silva

(2017), the authors present a process of gamified

experience with volunteers in a laboratory, aiming

the teaching of the subject of Software Engineering,

in order to know if “the elements of games and the

game design enhances learning in a Software

Engineering subject?”. In this work, the game

elements are well defined using elements such as:

points, levels, challenges, and feedback, among

others. The work being presented in this paper has

the differential of having been carried out the Case

Study in a Software Quality subject, so different

from volunteers the competitive aspect was more

impacting, as the result of the Case Study reflected

directly in grades of the students in the subject, and

later the Case Study was replicated making changes

that were collected through interviews in class with

students at the end of the case study, enabling an

improvement in the learning process and in the

structure of the gamification.

3 THE SOFTWARE QUALITY

SUBJECT

The Software Quality subject, which used in the

Case Study described in this paper, is offered every

semester to regular and special postgraduate (master

and doctoral courses) and undergraduate students in

Computer Science. It is a subject considered as

optional for the training of these students in the

Software Engineering research area. Currently, the

course is taught by a professor with a broad

academic experience (with a master's, doctoral and

postdoctoral degree in Software Quality, having

graduated more than 40 master's and doctoral

students in this research area) and professional

(being certified in MPS.BR, CMMI, CERTICS,

MEDE-PROS – Software Product Quality

Assessment Method). All students of this professor

act actively in the administration of this subject

based on monitoring and practices. The discipline

discusses aspects related to Software Engineering

and its subareas and its content is made through

expository classes, debates, exercises and

elaboration of practical works accompanied by the

responsible professor and his monitors.

Gamiﬁcation and Evaluation of the Use the Agile Tests in Software Quality Subjects: The Application of Case Studies

417

4 THE GAME OF AGILE

TESTING

The Software Quality subject was the context of the

case study that will be presented and had the

following Research Question (QP): Does the use of

gamification aid the engagement of the class and

increases the learning about the topic of Agile Tests

in the Software Engineering area?

In order to answer this QP, a gamified classroom

was created for this case study, which has the

following elements: physical space (classroom);

players (students); judges (monitors and professor);

the applied teaching methodology and rules of the

game; and other gamification elements that were

used, detailed in the next subsection.

4.1 The Methodology of the Case Study

The stages of accomplishment of the case study are

represented in Figure 1.

Figure 1: Stages of Realization of the Case Study.

The case study was carried out twice, the first in

a postgraduate class, divided in 6 days of classes,

being: 2 days of classic lectures with the Software

Testing theme, as well as their history, types of

Software Tests, Software Testing in traditional and

agile contexts, and especially the agile practice of

TDD and the "Agile Testing Quadrant" created by

Marick (2003) and modified by Gregory and Crispin

(2014); in the third class students participated in a

DOJO of the Randori type; in the fourth class there

was a LAB practice with pairing; and in the sixth

class all students in the Case Study participated in a

feedback class on the practices used for teaching and

if the gamification met the expectations of the QP.

The second case study was carried out in an

undergraduate class and the feedback that was

collected in the first case study was taken into

account, so there was an addition of one more class.

In both case studies, at the beginning of the first

day of class, the students were informed that they

would be participating in a gamified classroom on

Agile Testing, and a spreadsheet with a gamification

was presented, an instrument responsible for

punctuating all the practices that the students would

exercise in the subject, such as: Daily Penalties,

which could be Fouls without Warning, Fouls with

Warning, Delays and Penalties (use cellphone,

disrupt class, use of laptop, etc.); Daily Bonuses,

which could be the presence of the students, an

initiative to question something about what was

being taught, suggestions for possible improvements

of the classes and participation, which would be any

other question about the classes that generated a

discussion in the class; General Bonus of Case

Study, are unique bonuses that were worth more

points for students.

These punctuation generated the gamification

score, in which the result of the first case study for

the postgraduate students had that the student with

the highest grade received a prize, and also

generated part of the grade of the students in the

Software Quality subject, using the following

formula: LAB * 0.04 + EvaluativeActivity * 0.04 +

Participations * 0.2. It is worth emphasizing that for

the participations were added all the Daily Bonus,

and the students who obtained more than 10

participations would receive the maximum score,

otherwise they would have the participation number

multiplied by 0.2. In the second case study, the

formula was slightly updated in order to increase the

students' competitiveness and performance: LAB *

0.04 + EvaluativeActivity * 0.04 + Participations *

0.032 (Number obtained through a rule of 3 with the

highest grade of participation in the case study). It is

worth emphasizing that the student with the most

participation points in the gamification level scored

the grade of the group above in Participations, which

were 62 points of participation.

4.2 The Game

The main element of the game is the competition

between the students, in which the knowledge on the

subject of Agile Testing was verified.

At the beginning of each class was presented the

previous result of the competition, showing the

points and the current Ranking of gamification, this

was done with the intention of promoting a fierce

competition among students, as in the (Hanus and

Fox, 2015) work. All the activities of the game,

when performed successfully, gave points. In Table

1 we can identify which elements of gamification

ENASE 2018 - 13th International Conference on Evaluation of Novel Approaches to Software Engineering

418

were used in each class and also the forms that the

students were able to score in each of them.

Table 1: Relationship between Class Types and

Gamification Elements and Points.

Class

Types

Gamification

Elements

Grades on the

Spreadsheets

Classical

Expository

Class

Points, Ranking,

Narrative

Presence, Participations

and Suggestions

DOJO

Class

Points, Ranking, List

of Challenges,

Learning Curve and

Activities in Teams

Presence, Participations,

Suggestions, Initiative and

Participation in DOJO

LAB Class

Points, Ranking, List

of Challenges,

Learning Curve,

Monitoring

Presence, Participations,

Suggestions, Initiative,

Participation in LAB,

Concluding of LAB and

Mini Game (LAB)

Evaluative

Class

Points, Ranking, List

of Challenges

Presence and Grade in the

Test

Feedback

Class

Points, Ranking,

Winning Prizes

Presence and Participation

In the first class was made the narrative of the

importance of the students' participation in the case

study, how the gamified classroom would work and

the punctuations. It was reported that there were two

judges (monitors) who were responsible for

punctuating the students' participation and at the end

of each class they would discuss the results that had

been obtained. From the beginning it was

encouraged that the students did not miss and

participate as actively as possible in the classes,

since their attendance and participation would

generate points for gamification.

In the classical expository classes the topics

covered in the Case Study were presented to the

students, an introduction to Software Testing was

done, later presented the main techniques and tools

used in the Agile Testing quadrant, with a greater

emphasis on TDD. In these classes the students

scored with their presence, active participation with

questions and discussions, making suggestions of

new topics that could be approached and

improvements that could be made in the material

used.

In the third class there was a greater focus on the

agile practice of TDD, presenting a kind of class of

DOJO of the Randori type, in which a list of

challenges related to code construction and testing.

In Table 2 it is possible to visualize the

functionalities and the test cases of this challenge.

In the beginning of the challenge one of the

monitors performed the first functionality and the

first test case with the support of the students

participating in the case study. Afterwards, the

students stayed in pairs, one pilot and the other co-

pilot, to finish the list of challenges and the other

students were as audience, being able to analyse

what was being done by the pair and discuss the

decisions, and to build strategies for when their

respective times of action in the challenge. Each

student had a time of 5 minutes in each function.

Table 2: Functionalities and Test Cases of DOJO

Challenge.

Functionalities Test Cases

It must make a Deposit Making a Deposit

It must consult the Balance Making a Bank Draft

It must make a Bank Draft

Duplicating a value of a

deposit

Is must add up to Investment Cashing out if value is less

than what is available

It must validate if a Bank Draft

is valid

Invaliding the Bank Draft

greater than available

In the fourth class of the second Case Study the

focus was on unit testing. The students performed

this mini-game in the format of LAB, thus formed

pairs for one of the activities, in which the students

made a list of challenges, with the possibility of

adding more test cases to increase their scores. In

this challenge the system codes were delivered to the

pairs and they had to build five test cases at least as

seen in Table 3.

Table 3: Test Cases of LAB Challenge.

Store System Challenge

Test Cases

testingProductsValues;

testingProductsAmmount; testingAddProducts;

testingSearchProducts; testingRemoveProducts

In the fourth class (fifth class in the second case

study) the focus continued to be the use of TDD and

the students are in pairs to perform one of the

activities with greater grade of the case study, the

LAB. In this activity the students performed a list of

challenges, with the possibility of adding more test

cases to increase their scores. Table 4 shows the list

of challenges.

In this challenge, the monitors observed the

performance of the students and evaluated the

following criteria for the grade: if the activity was

completed; if the cycle of TDD was used by the

students; and whether the refactoring was used in the

development of functionalities; in addition to

completing the gamification with suggestions given

by the students about the LAB activity, evaluating

the initiative of each one in the activity to perform

functionalities and test cases more than what was

requested in the challenge.

Gamiﬁcation and Evaluation of the Use the Agile Tests in Software Quality Subjects: The Application of Case Studies

419

Table 4: Functionalities and Test Cases of LAB

Challenge.

Challenge 1

Functionalities

Writing Sum Function

Refactoring Sum Function (If necessary)

Test Cases

summingTwoPositiveNumbers

summingPositiveNumberWithNegative

summingDecimalNumbers

summingTwoZeroNumbers

Challenge 2

Functionalities

Writing Subtraction, Multiplication and

Division Functions

Refactoring the Created Functions (If

necessary)

Treating Division by Zero

Test Cases

Performing tests that validate its operation

(Operation with two positive numbers,

positive number with negative, decimal

numbers and operations with two zero

numbers)

Challenge 3

Test Cases

Making a test function with an assertion,

with the result of the expression (using the

classes already created): ((30 + 10) - 10) * 2

In the fifth class (sixth class in the second case

study), the results of the lessons learned were

evaluated, where an evaluation activity was carried

out with 9 objective questions and 1 discursive on

the topic of Agile Tests. This activity had greater

grade in the final score of the gamification, and from

it would be possible to analyze the students' learning

through the teaching practices that were chosen.

Finally, the sixth class (seventh class in the

second Case Study) was used to receive feedback

from students about what they found of the use of

gamification and the types of class adopted for the

case study, analyzing their interest in gamification,

agile testing, TDD, etc. In this class the students

earned punctuation with their presence and from the

following scale of participation: Little participation,

Regular Participation, Good Participation and

Excellent Participation.

5 THE EVALUATION OF THE

USE OF AGILE TESTING AND

THE STUDENTS MOTIVATION

In order to evaluate the learning, the results of the

LAB and the evaluation activity were used. Table 5

shows the LAB result from the first Case Study and

Table 6 from the second Case Study. In these tables

the ST acronym represents the Student.

Table 5: Results of the LAB II Challenge of the First Case

Study.

Challenge 1 Challenge 2 Challenge 3 TO

TAL

AC FT UR ND AC FT UR ND AC FT UR ND

ST 1 1 1 1 10 0 1 1 7 0 0 0 0 60

ST 2 1 1 1 10 1 1 1 10 1 1 1 10 100

ST 3 1 1 1 10 0 1 1 7 0 0 0 0 60

ST 4 1 1 1 10 1 1 1 10 1 1 1 10 100

ST 5 1 1 1 10 0 0 0 0 0 0 0 0 40

ST 6 1 1 1 10 1 1 1 10 1 1 1 10 100

ST 7 1 1 1 10 0 0 0 0 0 0 0 0 40

ST 8 1 1 1 10 1 1 1 10 1 1 1 5 90

ST 9 1 1 1 10 1 1 1 10 1 1 1 10 100

ST 10 1 1 1 10 1 1 1 8 0 0 0 0 66

ST 11 1 1 1 10 1 1 1 10 1 1 1 5 90

ST 12 1 1 1 10 1 1 1 8 0 0 0 0 66

ST 13 0 0 0 0 0 0 0 0 0 0 0 0 0

ST 14 1 1 1 10 1 1 1 10 1 1 1 10 100

The result obtained in the LAB activity had

criteria for the total grade of the students, where

from the three challenges the student needed: to

complete the activity (AC); to use the TDD flow to

perform each activity (FT); to use the code class or

the test class refactoring (UR). If the student

completed the criteria in the activity, he / she would

receive a flag of 1, otherwise a flag of 0. In the end,

one of the monitors would correct the code and give

a grade between 0 and 10 points, of the Challenge

Grade (ND). The formula of the total calculation for

each challenge was: AC * 4 + FT * 3 + UR * 3 +

ND * 2, which totalled 30 points per challenge, and

all participants were rewarded with 10 points for

participating in the LAB. In the first case study it

was evaluated that the 3 challenges would each be

worth 30 points plus the bonus, worth 10 points,

totalling the final value of the LAB challenge in 100

points. In the second case study it was continued that

each of the 3 challenges would be worth 30 points

each, but the bonus was changed so that the Tests

and the extra methods performed by the students’

pairs had a value of 0 to 10 points, as can be seen in

Table 6: Daily Bonus in Participations Class (PA5).

This was done with the intention of stimulating the

competitiveness and trying to extract the maximum

of the effort of the pairs in the activity.

ENASE 2018 - 13th International Conference on Evaluation of Novel Approaches to Software Engineering

420

Table 6: Results of the LAB II Challenge of the Second

Case Study.

Challenge 1 Challenge 2 Challenge 3

AC FT UR ND AC FT UR ND AC FT UR ND

ST1 1 1 1 9 1 1 1 9 1 1 1 10 6 92

ST2 1 1 1 10 1 1 1 10 1 1 1 10 8 98

ST3 1 1 1 10 1 1 1 8 1 1 1 10 5 91

ST4 1 0 0 10 0 0 1 7 0 0 0 0 0 41

ST5 1 1 1 9 1 1 1 9 1 1 1 10 6 92

ST6 1 1 1 10 1 1 1 9 1 1 1 10 2 90

ST7 1 1 1 10 1 1 1 10 0 0 0 0 0 60

ST8 1 1 1 10 1 1 1 10 1 1 1 10 8 98

ST9 1 1 1 10 1 1 1 10 1 1 1 10 8 98

ST10 1 1 1 10 1 1 1 9 1 1 1 10 2 90

ST11 1 1 1 10 1 1 1 10 0 0 0 0 0 60

ST12 1 0 0 10 0 0 1 7 0 0 0 0 0 41

ST13 1 1 1 10 1 1 1 10 1 1 1 10 8 98

ST14 1 1 1 10 1 1 1 9 0 0 0 0 0 58

ST15 1 0 1 10 1 0 1 10 1 1 1 10 10 94

ST16 1 1 1 10 1 1 1 9 0 0 0 0 0 58

ST17 1 0 1 10 1 0 1 10 1 1 1 10 10 94

ST18 1 0 0 10 0 0 1 7 0 0 0 0 0 41

ST19 0 0 0 0 0 0 0 0 0 0 0 0 0 0

ST20 1 1 1 10 1 1 1 8 1 1 1 10 5 91

The students' results in the LAB class in the first

case study were considered satisfactory, with a mean

of 7.7 as a grade, with the exclusion of the missing

student. The possible causes cited by the students as

determining factors for the successful execution of

this challenge were highlighted in the feedback

class, such as: the use of pairing in the activity; and

in the pre-LAB class a DOJO of the Randori type

was performed. Student 13 missed the LAB activity;

soon he received the grade 0.

The students’ results in the LAB class in the

second case study were considered very satisfactory,

because even with the stimulus of competitiveness in

which only one pair obtained the 10 points in

Participations in the class, the general average of the

students was of 7.8, the student 19 missed. The

possible causes cited by the students as determining

factors for the successful execution of this challenge

were highlighted in the feedback class, such as: the

use of pairing in the activity; in the pre-LAB class a

DOJO of the Randori type was performed; and the

use of a non-evaluative LAB that enabled a first

experience with the practice. The student 19 missed

the LAB activity; soon he received the grade 0 in

this practice.

The other activity used for performance

evaluation was a traditional evaluative activity

(Test). The students also had results considered very

satisfactory. The spreadsheet used in the first case

study, containing the results that were obtained in

addition to several observations, is available at:

https://drive.google.com/file/d/0B1VyGWbovbpVckIt

bm1JbW h0c3c. Still on the result obtained in the

traditional evaluative activity of the first case study,

a peculiar case happened with the Student 6, who

obtained a maximum score in the LAB class, having

the largest number of participations in the case

study, but obtained one of the lowest grades in the

evaluation activity and in the feedback class. The

student when asked about the evaluation method of a



tional test stated that he always had a great

difficulty concentrating and excessive nervousness

when he participated in activities of this nature.

Most of the students present agreed that evidence in

the traditional model could cause a considerable

amount of stress for everyone.

The spreadsheet that was used in the second case

study is available at https://drive.google.com/open?

id=0B1VyGWbovbpVSVpwUXltVnNwZjA. In this

second case study there was another peculiar

occurrence, where the Student 3 obtained excellent

grades in Evaluative LAB, 91, and evaluative

activity, 9, but it was verified that he had some

difficulties of expressing himself and of speaking.

Thus, in the extra participations of the gamification

he only punctuated in extra activities of codes

without at any moment having interacted with the

class. However, his results and learning was great

and raised a question for the authors of this paper

that will be taken into account in future case studies:

can gamification be used with any type of student

and situation? Or how to adapt the gamification so

that it is not punitive to some student who has some

difficulty or special need?

On students’ motivation in the context of the

case study, regarding the use of gamification as a

motivating tool was unanimity among the students

present in the feedback class, evaluating the use of

gamification as positive.

Gamiﬁcation and Evaluation of the Use the Agile Tests in Software Quality Subjects: The Application of Case Studies

421

5.1 Comparison between the Results

Obtained

The quantitative data that were collected for the

analysis of these Case Studies are allocated in the

spreadsheets of each respective case study, while the

qualitative data were collected by audio and

analysed by the authors of the paper.

5.1.1 Quantitative Results

Some of the feedbacks collected served as a way to

improve the gamification, aiming to improve the

students' achievement in the subject. This resulted in

a good increase in student participation in the second

case study, as can be seen in the graph of Figure 2,

which shows the sum of all the Daily Bonuses

obtained by the students in the case study, divided

by the number of students and number of classes.

Figure 2: Comparison the participations between the case

studies.

It is apparent that the stimulation of the

visualization the gamification results and its

competitive made that the students' grades were

levelled in the gamification in the second case study

by the student with greater participation in the

classroom, causing an exponential increase in the

participation of the students. However, as the highest

student levelled the grade in the classroom and this

student was very participative, this made the grades

of many students fall, as can be seen in the Figure 3.

As can be ascertained, using the levelling factor,

where there was a student with a total of 62

participations in the game, and taking into

consideration that the average participation of the

class was 31 participations, excluding the student 19

who only participated in the last day of the Case

Study, it can be concluded that most of the students

lost a grade compared to the average parameter that

was used as the levelling factor of the first case

study.

Analysing a comparison of concepts between the

classes, it was noticed that when we increased the

difficulty of gamification, imposing a levelling by

the greater participation, this portrayed directly in

the concepts of the students.

There was a greater balance in the concepts in

the first case study, even comparing that on average

the grade of the evaluative LAB in the first Case

Study had a mean achievement of 77.8% versus

78.1% of the second Case Study. In the issue of the

test grade, the first case study group was 7.2 versus

7.0 of the second case study, disregarding the

student who missed the activity. Therefore, the main

factor of difference of the concepts comes from the

choice of levelling of gamification.

Only one student achieved the concept of

excellent in the second Case Study, and considerably

increased the difficulty of the students who

participated in this model with levelling by the

participant with more points in the gamification.

5.1.2 Qualitative Results

The qualitative results were collected from

interviews in the feedback class with all the students

and the review of the quality of the codes obtained

during the Case Studies.

As in (Matsubara and Silva, 2017) work, there

were a lot of positive effects observed by students.

Figure 3: Comparison the grades between the case studies.

ENASE 2018 - 13th International Conference on Evaluation of Novel Approaches to Software Engineering

422

The most cited changes were: the support in the

study process; the greater stimulus for the study of

the subject; the better reception of students in doing

practical activities with techniques from Software

Engineering that have been adapted to become

gamification challenges. Other positive points were:

the greater interactivity of the class, which improved

communication among students; the greater

objectivity of the subject to come with a greater

focus on practical teachings, rather than theoretical

classes; the participation of motivated students in

motivating the other students; and the environment

being more open to questions, bringing more

dynamism to the subject. Another point highlighted

by the students was that the use of penalties for use

of mobile devices during the classes, causing the

students to pay more attention during the class, thus

participating and learning better the subject content.

Some of the difficulties highlighted by the

students were little practical knowledge of the Java

programming language, but they said that this fact

was not a great impediment and generated a lot of

learning in the language. Another negative point for

students that happened potentially in the first Case

Study was the poor understanding of how

gamification worked and how they should score.

6 CONCLUSIONS

This paper presented the results of a case study that

allowed improving the teaching of Agile Testing in.

The results obtained with the spreadsheet and

feedback from the participants make possible the

response of the QP that was presented, where the use

of gamification as a tool to support teaching

contributes positively to learning.

As future work, some improvement points are

possible in the game environment such as:

implementing a tool that supports learning using the

gamification concepts and serious games and can

return instant feedback to students; using a digital

coin that can be computed both in activities in the

tool and in activities outside it; creation of medals

for challenges and milestones to motivate student

participation; adapting the gamification to students’

cases who have some difficulty in communication

and learning, so as not to exclude them from the

game; performing an analysis between how to level

the gamification grade for an upcoming Case Study;

adding to the case study the use of pre and post

questionnaires, seeking to know the previous

knowledge of students in Agile Testing and TDD.

REFERENCES

Beck, K., 2010. TDD Test Driven Development. 1. ed.

Porto Alegre: Bookman Editora.

Chou, Y., 2015. Actionable Gamification - Beyond Points,

Badges, and Leaderboards. Octalysis Media.

Elgrably, I., Oliveira, S., 2017. The Importance of

Application of Agile Testing in the Software Industry:

An Exploratory Approach Using Interview. 14th

International Conference on Information Systems &

Technology Management.

Fardo, M. F., 2013. A Gamificação Aplicada em

Ambientes de Aprendizagem. Renote- Novas

Tecnologias na Educação. 11.

Freitas, S., Lima, T., Canedo, E., Costa, R. L., 2016.

Gamification and evaluation of student engagement in

a technical subject of undergraduate course. XXVII

Brazilian Symposium on Informatics in Education,

[s.l.], p.370-379.

Gartner, 2015. Gartner Highlights Five Key Steps to

Delivering an Agile I&O Culture. Available

in: http://www.gartner.com/newsroom/id/3032517.

Accessed in 10/2017.

Gregory, J., Crispin, L., 2014. Chapter 23-testing and

DevOps: In more agile testing: Learning journeys for

the whole team. Addison-Wesley Professional.

Hanus, M. D., Fox, J., 2015. Assessing the Effects of

Gamification in the Classroom. Computers and

Education, vol. 80, pp. 152–161.

Kapp, K. M., 2012. The Gamification of Learning and

Instruction: Game-based Methods and Strategies for

Training and Education. North Carolina: Pfeiffer,

366 p.

Luz, R. B., Neto, A., 2012. Using Programming Dojos for

Test-Driven Development Teaching. XXIII Brazilian

Symposium on Informatics in Education, p.25-35, 26.

Matsubara, P. G. F., Silva, C. L. C., 2017. Game Elements

in a Software Engineering Study Group: A Case

Study. 39th International Conference On Software

Engineering: Software Engineering Education and

Training Track (ICSE-SEET), p.160-169. IEEE.

Marick, B., 2003. Exploration Through Example: My

Agile testing project. 2003. Available in http://

www.exampler.com/old-blog/2003/08/21/#agile-testin

g-project-1. Accessed in 10/2017.

Nadalete, L. G., Kryszczun, J., 2013. Agile Test, How to

implement? Available in: http://eliasnogueira.com/o-

mundo-de-teste-de-software/capitulo-7-teste-agil-com

o-implementar/. Accessed in 10/2017.

Werbach, K., Hunter, F., 2012. For The Win: How Game

Thinking Can Revolutionize Your Business.

Filadélfia, Pensilvânia:Wharton Digital Press.

Gamiﬁcation and Evaluation of the Use the Agile Tests in Software Quality Subjects: The Application of Case Studies

423