A Comparative Study: Augmented and Virtual Reality Applications for

Improving Comprehension of Abstract Programming Concepts

Omer Emin Cinar

, Karen Rafferty

, David Cutting

and Hui Wang

School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, Belfast, U.K.

Keywords:

Augmented Reality, Booklet, Education, Engineering, Programming, Python, Virtual Reality.

Abstract:

This paper presents the development of a Virtual Reality (VR) application to enhance the learning experience

of Python collection data types for electrical and electronic engineering students, and the comparison of this

VR application with an Augmented Reality (AR) application developed for the same ﬁeld, and a paper-based

content. AR and VR, as emerging technologies, hold promise for transforming educational and cognitive

processes, especially in engineering education where students often struggle with abstract programming con-

cepts. The research focuses on using VR to make abstract concepts like Python collections more accessible

and understandable through immersive and interactive experiences. The VR application allows students to

engage with virtual representations of Python data types, promoting a deeper comprehension. A comparative

user study was conducted, involving participants using either the VR or AR application or learning through a

printed booklet. The effectiveness of these methods was assessed by pre- and post-comprehension tests. The

results indicated a signiﬁcant improvement in understanding Python collections with the VR application, as

evidenced by the substantial difference between its pre- and post-test score means. This ﬁnding suggests that

VR, more effectively than AR or traditional methods, aids in grasping abstract programming concepts.

1 INTRODUCTION

AR and VR are revolutionary technologies that have

the potential to reshape the landscape of education

and learning. In general terms, AR allows users to

perceive the actual world while overlaying or inte-

grating virtual elements onto it and this technology

encompasses three fundamental attributes: the fusion

of real and virtual environments, seamless real-time

interaction, and a three-dimensional aspect (Azuma,

1997). Besides, VR consists of interaction, immer-

sion, and imagination (Burdea and Coiffet, 2003) and

uses computer technology to simulate an interactive

three-dimensional environment, where objects have a

sense of spatial presence (Bryson, 2013).

In the realm of engineering education, these im-

mersive technologies offer unique opportunities to en-

hance the understanding of complex concepts. Build-

ing upon the AR application designed to improve the

comprehension of Python collection data types (Lists,

https://orcid.org/0000-0002-4422-4482

https://orcid.org/0000-0002-7443-7876

https://orcid.org/0000-0002-1088-4749

https://orcid.org/0000-0003-2633-6015

Tuples, Sets, and Dictionaries) for ﬁrst-year electrical

and electronic engineering students in (Cinar et al.,

2023), this paper introduces the development and im-

plementation of a VR application that takes the learn-

ing experience to new heights by adding interaction

using Oculus Integration SDK (Software Develop-

ment Kit).

The VR application represents a natural progres-

sion from the booklet and the AR application, allow-

ing users to delve into a fully immersive virtual en-

vironment. By integrating real-time interaction and

virtual elements, the VR application provides an en-

hanced learning experience with added layers of in-

teractivity.

1.1 Hypothesis

The paper posits three primary hypotheses as follows:

• Hypothesis 1 (H1). Both sample group 1 (users

of the AR application) and sample group 2 (users

of the VR application) will show a statistically

signiﬁcant improvement in comprehension from

the pre-test to the post-test. H1 is based on the

premise that both AR and VR technologies offer

enhanced learning experiences compared to tra-

538

Cinar, O., Rafferty, K., Cutting, D. and Wang, H.

A Comparative Study: Augmented and Virtual Reality Applications for Improving Comprehension of Abstract Programming Concepts.

DOI: 10.5220/0012565600003660

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2024) - Volume 1: GRAPP, HUCAPP

and IVAPP, pages 538-548

ISBN: 978-989-758-679-8; ISSN: 2184-4321

ditional methods. AR and VR can provide more

interactive and engaging ways to visualize and in-

teract with complex concepts, in this case, Python

collection data types. These technologies have

been shown to increase motivation and engage-

ment, which are key factors in learning.

• Hypothesis 2 (H2). The rate of comprehension

improvement from the pre-test to the post-test will

be greater in sample group 2 compared to sample

group 1. H2 is based on the premise that VR tech-

nology typically offers a more immersive learning

environment than AR. While AR overlays digital

information in the real world, VR creates a com-

pletely virtual environment that can more fully en-

gage the user’s senses. This total immersion in

VR is hypothesized to lead to a higher level of

concentration and focus on the subject matter, po-

tentially resulting in a greater comprehension im-

provement compared to AR.

• Hypothesis 3 (H3). The rate of comprehension

improvement from the pre-test to the post-test will

be greater in both sample group 1 and sample

group 2 compared to the control group (readers of

the booklet). H3 is based on the premise that the

control group, using traditional paper-based learn-

ing materials, is exposed to a more passive form

of learning. In contrast, AR and VR provide ac-

tive learning experiences, where users can engage

directly with the material. The interactive nature

of these technologies caters to various learning

styles (visual, kinesthetic, etc.), potentially lead-

ing to enhanced cognitive processing and reten-

tion of information.

1.2 Contribution

The addressed gaps include:

• How interaction is effectively used during the VR

application development process,

• How a VR-based educational application is devel-

oped for such a speciﬁc application area as Python

collection data types,

• How more robust and extensive content for this

kind of VR application can be designed in this di-

rection,

• What type of evaluation methods can be adopted

to analyse the outcomes of this kind of VR appli-

cation,

• How users can gain more effective learning expe-

riences with a VR application compared to paper-

based and AR content.

This VR application presents an innovative and

engaging method to improve understanding of com-

plex programming concepts, contributing to research

on VR’s educational use and demonstrating its effec-

tiveness for engineering students, with implications

for the design of future VR tools across various edu-

cational ﬁelds.

The research methodology involves a comparison

between the effectiveness of booklet, AR, and VR ap-

plications for learning by dividing participants into

three groups to use each method, and conducting pre-

and post-comprehension tests to evaluate their effec-

tiveness, with a focus on determining if the VR appli-

cation surpasses the AR application in user engage-

ment and learning outcomes.

The ﬁndings from this research hold signiﬁcant

implications for integrating immersive technologies

into engineering education. The VR application’s in-

teractive support is a key point here. Moreover, the

study contributes to the growing body of knowledge

on the potential of VR in education, highlighting the

beneﬁts of interactive and adaptive learning environ-

ments.

As VR technology continues to evolve, the out-

comes of this research will inform future advance-

ments in educational applications, paving the way

for more effective and inclusive learning experiences

across diverse domains. By blending the realms

of AR, VR, and interaction, this study represents a

forward-looking endeavor that strives to empower en-

gineering students on their educational journey.

In summary, the contributions of this work are

threefold, encompassing key advancements in the de-

velopment of the VR application for enhancing the

comprehension of abstract and/or complex program-

ming concepts, speciﬁcally focused on Python collec-

tion data types:

• Firstly, a cumulative design approach was adopted

when the VR application was developed based on

the AR application developed previously.

• Secondly, the incorporation of Oculus Integration

SDK for interaction within the VR environment

is a notable contribution, improving a deeper and

more permanent understanding of programming

concepts.

• Thirdly and ﬁnally, the research methodology em-

ployed, involving a comparative user study be-

tween the booklet, the previously developed AR

application, and the new VR application, aims

to provide insights into the effectiveness of VR-

based learning tools.

A Comparative Study: Augmented and Virtual Reality Applications for Improving Comprehension of Abstract Programming Concepts

539

2 BACKGROUND

2.1 Augmented and Virtual Reality in

Engineering Education

The integration of AR and VR technologies into engi-

neering education has become a subject of extensive

study and exploration, each offering unique beneﬁts

and applications.

Numerous studies have explored the use of AR

in engineering education. According to (Billinghurst

and Duenser, 2012), AR is gaining signiﬁcance in the

educational realm, particularly for its positive impact

on student memory retention and comprehension.

In (Luo and Mojica Cabico, 2018) an AR-based

learning tool was developed for construction engi-

neering students and it demonstrated improved learn-

ing experiences and enhanced understanding com-

pared to traditional 2D graphics. In (Kaur et al., 2020)

it was emphasised that AR is seen as a valuable tool

for enhancing student motivation by providing inter-

active learning experiences in engineering education

According to (Tuli and Mantri, 2020), while prac-

tical labs in science and engineering education face

limitations due to costs and resource constraints, AR

not only effectively overcomes these issues, particu-

larly in simplifying concepts like Fleming’s rule in

electromagnetism, but also surpasses web-based ap-

plications in improving student understanding and

ﬂow awareness in this ﬁeld.

Additionally, it was underlined in (Jesionkowska

et al., 2020) that AR technology is known to boost

motivation and improve perceptions of STEAM sub-

jects. It promotes active learning and curiosity and

empowers students to take charge of their educational

journey.

On the other hand, the use of VR in engineer-

ing education has been researched in many studies to

date. The studies analysed in (di Lanzo et al., 2020)

demonstrated that the utilisation of VR in engineering

education has the potential to provide advantageous

and promising outcomes concerning desired learning

achievements in tertiary and industry institutions.

According to (Radianti et al., 2020), there are cer-

tain gaps in terms of immersion, application areas of

VR, content, evaluation, methods, and theories in the

relevant literature.

In (Laseinde et al., 2015) the researchers em-

ployed VR as a technique to effectively communicate

engineering topics to ﬁrst-year engineering students.

The outcomes obtained through the implementation

of this teaching tool make a substantial contribution

to the ﬁeld of VR content development and enhance

students’ retention of engineering concepts. In (Di-

nis et al., 2017) their VR interface enables partici-

pants to immerse themselves in a three-dimensional

model of a building on the campus, facilitating the ex-

ploration of various disciplines within civil engineer-

ing. Users can reveal concealed components in the

building systems by switching the visibility of differ-

ent sets of building elements. In (Muller et al., 2017)

the utilisation of consumer-grade VR equipment for

immersive learning purposes within the ﬁeld of Me-

chanical Engineering was underlined. The design of

a VR immersive learning game was outlined and a

methodology was proposed for evaluating the user

experience, speciﬁcally examining three dimensions:

usability, utility, and acceptability.

In (Kami

nska et al., 2017), VR was highlighted

as a new trend in mechanical and electrical engineer-

ing education, by demonstrating its effectiveness in

immersive learning by reducing costs, eliminating the

need for expensive equipment, and enhancing mate-

rial retention and educational outcomes, with most

participants effectively interacting with the VR plat-

form and showing improved comprehension and re-

tention.

Brieﬂy, the dynamic synergy in AR and VR is

transforming the landscape of engineering education,

paving the way for more immersive, engaging, and ef-

fective learning experiences while addressing various

challenges, ultimately shaping the future of engineer-

ing instruction.

2.2 Augmented and Virtual Reality in

Computer Science and

Programming Concepts

The use of AR and VR for engineering education has

opened up new avenues for teaching Computer Sci-

ence (CS) and programming concepts, as evidenced

by a plethora of studies in the literature.

In the relevant literature, one can encounter stud-

ies that incorporate AR into engineering education to

teach CS concepts and topics.

In (Bacca Acosta et al., 2014) it was underlined

that AR techniques enhance the effectiveness of visu-

alisations and offer multiple beneﬁts for learning pro-

gramming, including improved learning outcomes,

increased motivation, interactivity, and collaboration.

In (Schez-Sobrino et al., 2021) a 2-D graphic no-

tation within a 3-D environment was introduced. This

innovative approach employed graphical representa-

tions inspired by roads and trafﬁc signs within the

realm of programming instruction, enabling the visu-

alisation of programs through AR. CS students evalu-

ated these visual representations, conﬁrming the clar-

ity and practicality of the suggested notation for con-

HUCAPP 2024 - 8th International Conference on Human Computer Interaction Theory and Applications

540

veying programming concepts.

Furthermore, (Theodoropoulos and Lepouras,

2021) highlighted the tremendous potential of AR as

a tool to support CS education. Immersive and engag-

ing AR tools are considered essential for effective CS

education.

In addition, it is inevitable to ﬁnd studies in the

literature in which VR is employed to teach CS con-

cepts as part of engineering education.

It was underlined in (Oktaviati and Jaharadak,

2018) that the integration of gamiﬁcation and virtual

environments in CS education enhances interactivity,

content absorption, interest, comprehension, and ef-

ﬁciency, leading to a more engaging and effective

learning experience.

According to (Batista et al., 2020), the utilisation

of virtual environments in CS education has yielded

positive outcomes in the learning process.

In conclusion, both AR and VR have emerged as

powerful tools for enhancing the teaching and learn-

ing of CS within engineering education, offering im-

mersive experiences, improved comprehension, and

heightened motivation, paving the way for more ef-

fective and engaging pedagogical approaches in this

dynamic ﬁeld.

2.3 The Use of Gamiﬁcation in

Augmented and Virtual Reality

According to numerous papers, the use of gamiﬁca-

tion in AR and VR applications yields signiﬁcant ad-

vantages in terms of comprehension.

According to (Lampropoulos et al., 2022), the in-

tegration of gamiﬁcation into AR holds promise for

fostering customised and collaborative learning expe-

riences.

On the other hand, in (Agbo et al., 2021) the au-

thors identiﬁed gaps in VR applications for Com-

puter Science (CS) education, emphasising the need

for contextualization of VR experiences to align with

CS educational goals and integrate real-world scenar-

ios, thereby enhancing the relevance, understanding,

and engagement of CS learners and preparing them

for practical applications.

In (Nicola et al., 2018) a VR application incor-

porating gamiﬁcation principles was developed for

learning the Bubble Sort algorithm, signiﬁcantly im-

proving student engagement and effectiveness in un-

derstanding the sorting technique. According to (Bro-

mana and Mårell-Olssonb, 2018), the primary objec-

tive of gamiﬁcation is to amplify students’ intrinsic

motivation by providing them with clues, challenges,

and opportunities for progression or "levelling up."

In summary, the strategic use of gamiﬁcation

within AR and VR applications emerges as a pivotal

approach to enhance comprehension and engagement,

offering students tailored, immersive, and contextu-

ally rich learning experiences in the ﬁeld of CS edu-

cation.

3 MATERIALS AND METHODS

The process is meticulously orchestrated for a thor-

ough comparative analysis between the booklet, AR,

and VR applications. In the study, details are provided

about the booklet and VR application, while details

about the AR application used for comparison can be

found in (Cinar et al., 2023).

3.1 Purpose and Development of the

Booklet

Initially, a comprehensive booklet was crafted, en-

compassing detailed examples of Python’s collection

data types: lists, tuples, and dictionaries. This booklet

meticulously presented the data types along with their

inputs and corresponding outputs in a lucidly written

format.

The inception of the booklet was driven by a

strategic purpose: to establish a control group for the

user study, distinct from those utilising the AR and

VR applications. This booklet served as a founda-

tional tool, offering a traditional, non-digital method

for comprehending abstract programming concepts.

Its role was pivotal in providing a baseline for com-

parison, ensuring that the study could accurately

gauge the effectiveness and added value of the AR

and VR applications in enhancing understanding of

Python’s collection data types. This comparative ap-

proach was instrumental in assessing the incremen-

tal beneﬁts of immersive technologies in educational

contexts, particularly in the realm of abstract pro-

gramming concepts.

Figures 1 and 2 show some list and tuple data type

examples in the booklet.

Figure 1: An Example of the List Data Type in the Booklet.

A Comparative Study: Augmented and Virtual Reality Applications for Improving Comprehension of Abstract Programming Concepts

541

Figure 2: An Example of the Tuple Data Type in the Book-

let.

3.2 Purpose and Development of the VR

Application

The ﬁnal phase of development witnessed the evolu-

tion of the AR content into a fully-ﬂedged Virtual Re-

ality (VR) application. Here, the essence of interac-

tion was signiﬁcantly ampliﬁed, allowing users to en-

gage with programming concepts in a highly interac-

tive and spatially contextualised virtual environment.

The purpose of the VR application is to facili-

tate users in enhancing their understanding of the sub-

ject by presenting basic commands and differences in

Python collection data types through 3D objects in a

virtual environment using interactive learning.

Python collection data types were categorised into

four subjects: Lists, Tuples, Dictionaries, and Sets be-

fore the VR application was developed. This categori-

sation was applied to the development process of the

application.

Unity 2021 was employed to develop this appli-

cation. It is a newer version of Unity 2020 and is a

game engine that provides the same functions to its

users with newly added features.

Since the VR application was developed for Ocu-

lus or Meta devices with its new name such as Oculus

or Meta Quest 2, Oculus Integration SDK was used as

a software development kit.

The VR application features several interactive el-

ements to boost user engagement and learning effec-

tiveness. These include:

• Hand Interaction for the natural manipulation of

objects, mirroring real-life hand movements, and

enhancing the learning of abstract concepts.

• Grab Interaction allows users to physically handle

virtual objects, vital for teaching programming

and data structures.

• Ray Interaction uses a virtual ’laser pointer’ for

distant object interaction, aiding in navigating

complex information.

• Poke Interaction, simulating ﬁnger poking, is

ideal for pressing buttons or handling small items.

Together, these features create an immersive en-

vironment, combining natural interaction, object ma-

nipulation, distant engagement, and simple, effective

interaction with smaller items, all contributing to an

enhanced learning experience in programming con-

cepts.

Accordingly, the necessary conﬁgurations were

completed. Every scene includes a panel including

a question about Python collection data types. In ad-

dition, buttons and pins were created to represent el-

ements of the indexes in the questions. Thus, users

can push the buttons or drag the pins with their hands

in the scenes to answer the questions. Using these

kinds of interactive elements, these concepts and no-

tions were thought to be able to be more understand-

able and tangible for users.

In the VR application, each index element is vi-

sualised as a button or pin, as shown in the slicing &

indexing example in Figure 3 with the list index [’a’,

’b’, ’c’, ’d’, ’e’, ’a’, ’f’]. Users are instructed to list

elements from the third to the ﬁfth, meaning pressing

the buttons representing ’d’ and ’e’.

Figure 3: A Slicing & Indexing Example in the List Data

Type.

When the ’Correct Answer’ toggle is pressed,

"Correct Answer" is displayed on the panel above,

and the correct answer [’d’, ’e’] is shown as in Fig-

ure 4, indicating the buttons ’d’, and ’e’ are pressed.

Figure 4: ’Correct Answer’ of the Slicing & Indexing Ex-

ample.

In the reverse sorting example shown in Figure 5,

the user is presented with a list index [123, 12321,

312, 45435, 35, 345, 1, 1] and asked to arrange the

HUCAPP 2024 - 8th International Conference on Human Computer Interaction Theory and Applications

542

elements, represented as pins, in reverse order.

Figure 5: A Reverse Sorting Example in the List Data Type.

When the ’Correct Answer’ toggle is pressed,

"Correct Answer" appears on the panel above and

the user views the correctly arranged index [45435,

12321, 345, 312, 123, 35, 1, 1], as depicted in Figure

6 with pins arranged in the respective order.

Figure 6: ’Correct Answer’ of the Reverse Sorting Exam-

ple.

Thanks to the before-mentioned concept design,

the users can interact with the index elements with

their own hands and get more familiar with how the

data type commands work and what kind of functions

they have.

C# programming language was used to write the

scripts of the VR application through Visual Studio.

The fact that the 3D objects created in the application

could be interactive was ensured in this way.

4 EXPERIMENT

A user study process was carefully organised and

executed to evaluate the performance and user-

friendliness of the VR application developed along

with the booklet and the AR application developed

previously. To conduct this user study, ethical ap-

proval was received from the Engineering and Phys-

ical Sciences Faculty Research Ethics Committee at

Queen’s University Belfast.

4.1 Objective

The main objective of the experiment was to assess

the difference in students’ comprehension of Python

collection data types. To achieve this, Sample Group

1 was tasked with answering a set of questions be-

fore and after using the AR application, while Sam-

ple Group 2 was given the same questions before and

after using the VR application. The control group,

on the other hand, answered the same questions be-

fore and after reading a printed booklet containing the

content covered in the applications.

4.2 Participants

39 people participated in the user study conducted.

All participants were electrical & electronic engineer-

ing students. They were divided into three groups

with an equal number of participants, namely 13 par-

ticipants in the control group, 13 participants in sam-

ple group 1, and 13 participants in sample group 2,

and were categorised as follows:

• Control Group. They read the booklet and did

not use the AR or VR applications.

• Sample Group 1. They used the AR application.

• Sample Group 2. They used the VR application.

4.3 Equipment

The booklet was used by the control group as paper-

based content during the user study.

The AR application was used by sample group 1

on Xiaomi 11 Lite 5G NE with the Android operating

system during the user study.

The VR application developed was used by sam-

ple group 2 on Meta Quest 2 as the VR headset dur-

ing the user study. The Meta Quest 2 HMD (Head-

Mounted Display) is a prominent device in the con-

sumer VR market. It boasts outstanding features such

as a display resolution of 1832 x 1920 per eye, a re-

fresh rate of 90Hz, and 6 degrees of freedom (DOF)

for a truly immersive experience.

4.4 Procedure

Participants who chose to take part were ﬁrst required

to provide their consent by approving a consent form.

Subsequently, they were divided into three approxi-

mately equal groups: Sample Group 1, Sample Group

2, and the Control Group. All participants provided

their consent before participating in the study.

The pre- and post-tests had 10 questions on

the primary aspects of data types. It included six

A Comparative Study: Augmented and Virtual Reality Applications for Improving Comprehension of Abstract Programming Concepts

543

multiple-choice questions and four true-false ques-

tions. Furthermore, the booklet contained example

commands for the AR and VR applications. The tasks

of the three groups are listed as follows:

• Control Group. They completed the pre-test,

then read the booklet, and ﬁnally completed the

post-test.

• Sample Group 1. They completed the pre-test,

then used the AR application, and ﬁnally com-

pleted the post-test.

• Sample Group 2. They completed the pre-test,

then used the VR application, and ﬁnally com-

pleted the post-test.

5 RESULTS AND DISCUSSION

The literature analysis highlights the scarcity of re-

search on the effects of educational VR and AR ap-

plications on users, particularly in comparison to tra-

ditional learning methods. It also highlights the grow-

ing signiﬁcance of VR- and AR-enhanced educational

tools.

Notably, the educational technology landscape

has also seen a surge in interest in VR applications

providing immersive and interactive learning experi-

ences that can enrich educational outcomes and en-

gagement. However, similar to AR, there remains

a scarcity of research examining the effects of VR-

based educational applications. This highlights the

importance of further exploration into the inﬂuence

of VR and AR technologies in education, given their

evolving roles in shaping the future of learning.

This study introduces the development of a VR

application aimed at enhancing student comprehen-

sion of Python collection data types and presents the

comparison of this VR application with the booklet

and the AR application developed previously. The

VR application offers various options for learning

and leverages the power of immersion and interac-

tion. Additionally, it has been created speciﬁcally for

the Meta Quest 2 HMD, while the booklet is used as

printed and the AR application is accessible on smart-

phones and tablets running the Android operating sys-

tem.

To contrast the pre- and post-test scores among the

control group, sample group 1, and sample group 2,

a paired sample t-test was employed as the statisti-

cal method. MATLAB was utilised to generate scat-

ter plots for visualising these comparisons. Notably,

when presenting the t-test results in the tables, most

values were rounded to two decimal places, but the

p-values were presented with greater precision, of-

ten extending to four decimal places after the decimal

point.

5.1 The Control Group (Readers of the

Booklet)

Figure 7 shows the pre- and post-test scores of each

participant in the control group in a scatter plot. In

Figures 7, 8, and 9, if a participant has only one of

the blue or red marks indicating the number of cor-

rect answers they gave, this indicates that the pre- and

post-test scores obtained by a participant are the same.

Figure 7: Scatter Plot of Pre- and Post-Test Scores of the

Control Group.

In Table 1, a t-test was conducted to examine the

achievement levels of the control group by comparing

pre- and post-test scores. The analysis demonstrated

a signiﬁcant difference between the pre- and post-test

scores, with a p-value of 0.0027, which falls below the

signiﬁcance threshold of 0.05. The analysis revealed

a signiﬁcant difference The average score for the pre-

test was recorded as 5.46, while the average score for

the post-test was recorded as 7.31.

Table 1: t-test results between Pre-Test and Post-Test for the

Control Group.

Mean Variance Standard P

Deviation

Pre-test 5.46 4.6 2.15 0.0027

Post-test 7.31 1.56 1.25

5.2 Sample Group 1 (Users of the AR

Application)

Figure 8 shows the pre- and post-test scores of each

participant in sample group 1 in a scatter plot.

In Table 2, a t-test was conducted to examine the

achievement levels of sample group 1 by comparing

pre- and post-test scores. The analysis demonstrated

a signiﬁcant difference between the pre- and post-test

scores, with a p-value of 0.0004, which falls below

HUCAPP 2024 - 8th International Conference on Human Computer Interaction Theory and Applications

544

Figure 8: Scatter Plot of Pre- and Post-Test Scores of Sam-

ple Group 1.

the signiﬁcance threshold of 0.05. The average score

for the pre-test was recorded as 6, while the average

score for the post-test was recorded as 8.23.

Table 2: t-Test results between Pre-Test and Post-Test for

Sample Group 1.

Mean Variance Standard P

Deviation

Pre-test 6 2.5 1.58 0.0004

Post-test 8.23 2.53 1.59

5.3 Sample Group 2 (Users of the VR

Application)

Figure 9 shows the pre- and post-test scores of each

participant in sample group 2 in a scatter plot.

Figure 9: Scatter Plot of Pre- and Post-Test Scores of Sam-

ple Group 2.

In Table 3, a t-test was conducted to examine the

achievement levels of sample group 2 by comparing

pre- and post-test scores. The analysis demonstrated

a signiﬁcant difference between the pre- and post-test

scores, with a p-value of 0.0001, which falls below the

signiﬁcance threshold of 0.05. The average score for

the pre-test was recorded as 5.85, while the average

score for the post-test was recorded as 8.92.

Table 3: t-Test results between Pre-Test and Post-Test for

Sample Group 2.

Mean Variance Standard P

Deviation

Pre-test 5.85 2.81 1.68 0.0001

Post-test 8.92 0.91 0.95

5.4 System Usability Scales

Two different system usability scales were prepared

to get feedback from the participants in the user study

and to make qualitative analyses about the booklet

and the AR and VR applications developed. The ﬁrst

one was presented to the control group (readers of

the booklet) because it was paper-based content. The

second one was presented to sample groups 1 and 2

(users of the AR and VR applications, respectively)

because the applications were used through certain

devices.

The statements for the control group and sam-

ple groups 1 and 2, along with their corresponding

numbers, are given in Tables 4 and 5, respectively.

Bar charts depicting the opinions of the control group

and sample groups 1 and 2 about the applications are

given in Figures 10 and 11, respectively.

Table 4: Statements in the System Usability Scale for the

Control Group.

Statements Number

I think I would like to use this booklet frequently. 1

I found the booklet unnecessarily complex. 2

I thought the booklet was easy to read. 3

I think that I would need the support of a technical person to be able to read this booklet. 4

I found the various functions in this booklet were well-integrated. 5

I thought there was too much inconsistency in this booklet. 6

I would imagine that most people would learn very quickly by reading this booklet. 7

I found the booklet very cumbersome to read. 8

I felt very conﬁdent reading the booklet. 9

I needed to learn a lot of things before I read this booklet. 10

Figure 10: System Usability Analysis for the Control

Group.

A Comparative Study: Augmented and Virtual Reality Applications for Improving Comprehension of Abstract Programming Concepts

545

Table 5: Statements in the System Usability Scale for Sam-

ple Groups 1 and 2.

Statements Number

I think I would like to use this application frequently. 1

I found the application unnecessarily complex. 2

I thought the application was easy to use. 3

I think that I would need the support of a technical person to be able to use this application. 4

I found the various functions in this application were well-integrated. 5

I thought there was too much inconsistency in this application. 6

I would imagine that most people would learn to use this application very quickly. 7

I found the application very cumbersome to use. 8

I felt very conﬁdent using the application. 9

I needed to learn a lot of things before I could get going with this tool. 10

Figure 11: System Usability Analysis for Sample Groups 1

and 2.

6 CONCLUSION

In recent times, it has been observed, in the context

of physics terminology, that AR and VR have con-

verted their potential energy into kinetic energy in

education, particularly in engineering education, and

the projected development of these technologies is ex-

pected to accelerate exponentially. Such technologies

provide users with an entirely new experience through

the elements of immersion and interaction.

Firstly, the tests and analyses conducted demon-

strated that both sample group 1 (users of the AR ap-

plication) and sample group 2 (users of the VR ap-

plication) showed a statistically signiﬁcant improve-

ment in comprehension from the pre-test to the post-

test (2.23 and 3.07, respectively, p < 0.05). Thus, hy-

pothesis 1 (H1) was conﬁrmed.

Secondly, it was concluded that the rate of com-

prehension improvement from the pre-test to the post-

test was greater in sample group 2 (3.07) compared to

sample group 1 (2.23). Thus, hypothesis 2 (H2) was

conﬁrmed.

Thirdly, it was suggested that the rate of compre-

hension improvement from the pre-test to the post-test

was greater in both sample group 1 (2.23) and sample

group 2 (3.07) compared to the control group (readers

of the booklet) (1.85). Thus, hypothesis 3 (H3) was

conﬁrmed.

These ﬁndings ﬁnally conclude that VR and AR

applications are more effective than conventional

methods in enhancing learning comprehension, with

VR showing the most substantial impact.

Following the analysis of system usability for the

control group, it was determined that 84.62% of this

group expressed a favorable inclination towards fre-

quent utilisation of the booklet. Moreover, 92.31% of

the group did not perceive the booklet as excessively

intricate. Furthermore, 76.92% of respondents in the

group regarded the booklet as user-friendly. An ad-

ditional noteworthy ﬁnding was that 84.62% of the

group did not deem it necessary to seek technical as-

sistance to operate the booklet effectively.

In the assessment of functional integration for the

control group, 84.62% of the group reported a high

degree of coherence in the various functions within

the booklet. Additionally, a unanimous consensus

was reached, with 100% of the participants in the

control group concurring that there was no signiﬁcant

inconsistency in the booklet. Furthermore, 69.23%

of the group expressed a belief that most individuals

would quickly acquire proﬁciency in using the book-

let.

In terms of usability challenges for the control

group, 100% of the group did not think that there were

signiﬁcant issues regarding the cumbersome nature of

the booklet. As for user conﬁdence, 84.62% of indi-

viduals in the control group reported a high level of

self-assurance in their ability to navigate and utilise

the booklet. Moreover, it was revealed that 84.62% of

the group did not perceive a substantial requirement

for extensive learning before initiating the use of the

booklet.

Following the analysis of system usability for

sample groups 1 and 2, it was determined that 76.92%

of sample groups 1 and 2 expressed a favorable incli-

nation towards frequent utilisation of the applications.

Moreover, 76.92% of sample group 1 and 84.62% of

sample group 2 did not perceive the applications as

excessively intricate. Furthermore, 84.62% of respon-

dents in sample group 1 and 69.23% of respondents

in sample group 2 regarded the applications as user-

friendly. An additional noteworthy ﬁnding was that

100% of individuals in sample groups 1 and 2 did not

deem it necessary to seek technical assistance to op-

erate the applications effectively.

In the assessment of functional integration for

sample groups 1 and 2, 84.62% of sample group 1

and 100% of sample group 2 reported a high degree

of coherence in the various functions within the ap-

plications. Additionally, a unanimous consensus was

reached, with 100% of participants in sample groups

1 and 2 concurring that there was no signiﬁcant incon-

sistency in the applications. Furthermore, the major-

HUCAPP 2024 - 8th International Conference on Human Computer Interaction Theory and Applications

546

ity of respondents, speciﬁcally 100% of sample group

1 and 92.31% of sample group 2, expressed a be-

lief that most individuals would quickly acquire pro-

ﬁciency in using the applications.

In terms of usability challenges for sample groups

1 and 2, 84.62% of sample groups 1 and 2 did not

think that there were signiﬁcant issues regarding the

cumbersome nature of the applications. As for user

conﬁdence, 92.31% of individuals in sample group 1

and 84.62% of individuals in sample group 2 reported

a high level of self-assurance in their ability to nav-

igate and utilise the applications. Moreover, it was

revealed that 84.62% of participants in sample group

1 and 100% of participants in sample group 2 did not

perceive a substantial requirement for extensive learn-

ing before initiating the use of the applications.

7 LIMITATIONS

The primary limitation of this study is the sample size.

Our user study involved a total of 39 participants, with

each teaching environment (booklet, AR, VR) being

tested by 13 participants. While this provided initial

insights into the comparative effectiveness of these

methods, the small sample size limits the generalis-

ability of our ﬁndings. A larger participant pool could

offer a more comprehensive understanding of the ef-

fectiveness of these educational tools across different

learning styles.

8 FUTURE WORK

In response to the limitations and to further our re-

search, we plan to expand the scale of the study. Fu-

ture work will involve increasing the number of par-

ticipants to enhance the reliability and validity of our

results. By involving a larger group of learners, we

aim to obtain more robust data that can offer deeper

insights into the effectiveness of VR and AR in edu-

cational settings.

Additionally, we plan to enhance the VR applica-

tion by integrating more interactive agents and per-

sonalised learning elements. This development aims

to create a more engaging and adaptive learning en-

vironment, catering to individual learning needs and

styles. We anticipate that these improvements will not

only make the VR experience more immersive and in-

teractive but also contribute to a more nuanced under-

standing of how personalised and interactive elements

in VR can enhance the learning of abstract program-

ming concepts.

Through these enhancements, we aim to address

the current limitations and signiﬁcantly contribute to

this evolving ﬁeld, particularly in the context of teach-

ing complex and abstract subjects like programming.

ACKNOWLEDGEMENTS

The authors thank those who participated in the user

study for their time and Dr Michael Cregan, Dr

Giuseppe Trombino, and Dr Reza Raﬁee for their help

in ﬁnding the participants. The ﬁrst author is funded

by Türkiye Ministry of National Education.

REFERENCES

Agbo, F. J., Sanusi, I. T., Oyelere, S. S., and Suhonen, J.

(2021). Application of Virtual Reality in Computer

Science Education: A Systemic Review Based on Bib-

liometric and Content Analysis Methods. Education

Sciences, 11(3):142.

Azuma, R. T. (1997). A Survey of Augmented Real-

ity. Presence: Teleoperators & Virtual Environments,

6(4):355–385.

Bacca Acosta, J. L., Baldiris Navarro, S. M., Fabregat Gesa,

R., and Graf, S. (2014). Augmented Reality Trends

in Education: A Systematic Review of Research and

Applications. Journal of Educational Technology and

Society, 2014, vol. 17, núm. 4, p. 133-149.

Batista, A. F., Thiry, M., Gonçalves, R. Q., and Fernandes,

A. (2020). Using Technologies as Virtual Environ-

ments for Computer Teaching: A Systematic Review.

Informatics in Education, 19(2):201.

Billinghurst, M. and Duenser, A. (2012). Augmented Real-

ity in the Classroom. Computer, 45(7):56–63.

Bromana, K. and Mårell-Olssonb, E. (2018). Digital Tools

in Chemistry Education-Virtual/Augmented Reality

& Gamiﬁcation. In Book of Abstracts, page 77.

Bryson, S. (2013). Virtual Reality: A Deﬁnition History-A

Personal Essay. arXiv preprint arXiv:1312.4322.

Burdea, G. C. and Coiffet, P. (2003). Virtual Reality Tech-

nology. John Wiley & Sons.

Cinar, O. E., Rafferty, K., Cutting, D., and Wang, H. (2023).

Using Augmented Reality to Enhance Learning and

Understanding of Abstract Programming Concepts. In

2023 9th International Conference on Virtual Reality

(ICVR), pages 498–504. IEEE.

di Lanzo, J. A., Valentine, A., Sohel, F., Yapp, A. Y., Mu-

paradzi, K. C., and Abdelmalek, M. (2020). A Review

of the Uses of Virtual Reality in Engineering Educa-

tion. Computer Applications in Engineering Educa-

tion, 28(3):748–763.

Dinis, F. M., Guimarães, A. S., Carvalho, B. R., and Mar-

tins, J. P. P. (2017). An Immersive Virtual Reality In-

terface for Civil Engineering Dissemination amongst

Pre-University Students. In 2017 4th Experiment@

A Comparative Study: Augmented and Virtual Reality Applications for Improving Comprehension of Abstract Programming Concepts

547

International Conference (exp. at’17), pages 157–158.

IEEE.

Jesionkowska, J., Wild, F., and Deval, Y. (2020). Ac-

tive Learning Augmented Reality for STEAM Educa-

tion—A Case Study. Education Sciences, 10(8):198.

Kami

nska, D., Sapi

nski, T., Aitken, N., Rocca, A. D.,

Bara

nska, M., and Wietsma, R. (2017). Virtual Re-

ality as a New Trend in Mechanical and Electrical En-

gineering Education. Open Physics, 15(1):936–941.

Kaur, D. P., Mantri, A., and Horan, B. (2020). Enhanc-

ing Student Motivation with Use of Augmented Real-

ity for Interactive Learning in Engineering Education.

Procedia Computer Science, 172:881–885.

Lampropoulos, G., Keramopoulos, E., Diamantaras, K., and

Evangelidis, G. (2022). Augmented Reality and Gam-

iﬁcation in Education: A Systematic Literature Re-

view of Research, Applications, and Empirical stud-

ies. Applied Sciences, 12(13).

Laseinde, O. T., Adejuyigbe, S. B., Mpofu, K., and Camp-

bell, H. M. (2015). Educating Tomorrows Engineers:

Reinforcing Engineering Concepts through Virtual

Reality (VR) Teaching Aid. In 2015 IEEE Inter-

national Conference on Industrial Engineering and

Engineering Management (IEEM), pages 1485–1489.

IEEE.

Luo, X. and Mojica Cabico, C. D. (2018). Development and

Evaluation of an Augmented Reality Learning Tool

for Construction Engineering Education. In Construc-

tion Research Congress 2018, pages 149–159.

Muller, N., Panzoli, D., Galaup, M., Lagarrigue, P., and

Jessel, J.-P. (2017). Learning Mechanical Engineer-

ing in a Virtual Workshop: A Preliminary Study on

Utilisability, Utility and Acceptability. In 2017 9th In-

ternational Conference on Virtual Worlds and Games

for Serious Applications (VS-Games), pages 55–62.

IEEE.

Nicola, S., Stoicu-Tivadar, L., and Patrascoiu, A. (2018).

VR for Education in Information and Tehnology: Ap-

plication for Bubble Sort. In 2018 International

Symposium on Electronics and Telecommunications

(ISETC), pages 1–4. IEEE.

Oktaviati, R. and Jaharadak, A. A. (2018). The Impact of

Using Gamiﬁcation in Learning Computer Science for

Students in University. International Journal of Engi-

neering & Technology, 7(4.11):121–125.

Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlge-

nannt, I. (2020). A Systematic Review of Immer-

sive Virtual Reality Applications for Higher Educa-

tion: Design Elements, Lessons Learned, and Re-

search Agenda. Computers & Education, 147:103778.

Schez-Sobrino, S., García, M. Á., Lacave, C., Molina, A. I.,

Glez-Morcillo, C., Vallejo, D., and Redondo, M. A.

(2021). A Modern Approach to Supporting Program

Visualization: From a 2D Notation to 3D Represen-

tations Using Augmented Reality. Multimedia Tools

and Applications, 80:543–574.

Theodoropoulos, A. and Lepouras, G. (2021). Augmented

Reality and Programming Education: A Systematic

Review. International Journal of Child-Computer In-

teraction, 30:100335.

Tuli, N. and Mantri, A. (2020). Experience ﬂeming’s Rule

in Rlectromagnetism Using Augmented Reality: An-

alyzing Impact on Students Learning. Procedia Com-

puter Science, 172:660–668.

HUCAPP 2024 - 8th International Conference on Human Computer Interaction Theory and Applications

548