Comparing the Performance of an Immersive Virtual Reality and

Traditional Desktop Cultural Game

Brian An, Forrest Matteo, Matt Epstein and Donald E. Brown

Department of Systems and Information Engineering, University of Virginia, Charlottesville, VA, U.S.A.

Keywords:

Virtual Reality, Serious Games, Cross Cultural Competence.

Abstract:

The recent popularization and affordability of Head Mounted Display (HMD) Virtual Reality (VR) systems

such as the Oculus Rift has accelerated the expansion of the application of these devices beyond entertainment.

One of the targeted areas of expansion is in social interaction serious games where it has been often hypot-

hesized that the immersion of HMD VR would increase the learning effectiveness of these systems. Despite

this growth, few studies in the literature examine the effectiveness of these types of games in HMD VR as

compared to more traditional desktop systems. This study evaluates the performance difference between a

traditional desktop version and an HMD VR version of a cultural serious game designed to teach U.S. Army

soldiers how to communicate competently with Chinese soldiers in a joint humanitarian mission. The study

found no performance difference between participants who played the desktop or the HMD VR version of the

game. The study did ﬁnd a strong positive interaction correlation between gender and participants who played

the HMD VR version of the game. These ﬁndings motivate further research into why this correlation exists

and if, through game design, can also be instilled in female participants.

1 INTRODUCTION

As the world becomes more interconnected through

global business and coalition military operations, it

has become a necessity for individual’s to exhibit

Cross Cultural Competence (3C). 3C is characteri-

zed by the Department of Defense as the “set of kno-

wledge, skills, and affect/motivation that enable in-

dividuals to adapt effectively in cross-cultural envi-

ronments(Gallus et al., 2014).” This study evalua-

tes the effectiveness of a game designed to improve

3C delivered in a Head-Mounted Display (HMD) Vir-

tual Reality (VR) to the more traditional desktop ver-

sion of the game. While the evolution of HMD VR

technology has garnered much attention with the in-

troduction of devices such as the Oculus Rift and the

HTC Vive, few studies have empirically assessed the

added effectiveness of HMD VR systems over traditi-

onal desktop channels in improving social-interaction

skills such as 3C. This study speciﬁcally investigates

the effectiveness of a 3C training game presented in

both HMD VR and a traditional desktop medium in

order to inform future investments of 3C training sys-

tems.

2 PREVIOUS WORK

Since the introduction of game-based learning, 3C ga-

mes have taken many forms(Fowler and Pusch, 2010).

One of the ﬁrst games, BAFA BAFA

, was desig-

ned as a moderated board game where participants

would take on the role of ﬁctional cultures and at-

tempt to communicate and collaborate with other par-

ticipants(Fowler and Pusch, 2010).

As gaming technologies improved and theories

behind 3C evolved, more sophisticated computer-

based games emerged(Lane et al., 2013)(Tasdemir

and Prasolova-Førland, 2014)(Fishwick et al., 2008).

These games targeted various aspects of 3C to include

communication, negotiation, culture-speciﬁc know-

ledge, and behavioral norms.

Though these games have demonstrated varying

degrees of effectiveness in improving 3C, one area

that has yet to be investigated is the use of highly im-

mersive modern VR systems to train 3C. Given the

cognitive and sensory attributes associated with 3C, it

would seem highly plausible that VR could be a better

learning environment for 3C than the more traditional

digital means previously described(Van Dyne et al.,

2012)(Matsumoto and Hwang, 2013).

An, B., Matteo, F., Epstein, M. and Brown, D.

Comparing the Performance of an Immersive Virtual Reality and Traditional Desktop Cultural Game.

DOI: 10.5220/0006922800540061

In Proceedings of the 2nd International Conference on Computer-Human Interaction Research and Applications (CHIRA 2018), pages 54-61

ISBN: 978-989-758-328-5

2.1 Virtual Reality

At its most abstract deﬁnition, Virtual Reality has

been deﬁned as ”an artiﬁcial environment which

is experienced through sensory stimuli (as sights

and sounds) provided by a computer and in which

one’s actions partially determine what happens in the

environment(Merriam-Webster Online, 2017).” Gi-

ven this deﬁnition, the term Virtual Reality is in-

terchangeably used for a wide variety of technolo-

gies and experiences. Previously, Virtual Reality des-

cribed lower quality and lower speciﬁcation systems

using computer or projection screens to display con-

tent(Freina and Ott, 2015). Compared to today’s stan-

dards of interaction and immersion, these systems

would hardly be considered virtual reality. Modern

virtual reality systems, however, take many forms.

Of these systems, two technologies have emerged as

industry leaders: Cave Automatic Virtual Environ-

ments (CAVE) and Head-Mounted Display Virtual

Reality(HMD VR) systems.

Though many conﬁgurations of CAVE systems

exist, an example of a CAVE system consists of four

projection screens to serve the content using stereo

projection. Within the the environment, users wear

a set of tracked equipment ranging from simple 3D

glasses to full body suits to capture all body mo-

tion. Generally, tracking is conducted either with in-

ertial sensors or more commonly with infrared mar-

kers(Kageyama and Tomiyama, 2016)(Katsouri et al.,

2015).

HMD VR systems utilize a notably different met-

hodology to achieve immersion. HMD VR systems

consist of a goggle-like headset which projects an in-

dependently rendered image for each eye which cor-

rects for interpupillary distance. The HMD position

is tracked (typically with infrared markers) in order to

accurately render and project the appropriate image of

the head-based gaze direction. Rapid refresh rates of

the projection allow for users to naturally move their

head and continuously see a real-time 360 degree ste-

reo projection of the scene (See Fig 3). Due to its wi-

despread commercial adoption and optimized content

development tools, HMD VR systems have been the

target of many commercial entertainment and educa-

tional research ventures(Bastiaens et al., 2014)(Free-

man et al., 2017).

2.2 Immersive VR DGBL Systems

Recent innovations and cost-efﬁciencies with immer-

sive HMD VR have spurred a lot of interest with its

use in digital game based learning (DGBL) systems

speciﬁcally where sensory immersion is hypothesized

to augment learning beneﬁts. Freina et al. propose

that VR can add learning value in situations that “can-

not be physically accessed(Freina and Ott, 2015).”

They postulate that this limitation can be due to a

number of reasons to include temporal limitations (i.e.

situation occurred in a historic time period), safety li-

mitations (i.e. hostile or emergency situations), and

ethical limitations (i.e. performing a high-risk sur-

gery by non-experts). Though studies have shown that

immersive technologies such as HMD VR can incre-

ase Quality of Experience and Engagement, there is

a lack of empirical evidence comprehensively charac-

terizing the learning advantages and performances of

these technologies(Hupont et al., 2015).

Freeman et al. conducted a comprehensive litera-

ture review of empirical studies of the use of virtual

reality in mental health treatments and found large in-

consistencies in the use of the term ’virtual reality’

with many of the recent studies not actually referring

to modern VR as previously described(Freeman et al.,

2017). In the experiments that, in fact, did experiment

with modern VR, results were mixed. In one instance,

a research group found no difference in performance

between an HMD VR and a desktop version of DGBL

system designed to teach Spatial Perspective Taking

to mildly intellectually impaired teenagers(Freina and

Ott, 2015)(Freina et al., 2016).

Also, several studies have assessed the impact of

immersive VR on teaching various academic con-

cepts. A research group found that when participants

performed a musical signal ﬂow task in HMD VR

versus a traditional computer screen, no performance

difference was found(Tackett, 2016). Makransky et

al. conducted an experiment comparing immersive

VR an a traditional screen to teach science concepts.

Using electroencephalogram (EEG) measurements,

they found that though immersive VR increased sense

of presence, it also resulted in poorer learning outco-

mes(Makransky et al., 2017). On the contrary, other

studies have shown positive learning results in immer-

sive VR(Alhalabi, 2016)(Webster, 2016). In the area

of academic learning systems, the current research is

inconclusive as to whether immersive VR results in

better learning.

In the area of social and cultural serious games,

we found a limited number of studies evaluating per-

formance in immersive VR. One related area of rese-

arch addresses the effects of immersive VR in treating

cognitive and social disabilities. Several studies have

shown promising results in the use of immersive VR

to treat High-Functioning Autistic children though the

results are far from conclusive and further research is

recommended before any treatment recommendations

are considered(Bashiri et al., 2017)(Bradley and New-

Comparing the Performance of an Immersive Virtual Reality and Traditional Desktop Cultural Game

butt, 2018). Therefore, the limited number of studies

make it necessary to further investigate the learning

beneﬁts of social and cultural serious games.

2.3 Immersion and Gender

Previous studies comparing various levels of immer-

sive game environments have shown differences in

learning performance between males and females.

Though no studies were found evaluating modern im-

mersive VR systems, several studies compared tradi-

tional or lower ﬁdelity VR systems to lower immer-

sion systems and found that males generally perfor-

med better.

Given previous ﬁndings that females use virtual

spaces for communal purposes, Coffey et al. hypot-

hesized that females would score higher in a 3D vir-

tual environment (non HMD-VR) versus a static web

environment(Guadagno et al., 2011)(Coffey et al.,

2017). However, from their experiment, they found

the opposite to be true and that, in fact, males showed

more improvement in the 3D virtual environment than

females.

In an experiment measuring recall after being

delivered a lecture in VR, Bailenson et al. found

that males performed signiﬁcantly better than fema-

les(Bailenson et al., 2008). They hypothesized that

these gains could be attributable to ﬁndings repor-

ting that men were more experienced using video-

games(Yee, 2006).

Ausburn et al. speciﬁcally investigated the gen-

der effect in virtual learning environments and pro-

posed a theoretical framework to explain the diffe-

rence(Ausburn et al., 2009). They hypothesized that

these differences can be attributed to gender differen-

ces in “Technology Self-Efﬁcacy” characterized by a

set of experiences, skills, and perceptions that various

studies have also shown gender differences (See Fig

1)(Ausburn et al., 2009).

Though our literature review did not ﬁnd any spe-

ciﬁc studies that found similar gender differences in

modern immersive VR systems, the previously men-

tioned studies would lead one to believe that males

would also achieve better results in these systems.

3 RESEARCH QUESTIONS

The primary research question of this study is to in-

vestigate whether a 3C game played in immersive

HMD VR has a signiﬁcant effect on the acquisition

of 3C as compared to a game played on a traditio-

nal desktop medium. Given previous works related to

HMD VR, we hypothesize the following:

Figure 1: Ausburn‘s theoretical framework explaining gen-

der differences in virtual learning environments(Ausburn

et al., 2009).

• H1: Participants exposed to the HMD VR version

of the CMTS will exhibit greater improvement

than those that are exposed to the desktop version.

• H2: Males exposed to the HMD VR version of

the CMTS will exhibit greater improvement than

females.

4 EXPERIMENTAL DESIGN

Our experimental design was a 2 (Game Version)

x 2 (Gender) mixed factorial design. Using a

randomized-block design, participants were rand-

omly assigned to either the traditional or VR group

blocking for reported Cultural Intelligence (CQ)

score. Cultural Intelligence is a self-report inven-

tory designed to evaluate a participants 3C develo-

ped by Van Dyne et al(Van Dyne et al., 2012). Re-

cent independent studies have shown the CQ model

to be a promising measure of 3C and related persona-

lity traits(Matsumoto and Hwang, 2013)(Alon et al.,

2018)(Chua and Ng, 2017). In order to ensure an even

distribution of pre-experiment 3C, participants were

blocked around a predetermined CQ score threshold.

4.1 Stimuli

The stimuli used in this study was a custom develo-

ped serious game designed to teach participants Cross

Cultural Competence. The game was designed using

the Cultural Simulation Design Process(An et al.,

2017). The game is comprised of two military trai-

ning scenarios whereby the participant plays the role

of a US Army Ofﬁcer in charge of humanitarian ef-

forts. In each scenario, the player is expected to work

with a Chinese People’s Liberation Army Ofﬁcer in

CHIRA 2018 - 2nd International Conference on Computer-Human Interaction Research and Applications

order to conduct joint humanitarian efforts. Players

are presented with dialogue options of varying de-

grees of appropriateness. When a dialogue option is

selected, a feedback system provides speciﬁc guide-

lines as to why a particular selection was or was not

appropriate. In the ﬁrst scenario called the Disaster

Management Exercise (DME) scenario, US and Chi-

nese forces are working together to provide disaster

relief to an Ebola-stricken community in the Liberian

Lowlands (see Fig 2). In the second scenario called

the Earthquake scenario, US and Chinese forces are

providing aid and assistance to injured civilians after

a deadly earthquake in Nsunga, Tanzania. Both ver-

sions of the game had identical visual and dialogue

content.

Figure 2: The player is interacting with the PLA Ofﬁcer-in-

Charge in the DME Scenario of the HMD VR game.

4.1.1 Desktop Variant

Though both versions of the game have identical vi-

sual and dialogue content, each game version has no-

table differences in User Interface. The Desktop ver-

sion has a locked camera preventing the user from

changing what was visible on the screen. The dia-

logue system is locked to the bottom of the viewing

area and the primary mode of user interaction is by

clicking on the preferred dialogue option.

4.1.2 HMD VR Variant

The HMD VR version of the game has a notably dif-

ferent user experience and User Interface. This ver-

sion was developed speciﬁcally for the Oculus Rift

CV1 HMD VR system (see Fig 3). First, the camera

motion is tied to the motion-tracking system of the

Oculus Rift. This allows the participant to see the en-

tire scene around the player. Second, the dialogue sy-

stem is presented as a ﬂoating dialogue box as scene

in Fig 2. Dialogue selections are made through pro-

longed gaze and simultaneous controller selection on

the preferred option.

Figure 3: A participant playing the HMD VR version of the

game.

4.2 Participants

21 participants (Male=12, Female=9) were recruited

for the experiment. The distribution of participants

was 11 Reserve Ofﬁcer Training Corp (ROTC) cadets

and 10 non-military afﬁliated participants. The parti-

cipants mean age was 20.58 years (Median = 21, SD

= 1.06). Participants were equally split into the HMD

VR group (n=11) and the Desktop group (n=10). For

completing the experiment, each participant was com-

pensated with a $20 gift card.

4.3 Procedures

Students were individually assessed. Participants

were ﬁrst e-mailed an online survey intended to cap-

ture basic demographic information as well as their

CQ score. This information was used to ensure equal

representation in the test and control group while

blocking for the relevant factors. When participants

arrived for the experiment, they were ﬁrst administe-

red the pre-test. Following the pre-test, participants

engaged in a tutorial speciﬁc to the game version

they were playing. Following the tutorial, partici-

pants played both the DME and Earthquake scenario

of the game. Once the game was complete, the par-

ticipants were administered the post-test as well as a

feedback survey to gauge user experience. The entire

testing process for a single participant was approxi-

mately one hour. See Fig 4.

4.4 Measures

The pre-test and post-test used to determine perfor-

mance improvement were conducted as real-time dia-

Comparing the Performance of an Immersive Virtual Reality and Traditional Desktop Cultural Game

Figure 4: Experiment Flow.

logue situational judgment test (SJT). SJTs have of-

ten been use to evaluate 3C though previous versi-

ons have typically been written exams with multi-

ple choice responses(Lane et al., 2013)(Wray et al.,

2009). Though traditional SJTs have been effective in

measuring cognitive and metacognitive thought pro-

cesses, they do not capture any performance with re-

spect to culturally appropriate communication and be-

haviors. As such, we adapted the SJT to be a role-

playing dialogue with a Chinese actor hereby referred

to as the role-playing SJT (rSJT). The rSJT is initia-

ted by presenting the participant with a scenario dos-

sier which describes the role the player must play as

well as the objectives that player is trying to achieve

through the dialogue. Once the participant has read

through the dossier, the actor initiates the conversa-

tion with a scripted initial response at which point the

conversation becomes unscripted and free-ﬂowing.

The rSJTs are audio visually recorded and these

recordings are then rated by independent Chinese cul-

tural experts for cultural proﬁciency. The experts

score the rSJTs with a previously developed rubric

which then produces a numeric score for each rSJT. In

order to ensure the reliability of the ratings, a Cohen’s

Kappa is calculated on the ratings(Fleiss and Cohen,

1973).

Table 1: Cultural Intelligence Sub-Dimension Model.

Main Dimension Sub-Dimension

Cognitive Cultural General Knowledge

Context Speciﬁc Knowledge

Motivation Intrinsic Motivation

Extrinsic Motivation

Self Efﬁcacy to Adjust

Metacognitive Planning

Awareness

Checking

Behavior Verbal Behavior

Non-Verbal Behavior

Speech Acts

4.5 Data

4.5.1 Collection

Between the demographic survey and the CQ inven-

tory, a variety of data was collected on each partici-

pant. The demographic survey consisted of two ca-

tegories of items characterized by 1) Personal Infor-

mation and 2) International Exposure for a total of 12

unique survey items. The previously described CQ in-

ventory is composed of 37 items subsetted into 4 main

factors and 11 sub-factors (see Table 1 (Van Dyne

et al., 2012). The score of each main factor or sub-

factor is calculated as the average of each of the items

within the main factor or the sub-factor.

4.5.2 Analysis Methods

Multiple Linear Regression was the primary method

used to identify any performance increases with re-

spect to the pre/post test and the independent fac-

tors collected in the experiment. Initial factor sets

were based upon factors found to be relevant in pre-

vious studies as well as those that theoretically sho-

wed the most promise. In order to identify the most

signiﬁcant models, AIC-based Stepwise regression

was utilized to determine the most signiﬁcant mo-

dels(Akaike, 1987).

5 RESULTS

The results are organized by the hypotheses previ-

ously described.

CHIRA 2018 - 2nd International Conference on Computer-Human Interaction Research and Applications

5.0.1 H1 Analysis Results

Participants exposed to the HMD VR version of the

CMTS will exhibit greater improvement than those

that are exposed to the desktop version.

To examine the hypothesis H1, a Welch’s t-test

was performed between the HMD VR group and the

desktop group with respect to the performance impro-

vement. This t-test realized an insigniﬁcant difference

(t=0.47931, df =15,521, p=0.6384).

Additionally a multiple linear regression model

was generated with the independent behaviors con-

sisting of the Metacognitive and Behavior subdimen-

sions of CQ as well as various demographic factors

as shown in Table 2. The Metacognitive CQ com-

ponent “reﬂects the mental capability to acquire and

evaluate cultural knowledge.(Van Dyne et al., 2012)”

The Behavior CQ component “reﬂects the capabi-

lity to ﬂex behaviors to ﬁt different cultural con-

texts.(Van Dyne et al., 2012)” The Metacognitive and

Behavior CQ components were considered in the mo-

del for their speciﬁc relevance to skills that we hypot-

hesized would impact the pre-test and post-test sco-

res.

This model showed a negative signiﬁcant correla-

tion for those participants in the HMD VR group. An

AIC stepwise regression of that model also resulted

in a similar negative signiﬁcant correlation as seen in

Table 3.

5.0.2 H2 Analysis Results

Males exposed to the HMD VR version of the CMTS

will exhibit greater improvement than females.

With respect to H2, we explored the signiﬁcance

of an interaction effect between gender and VR. The

linear regression model shown in Table 2 showed

a signiﬁcant interaction effect as seen in the model

as “VR:genderMale.” Additionally, the AIC step-

wise regression resulted in a similar effect though the

“VR:genderMale” interaction effect was removed as

seen in Table 3.

6 DISCUSSION

The ﬁndings are discussed by the previously stated

hypotheses.

The ﬁrst hypothesis investigates whether the

HMD VR variant of a game results in more learning

performance as compared to the traditional desktop

variant. We found that, generally, the two variants

Table 2: Initial Regression Model of Performance Impro-

vement.

Dependent variable:

Improvement

VR −0.356

∗

(0.187)

meta Planning −0.117

∗

(0.064)

meta Awareness 0.035 (0.103)

meta Checking 0.364

∗∗∗

(0.105)

beh Verbal 0.062 (0.084)

beh Non Verbal 0.097 (0.054)

beh Speech −0.202

∗

(0.095)

genderMale −0.073 (0.184)

ROTC 0.169 (0.140)

VR:genderMale 0.484

∗∗

(0.180)

VR:ROTC 0.169 (0.184)

Constant −1.531

∗∗

(0.574)

Observations 21

Adjusted R

0.595

RSE 0.173 (df = 9)

F Statistic 3.672

∗∗

(df = 11; 9)

Note:

∗

p<0.1;

∗∗

p<0.05;

∗∗∗

p<0.01

Table 3: AIC Stepwise Regression Model of Performance

Improvement.

Dependent variable:

Improvement

VR −0.277

∗

(0.148)

meta Planning −0.082 (0.047)

meta Checking 0.356

∗∗∗

(0.075)

beh Non verbal 0.107

∗

(0.050)

beh Speech −0.170

∗∗

(0.062)

GenderMale −0.091 (0.164)

ROTC 0.292

∗∗∗

(0.084)

VR:genderMale 0.466

∗∗

(0.166)

Constant −1.407

∗∗

(0.503)

Observations 21

Adjusted R

0.647

RSE 0.161 (df = 12)

F Statistic 5.575

∗∗∗

(df = 8; 12)

Note:

∗

p<0.1;

∗∗

p<0.05;

∗∗∗

p<0.01

did not perform differently when other factors were

not considered. Additionally, when the test group was

considered along with the demographic factors, it was

found that generally the HMD VR group performed

slightly worse that the control group as shown in Ta-

bles 2 and Table 3. This ﬁnding parallels the results of

studies in other domains that found no difference or

a negative impact for HMD VR(Tackett, 2016)(Ma-

kransky et al., 2017). It is possible that the perceptual

Comparing the Performance of an Immersive Virtual Reality and Traditional Desktop Cultural Game

realism added through HMD VR is in fact distracting.

Through EEG measurements, Makransky et al. found

that HMD VR increased cognitive load as compared

to desktop versions of the same biology lab learning

system. They explained that this increased load may

overload participants and thus ”result in less oppor-

tunity to build learning outcomes.(Makransky et al.,

2017)”

In analyzing the results for the second hypothe-

sis, we discovered a more nuanced explanation to the

conclusion drawn from the ﬁrst hypothesis. The se-

cond order effect between participants who were male

and were in the HMD VR test group was found to be

highly signiﬁcant. The positive coefﬁcient of this in-

teraction would lead one to conclude that HMD VR

does, in fact, have a positive effect on learning out-

comes for males. This extends the ﬁndings of previ-

ous studies that have also found correlation between

higher immersion digital learning systems and gen-

der(Coffey et al., 2017)(Ausburn et al., 2009). Spe-

ciﬁcally, we conclude that this correlation between

immersion and gender can be extended through the

immersive experience of HMD VR. Causality of this

ﬁnding continues to be an open question that requires

further investigation though the theoretical construct

of technological self efﬁcacy is supported by our ﬁn-

ding has well as other independent ﬁndings(Ausburn

et al., 2009).

7 LIMITATIONS

A notable limitation in this study is the relatively

small sample size used in this experiment. Due to

the speciﬁc military-based application of the stimuli,

the pool of participants with ROTC backgrounds in

the local area was small. Future studies should be

conducted in areas with a higher concentration of mi-

litary participants in order to facilitate the recruitment

of military personnel.

8 CONCLUSION

This study has further advanced our understanding of

the impacts of using highly immersive virtual envi-

ronments in social and cultural DGBL systems as well

as the speciﬁc factors that explain these impacts. We

found that generally more immersive systems (e.g.

HMD VR) do not necessarily result in better learning

outcomes in a cultural DGBL system. However, the

strong interaction found between HMD VR and male

participants also indicates that demographic factors

such as gender should be considered when develo-

ping cultural DGBL systems and, perhaps, all types

of DGBL systems. This study motivates further rese-

arch with highly immersive DGBL systems in other

domains in order to draw more concrete conclusions

about general immersive systems beyond those drawn

in this study.

ACKNOWLEDGEMENTS

We would like to acknowledge our Chinese actors and

raters Wen Ding, Xiang Gao, Yichen Jiang, Chang

Xu, Yingjie Liu, and Wanyi Duan for administe-

ring the pre/post test and rating the pre/post test re-

cordings. Additionally, we would like to thank our

partners at the United States Military Academy for

their support in developing the cultural content in the

CMTS.

We would also like to thank the creators of the

Stargazer R package for contributing to the R com-

munity of researchers(Hlavac, 2015).

REFERENCES

Akaike, H. (1987). Factor analysis and aic. Psychometrika,

52(3):317–332.

Alhalabi, W. S. (2016). Virtual reality systems enhance stu-

dents achievements in engineering education. Behavi-

our & Information Technology, 35(11):919–925.

Alon, I., Boulanger, M., Elston, J. A., Galanaki, E.,

Mart

ınez de Ibarreta, C., Meyers, J., Mu

niz-Ferrer,

M., and V

elez-Calle, A. (2018). Business cultural in-

telligence quotient: A ﬁve-country study. Thunderbird

International Business Review, 60(3):237–250.

An, B., Brown, D. E., Hazell, R. M., and Grazaitis, P.

(2017). Cultural dimension theory based simulations

for us army personnel. In International Conference on

Social Computing, Behavioral-Cultural Modeling and

Prediction and Behavior Representation in Modeling

and Simulation, pages 65–70. Springer.

Ausburn, L. J., Martens, J., Washington, A., Steele, D., and

Washburn, E. (2009). A cross-case analysis of gen-

der issues in desktop virtual reality learning environ-

ments.

Bailenson, J. N., Yee, N., Blascovich, J., Beall, A. C., Lund-

blad, N., and Jin, M. (2008). The use of immersive

virtual reality in the learning sciences: Digital trans-

formations of teachers, students, and social context.

The Journal of the Learning Sciences, 17(1):102–141.

Bashiri, A., Ghazisaeedi, M., and Shahmoradi, L. (2017).

The opportunities of virtual reality in the rehabili-

tation of children with attention deﬁcit hyperactivity

disorder: a literature review. Korean journal of pedia-

trics, 60(11):337–343.

CHIRA 2018 - 2nd International Conference on Computer-Human Interaction Research and Applications

Bastiaens, T., Wood, L., and Reiners, T. (2014). New lands-

capes and new eyes: The role of virtual world de-

sign for supply chain education. Ubiquitous Learning,

6(1):37–49.

Bradley, R. and Newbutt, N. (2018). Autism and virtual

reality head-mounted displays: A state of the art sys-

tematic review. Journal of Enabling Technologies, 11.

Chua, R. Y. and Ng, K. Y. (2017). Not just how much you

know: Interactional effect of cultural knowledge and

metacognition on creativity in a global context. Ma-

nagement and Organization Review, 13(2):281–300.

Coffey, A. J., Kamhawi, R., Fishwick, P., and Henderson, J.

(2017). The efﬁcacy of an immersive 3d virtual versus

2d web environment in intercultural sensitivity acqui-

sition. Educational Technology Research and Deve-

lopment, 65(2):455–479.

Fishwick, P. A., Henderson, J., Fresh, E., Futterknecht, F.,

and Hamilton, B. D. (2008). Simulating culture: an

experiment using a multi-user virtual environment. In

Simulation Conference, 2008. WSC 2008. Winter, pa-

ges 786–794. IEEE.

Fleiss, J. L. and Cohen, J. (1973). The equivalence of weig-

hted kappa and the intraclass correlation coefﬁcient as

measures of reliability. Educational and psychologi-

cal measurement, 33(3):613–619.

Fowler, S. M. and Pusch, M. D. (2010). Intercultural si-

mulation games: A review (of the united states and

beyond). Simulation & Gaming, 41(1):94–115.

Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark,

D., Spanlang, B., and Slater, M. (2017). Virtual re-

ality in the assessment, understanding, and treatment

of mental health disorders. Psychological medicine,

47(14):2393–2400.

Freina, L., Bottino, R., Tavella, M., and Dagnino, F. (2016).

Immersions impact on performance in a spatial reaso-

ning task. In International Conference on Games and

Learning Alliance, pages 211–220. Springer.

Freina, L. and Ott, M. (2015). A literature review on im-

mersive virtual reality in education: state of the art

and perspectives. In The International Scientiﬁc Con-

ference eLearning and Software for Education, vo-

lume 1, page 133. ” Carol I” National Defence Uni-

versity.

Gallus, J. A., Gouge, M. C., Antolic, E., Fosher, K., Jas-

parro, V., Coleman, S., Selmeski, B., and Klafehn,

J. L. (2014). Cross-cultural competence in the depart-

ment of defense: an annotated bibliography. Technical

report, DTIC Document.

Guadagno, R. E., Muscanell, N. L., Okdie, B. M., Burk,

N. M., and Ward, T. B. (2011). Even in virtual envi-

ronments women shop and men build: A social role

perspective on second life. Computers in Human Be-

havior, 27(1):304–308.

Hlavac, M. (2015). stargazer: Well-Formatted Regression

and Summary Statistics Tables. Harvard University,

Cambridge, USA. R package version 5.2.

Hupont, I., Gracia, J., Sanagust

ın, L., and Gracia, M. A.

(2015). How do new visual immersive systems inﬂu-

ence gaming qoe? a use case of serious gaming with

oculus rift. In Quality of Multimedia Experience (Qo-

MEX), 2015 Seventh International Workshop on, pa-

ges 1–6. IEEE.

Kageyama, A. and Tomiyama, A. (2016). Visualization fra-

mework for cave virtual reality systems. International

Journal of Modeling, Simulation, and Scientiﬁc Com-

puting, 7(04):1643001.

Katsouri, I., Tzanavari, A., Herakleous, K., and Poullis, C.

(2015). Visualizing and assessing hypotheses for ma-

rine archaeology in a vr cave environment. Journal on

Computing and Cultural Heritage (JOCCH), 8(2):10.

Lane, H. C., Hays, M. J., Core, M. G., and Auerbach, D.

(2013). Learning intercultural communication skills

with virtual humans: Feedback and ﬁdelity. Journal

of Educational Psychology, 105(4):1026.

Makransky, G., Terkildsen, T. S., and Mayer, R. E. (2017).

Adding immersive virtual reality to a science lab si-

mulation causes more presence but less learning. Le-

arning and Instruction.

Matsumoto, D. and Hwang, H. C. (2013). Assessing cross-

cultural competence: A review of available tests.

Journal of cross-cultural psychology, 44(6):849–873.

Merriam-Webster Online (2017). Merriam-Webster Online

Dictionary.

Tackett, J. (2016). Using a 3D immersive environment to

study signal ﬂow in music technology. PhD thesis,

Colorado Technical University.

Tasdemir, S. A. and Prasolova-Førland, E. (2014). Visuali-

zing afghan culture in a virtual village for training cul-

tural awareness in military settings. In Information Vi-

sualisation (IV), 2014 18th International Conference

on, pages 256–261. IEEE.

Van Dyne, L., Ang, S., Ng, K. Y., Rockstuhl, T., Tan,

M. L., and Koh, C. (2012). Sub-dimensions of the four

factor model of cultural intelligence: Expanding the

conceptualization and measurement of cultural intel-

ligence. Social and personality psychology compass,

6(4):295–313.

Webster, R. (2016). Declarative knowledge acquisition in

immersive virtual learning environments. Interactive

Learning Environments, 24(6):1319–1333.

Wray, R., Lane, H. C., Stensrud, B., Core, M., Hamel, L.,

and Forbell, E. (2009). Pedagogical experience mani-

pulation for cultural learning. In Proceedings of the

2nd Workshop on Culturally Aware Tutoring Systems

at the 14th International Conference on Artiﬁcial In-

telligence in Education, pages 35–44.

Yee, N. (2006). The psychology of mmorpgs: Emotional

investment, motivations, relationship formation, and

problematic usage. Avatars at work and play: Col-

laboration and interaction in shared virtual environ-

ments, 34:187–207.

Comparing the Performance of an Immersive Virtual Reality and Traditional Desktop Cultural Game