Exploring Concept Design of User Experience with Diegetic

Approach for Cultural Heritage Virtual Realtity Exhibition

Cahya Miranto

1,2

, Hestiasari Rante

and Sritrusta Sukaridhoto

Department of Informatics and Computer Engineering, Politeknik Elektronika Negeri Surabaya, Surabaya, Indonesia

Department of Informatics Engineering, Politeknik Negeri Batam, Batam, Indonesia

Keywords: Virtual Reality, Virtual Exhibition, User Experience, Diegetic, Game.

Abstract: Museum and exhibition event have started to digitalize the artifact or artwork collection for preservation and

presentation. Virtual Reality (VR) medium affecting the way people experience and perceive heritage. Virtual

museum and exhibition does not only focus on displaying artwork or artifact but also but also the narration of

its story. Game interfaces consist of both diegetic and non-diegetic components. Mostly in virtual exhibition,

user interface using non-diegetic causing continuous interruptions of the user experience. Player involvement

and immersion experience can be influenced by a range of factors, from the controllers to the user interface.

It is crucial to represent cultural information and system interaction to the user through interfaces without

distracting user’s attention. In game industry, the use of diegetic design transformation of graphical user

interface has already been used. In this paper, we will look at the benefits and also highlight the possibility

and issue of the diegetic approach in designing the features required by the system. In our research, there are

3 features in virtual exhibitions that can be studied using a diegetic approach: interfaces design regarding to

artifacts, interfaces to the system, and interfaces to applications. Our findings suggest that using diegetic

interface is possible and have a potential to extend user experience.

1 INTRODUCTION

The use of Virtual Reality (VR) has grown

significantly, entering many different aspects. Most

of the museum and exhibition event have started to

digitalize the artifact or artwork collection for

preservation and presentation. With device based

experiences like Head Mounted Displays (HMDs)

with input control like Leap Motion Controller,

Microsoft Kinect, VR technologies can create

immersive and fun experience. The use of a virtual

medium allows artists to display their work. Due to

pandemic condition, two exhibition event in

Indonesia named “Tales of Nowhere” by Museum

MACAN and OPPO Art Jakarta Virtual 2020 by Art

Jakarta were held virtually using web-based virtual

tour (Kemenparekraf, 2021).

VR medium affecting the way people experience

and perceive heritage. VR has been used for

reconstructing historical environments (Technology,

2016), for increased visitor engagement and

https://orcid.org/0000-0002-1298-9800

https://orcid.org/0000-0002-8157-5863

education (Bozzelli et al., 2019), and for creating

interactive, engaging, and immersive experiences in

heritage environments (Pagano et al., 2020). Instead

of just displaying content, the app can also provide

visitors or users with knowledge through interactive

informal learning (Tengku Wook et al., 2016).

Although, according to (Yazid & Jantan, 2019), value

in UX is important and should be considered from the

holistic point of view to develop a good application

and UX also become the key focus in order to create

emotion quality into interaction. This sense of

presence is not only influenced by a hardware

solution but also by graphical user interface (GUI)

(Salomoni et al., 2017).

In this research, we introduce the second stage of

testing for our exhibition Cultural Heritage (CH)

themed VR prototype (Miranto et al., 2020).

Designing diegetic interface elements with the latest

evolution of GUIs in the gaming industry, to enhance

user experience. We aim to analysis of their

suitability in the cultural heritage domain and to

272

Miranto, C., Rante, H. and Sukaridhoto, S.

Exploring Concept Design of User Experience with Diegetic Approach for Cultural Heritage Virtual Realtity Exhibition.

DOI: 10.5220/0010943900003260

In Proceedings of the 4th International Conference on Applied Science and Technology on Engineering Science (iCAST-ES 2021), pages 272-277

ISBN: 978-989-758-615-6; ISSN: 2975-8246

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

provide users with a natural and immersive

environment which, however, does not shift the focus

of the user away from the virtual cultural artwork.

The main goal of this research is to analyze the

possibilities offered by the application of a diegetic

solution to the design of an interface for a cultural

heritage virtual application

2 RELATED WORKS

2.1 Current Virtual Museum

There have been many study in Virtual Environment

(VE) recently due to the enormous urgent need

caused by the pandemic. Several virtual exhibition

and museum development studies focus more on 3D

reconstructions of building or event(Fazio et al.,

2019; Previtali, 2019; Walmsley & Kersten, 2020),

artwork (Adamopoulos et al., 2020; McCarthy et al.,

2020; Tanasi et al., 2021; Zhenrao et al., 2021) with

historical and esthetic value. Apart from the

entertainment aspect, virtual museums and

exhibitions are also able to display information as a

learning medium. With the following requirements,

system requires interaction and feedback from the

user to the system, then displaying related

information through the interface. In our findings,

some virtual developments using Head Mounted

Device (HMD) present this information with

Graphical User Interface (GUI) by using non-diegetic

type causing continuous interruptions of the user

experience.

For instance, development virtual museum for the

Stoa of Attalos (Skamantzari et al., 2017), allowing

user to exploring a 3D reconstructed artifact and

environment, application provide information about

related artifact by using a non-diegetic

semitransparent windows shown at the screen layer

appearing superimposed over the virtual

environment. An investigation of visualization and

interaction with reconstructed Archaeological

Museum was performed in (Clini et al., 2018). In this

case, user can interact with artifact object and gain

information using diegetic GUI near the artifact

object. However some function like recall map,

switch panorama and changing scene still using non-

diegetic design.

Extending the analysis to other fully immersive

CH applications using Cave Automatic Virtual

Environment (CAVE). In (Roussou & Slater, 2020)

using CAVE system focusing on the topic of math

fractions interactive methods and passive methods.

We recognize that diegetic interface is also

underexplored in these circumstances.

2.2 GUI Elements and Type

GUI behave like a 2D plate to put stickers on screen.

When player exploring the game world, screen GUI

stays in the same place. User Interface (UI) in video

games is the way players can interact with the game

and receive feedback of their interaction. This

includes several aspects of gameplay such as the

storyline, controls, graphics, visual perspectives

(point-of-view), behaviors of AIs and level designs.

A suitable description of the design space of GUI for

games was initially proposed by Fagerholt and

Lorentzon (Fagerholt & Lorentzon, 2009). There are

four types of interfaces depending on how linked to

the narrative (see Figure 1).

Figure 1: Four interface type Fagerholt and Lorentzon’s

terminology

1. Non-diegetic interface is the category that a

game’s HUD. This information exists purely

for the player to view on screen, and the

characters within the game are not aware of

its presence.

2. Diegetic interface is one that exists within the

game world and the characters within the

game are aware of its presence.

3. Meta interfaces are interface that the

characters in the game are aware of, but they

are not physically display within the scene.

4. Spatial interface exists in the scene, but the

characters within the game are not aware of

it.

In the previous topic about VR in CH application,

GUI mainly use to display information consisting of

text, icons, button, simple effect and other multimedia

Exploring Concept Design of User Experience with Diegetic Approach for Cultural Heritage Virtual Realtity Exhibition

273

source that occasionally obscure areas of the screen.

Therefore, the first attempt with an HMD should

focus on putting GUI components into the Virtual

Environment (VE). This way provides visual comfort

by reducing the range of parallaxes, which may

interfere with the fictional experience and

consequently with the final immersion perceived by

the user. As a result, deciding how to arrange the GUI

elements in 3D space becomes critical.

Diegetic concept in (Fagerholt & Lorentzon,

2009) describe that all indicator line menu bar or

status icon on screen are considered as not being part

of the game world, because a game character is

unaware about them. They are defined as extra-

diegetic. On the other hand, the intra-diegetic

comprises the characters, its objects, narrative of the

game and actions.

In (Schild & Masuch, 2012) proposed GUI

element and game information into three group :

extra-diegetic, intra-diegetic and spatial references.

Extra-diegetic is common UI elements to show

information to the user. Intra-diegetic object element

in game to show information to the user (e.g. watch

to show time). Spatial reference which are technically

part of the Head-Up Display (HUD) like a crosshair

in shooter game.

2.3 Diegetic UI Experiences in Video

Games

In game industry, there are some games have totally

or partially integrated information in basic HUDs into

the narrative. Dead Space is one of the famous

example, which places the life bar on the back of the

armor. With third person view, player can see health

bar from the back of the armor without needed to use

HUD. Also another game element in Dead Space is

show in form of holographic view near the object like

basic information about pick able object or near the

weapon to indicate ammo.

Removing non-diegetic elements increased

immersion only from a certain level of expertise and

difficulty (Iacovides et al., 2015). Without HUD,

using only diegetic interface, experts were less

distracted. However, for novice user level, the

presence of a basic HUD was essential. HUD giving

the important information that, if absent, significantly

affect their gaming experience. Another study with

experts only (Peacocke et al., 2018) have a relatively

similar result for first-person shooter task like ammo

monitoring performance was higher when the ammo

indicator using diegetic design that plant in to the gun.

Experiencing full immersion experience from

player character in a first-person perspective serves to

enhance the user engagement. One recent study found

that perceived sense of presence was higher using

HMD VR compared to the desktop application

(Pallavicini et al., 2019). Diegetic interfaces designed

to respecting the narrative of the game without

compromise user immersion, seem to offer a possible

solution to the basic HUD problem.

3 ROLE ANALYSIS OF

DIEGETIC INTERFACES FOR

CULTURAL HERITAGE VR

APPLICATIONS

The emerge of VR technologies in video games

industry has led to a re-adapting of game mechanic

and their interfaces study, we argue that a similar

change ought to be attempted for the enjoyment of the

digital heritage. However, the solutions applied to

game are not suitable for cultural heritage domain due

to the few but fundamental differences between the

two areas that require a specific analysis.

1. In game development, player may have

complex status and list of task. So all the

information needed at that point must be

display on screen to make easier for player to

see. On the contrary in CH application, player

only need to interact and gain information

related to artwork through an interface that

must not occlude the artwork.

2. In game development, sometimes it is the

interface itself that tries to draw the player’s

attention (e.g. flashing bar to emphasize a low

health). While in CH application, it is the user

who decides when to pay attention to the

interface.

In this context, with a proper design interface, will

enhance the sense of presence and immersion of the

user. It becomes important to evaluate, depending on

requirement system and on the type of information or

data to be presented, the most suitable non-diegetic or

diegetic design solutions that may contribute to the

enhancement of the virtual environment.

3.1 Cultural Heritage Virtual

Exhibition

For the test mockup, we using our early development

research Cultural Heritage VR Exhibition (Miranto et

al., 2020). The application using HMD with VR input

control to interact with object and system. For now,

iCAST-ES 2021 - International Conference on Applied Science and Technology on Engineering Science

274

player can explore the exhibition area. Exhibition

area fill with quantity of booth from several vendor or

artist (see Figure 2).

Figure 2: Early development cultural heritage VR

exhibition

First, we describe some of the features of the

exhibition system that need to be design for user

interface. Based on early development:

1. User is able to view additional information

about the related artifact, general information

about exhibition event and each vendor,

2. User are able to use the functions offered by

the system to explore the exhibition area, like

using a map.

3. User are able to set configurations on the

system.

With this requirement, next we plan concept for

user interface based on diegetic approach.

3.2 Diegetic Interface for Virtual

Reality Exhibition

The primary objective of this study is to look at the

possibility of using a diegetic solution to build an

interface for a virtual cultural heritage application. In

this research, we focused on the creation of a virtual

exhibition where visitors may appreciate the artworks

while learning new things about them. In order to

optimize user experience, we specify the sorts of

cultural data that should be supplied to visitors

throughout an exhibition. The identification of a

feasible method for integrating this information into

the interface will be presented for each type of data,

as well as our thoughts on the issues that arise from

attempting to include fiction into the design of UI

components in the CH domain.

Our virtual exhibition may be divided into three

categories based on this requirement: those connected

to the artefact, those related to the user experience,

and those related to the application. The three groups

are based on common design patterns found in

gaming interfaces.

Figure 3: Possible visualization of cultural data related to

the artwork. (a) the non-diegetic representation. (b) the

diegetic visualization.

First about data related to artwork. These data are

related to the visualization of all that additional

information that is used to augment a virtual object.

In a GUI design for a cultural heritage experience,

this represents the main requirement that needs to be

satisfied by developers. Data mostly composed of

text, images or even videos. As demonstrated in

Figure 3, with non-diegetic and diegetic concept, user

will have optimal readability of the content. However

with non-diegetic, GUI will covering half of the

screen. With diegetic concept, user can control where

to see or read base on VR hand control position. This

requires the designer to add a new 3D objects

fictionally integrated only for this case, like books,

electronic tablet or multimedia boards into the scene

or attach to the player. In our opinion, with diegetic

solution in addition to offering a good readability of

the data, and more flexible with the position based on

player input control.

Figure 4: Visualization of information related to the user

experience. (a) non-diegetic represantation placed at the

screen layer. (b) the diegetic version.

Next visualization of information related to the

user experience. For this case, we are referring to all

function that offer from the application to the user. As

Exploring Concept Design of User Experience with Diegetic Approach for Cultural Heritage Virtual Realtity Exhibition

275

an example, we can consider information related to

user’s position using some kind of map or GPS

tracker. As demonstrated in Figure 4, with non-

diegetic design have similar problem as before.

Occluding the virtual environment and can disturb the

user. Here again, a diegetic design solution appears as

the most reasonable. With using VR controller, user

can grab object like a map or GPS tracker to show

their location. The map should be designed also as a

multimedia map able to show other information such

booth logo or brand or exhibition area that already

been visited. In this way, the information which

contributes to improving the user experience is

presented in a discreet manner without diverting user

attention.

Figure 5: Approaches to show information related to the

application. (a) the non-diegetic representation. (b) a

possible representation.

Lastly about data related to the application. This

section refers to the configuration of the virtual

environment. While in the gaming context this

generally corresponds to configuration such as the

exit or changing location, lowering music volume in

the exhibition, saving and loading of a game state.

Certainly, this group consists of the most complex

interaction and information to manage.

As demonstrated in Figure 4, in our opinion, a

fully diegetic design solution requires a greater effort.

To exit the exhibition, user need to walk through exit

door. With simple animation, application can tell to

the user that they can interact with the exit door. For

another idea about lowering volume music in

exhibition area, designer need to create object and

interaction system for user. Design of the object must

be self-explanatory for the user to avoid confusion.

Finally, the solution involving the creation of

diegetic design interfaces applicable to all three types

of group. However, in some cases, this diegetic

solution will limit the user interaction with the object.

These solutions could be more suitable for the first

two types of group we have analyzed because they do

not need to be always available to the user. Recent

studies have shown that the diegetic interface

approach is very suitable for VR applications that

focus on story-driven games that require immersive

experiences with the surrounding environment

(Hoppe et al., 2021). Our future research is to testing

the diegetic approach by users with a varied

background regarding their experience in virtual

reality.

4 CONCLUSIONS

A virtual museum or exhibition not only represents an

artwork or artifact, but also tells the story behind it.

The challenge in designing immersive virtual

applications is presenting such information to the user

without sacrificing the system's usability, and in

particular in the cultural sector, maintaining the work

of art at the center of the user’s attention.

The purpose of this study was to explore the

possibility of diegetic design UI using HMD and VR

control that already implemented in games can also

be implemented to cultural heritage domain. With

more effort on the UI design side of implementing a

diegetic interface, give a possible solution to

represent more value to the work. In conclusion, we

believe that with using diegetic design interface may

enhance involvement and immersion user to the

cultural heritage content.

ACKNOWLEDGEMENTS

We would like to thank the Politeknik Elektronika

Negeri Surabaya for supporting and making this

project possible.

REFERENCES

Adamopoulos, E., Bovero, A., & Rinaudo, F. (2020).

Image-based metric heritage modeling in the near-

infrared spectrum. Heritage Science, 8(1), 1–13.

https://doi.org/10.1186/s40494-020-00397-w

Bozzelli, G., Raia, A., Ricciardi, S., De Nino, M., Barile,

N., Perrella, M., Tramontano, M., Pagano, A., &

Palombini, A. (2019). An integrated VR/AR framework

for user-centric interactive experience of cultural

heritage: The ArkaeVision project. Digital Applications

in Archaeology and Cultural Heritage, 15(September).

https://doi.org/10.1016/j.daach.2019.e00124

Clini, P., Ruggeri, L., Angeloni, R., & Sasso, M. (2018).

Interactive immersive virtualmuseum: Digital

documentation for virtual interaction. International

Archives of the Photogrammetry, Remote Sensing and

Spatial Information Sciences - ISPRS Archives, 42(2),

251–257. https://doi.org/10.5194/isprs-archives-XLII-

2-251-2018

iCAST-ES 2021 - International Conference on Applied Science and Technology on Engineering Science

276

Fagerholt, E., & Lorentzon, M. (2009). Beyond the HUD.

User Interfaces for Increased Player Immersion in FPS

Games. In Chalmers University. http://scholar.google.

com/scholar?hl=en&btnG=Search&q=intitle:Beyond+

the+HUD+User+Interfaces+for+Increased+Player+Im

mersion+in+FPS+Games#0

Fazio, L., Lo Brutto, M., & Dardanelli, G. (2019). Survey

and virtual reconstruction of ancient roman floors in an

archaeological context. ISPRS Annals of the Photo-

grammetry, Remote Sensing and Spatial Information

Sciences, 42(2/W11), 511–518. https://doi.org/10.51

94/isprs-Archives-XLII-2-W11-511-2019

Hoppe, M., Schmidt, A., & Ville, M. (2021). Diegetic and

Non-diegetic Health Interfaces in VR Shooter Games

Diegetic and Non-Diegetic Health Interfaces in VR

Shooter Games. Human-Computer Interaction -

INTERACT 2021, August, 3–11.

https://doi.org/10.1007/978-3-030-85613-7

Iacovides, I., Cox, A., Kennedy, R., Cairns, P., & Jennett,

C. (2015). Removing the HUD: The impact of non-

diegetic game elements and expertise on player

involvement. CHI PLAY 2015 - Proceedings of the

2015 Annual Symposium on Computer-Human

Interaction in Play, 13–22. https://doi.org/10.1145/

2793107.2793120

Kemenparekraf. (2021). Pameran Seni Rupa yang Sukses

Digelar Secara Virtual. Kemenparekraf.Go.Id.

https://kemenparekraf.go.id/ragam-ekonomi-

kreatif/Pameran-Seni-Rupa-yang-Sukses-Digelar-

Secara-Virtual

McCarthy, J., Sebo, E., Wilkinson, B., & Sheehan, F.

(2020). Open workflows for polychromatic

reconstruction of historical sculptural monuments in

3D. Journal on Computing and Cultural Heritage,

13(3). https://doi.org/10.1145/3386314

Miranto, C., Rante, H., Sukaridhoto, S., Pasila, F., & Kiwi,

H. K. (2020). Preliminary Development of Virtual

Reality for Batik Exhibition. Psychology and Education

Journal, 57(9), 2286–2292.

Pagano, A., Palombini, A., Bozzelli, G., De Nino, M.,

Cerato, I., & Ricciardi, S. (2020). ArkaeVision VR

game: User experience research between real and

virtual paestum. Applied Sciences (Switzerland), 10(9),

1–38. https://doi.org/10.3390/app10093182

Pallavicini, F., Pepe, A., & Minissi, M. E. (2019). Gaming

in Virtual Reality: What Changes in Terms of Usability,

Emotional Response and Sense of Presence Compared

to Non-Immersive Video Games? Simulation and

Gaming, 50(2), 136–159. https://doi.org/10.1177/

1046878119831420

Peacocke, M., Teather, R. J., Carette, J., MacKenzie, I. S.,

& McArthur, V. (2018). An empirical comparison of

first-person shooter information displays: HUDs,

diegetic displays, and spatial representations.

Entertainment Computing, 26(April 2017), 41–58.

https://doi.org/10.1016/j.entcom.2018.01.003

Previtali, M. (2019). A virtual logbook for the

documentation of a continuously changing

archaeological site: The san clemente site in Albenga

(Italy).

Virtual Archaeology Review, 10(21), 56–66.

https://doi.org/10.4995/var.2019.11916

Roussou, M., & Slater, M. (2020). Comparison of the Effect

of Interactive versus Passive Virtual Reality Learning

Activities in Evoking and Sustaining Conceptual

Change. IEEE Transactions on Emerging Topics in

Computing, 8(1), 233–244. https://doi.org/10.1109/

TETC.2017.2737983

Salomoni, P., Prandi, C., Roccetti, M., Casanova, L.,

Marchetti, L., & Marfia, G. (2017). Diegetic user

interfaces for virtual environments with HMDs: a user

experience study with oculus rift. Journal on

Multimodal User Interfaces, 11(2), 173–184.

https://doi.org/10.1007/s12193-016-0236-5

Schild, J., & Masuch, M. (2012). Designing stereoscopic

information visualization for 3D-TV: What can we

learn from S3D gaming? Stereoscopic Displays and

Applications XXIII, 8288, 82882P. https://doi.org/

10.1117/12.911967

Skamantzari, M., Kontogianni, G., Georgopoulos, A., &

Kazanis, S. (2017). Developing a virtual museum for

the Stoa of Attalos. 2017 9th International Conference

on Virtual Worlds and Games for Serious Applications,

VS-Games 2017 - Proceedings, September, 260–263.

https://doi.org/10.1109/VS-GAMES.2017.8056611

Tanasi, D., Hassam, S., Kingsland, K., Trapani, P., King,

M., & Calì, D. (2021). Melite Civitas Romana in 3D:

Virtualization Project of the Archaeological Park and

Museum of the Domus Romana of Rabat, Malta. Open

Archaeology, 7(1), 51–83. https://doi.org/10.1515/

opar-2020-0126

Technology, I. (2016). Reconstructing the original

Splendour of the House of Caecilius Iucundus : a

complete methodology for virtual archaeology aimed at

digital exhibition. Scires, 6(1), 51–66.

https://doi.org/10.2423/i22394303v6n1p51

Tengku Wook, T. S. M., Mohd Judi, H., Sahari @ Ashaari,

N., Mohamed, H., Mat Noor, S. F., & Rahim, N. (2016).

Interaction Design Model In Virtual Museum

Environment. Asia-Pacific Journal of Information

Technology & Multimedia, 05(01), 71–81.

https://doi.org/10.17576/apjitm-2016-0501-07

Walmsley, A. P., & Kersten, T. P. (2020). The imperial

cathedral in Königslutter (Germany) as an immersive

experience in virtual reality with integrated 360°

panoramic photography. Applied Sciences

(Switzerland), 10(4). https://doi.org/10.3390/app10041

517

Yazid, M. A., & Jantan, A. H. (2019). An integrated

conceptual model of visually impaired users’

experience and technology acceptance of a website.

International Journal of Advanced Trends in Computer

Science and Engineering, 8(1.4 S1), 318–322.

https://doi.org/10.30534/ijatcse/2019/4981.42019

Zhenrao, C., Chaoyang, F., Qian, Z., & Fulong, C. (2021).

Joint development of cultural heritage protection and

tourism : the case of Mount Lushan cultural landscape

heritage site. Heritage Science, 9, 1–17.

https://doi.org/10.1186/s40494-021-00558-5

Exploring Concept Design of User Experience with Diegetic Approach for Cultural Heritage Virtual Realtity Exhibition

277