A Framework for Visualizing the Dynamic Events of Carbon

Nanocomposites using Virtual and Augmented Reality Tools

Razib Iqbal, Taylor Kuttenkuler, Chad Brewer and Ridwan Sakidja

College of Natural and Applied Sciences, Missouri State University, Springfield, MO, U.S.A.

Keywords: Carbon Nanocomposites, Molecular Dynamics, Oxidization, Unity Game Engine, Google Daydream, HTC

Vive, Virtual Reality, Augmented Reality.

Abstract: Atomic interactions pertaining to carbon-nanocomposites can be elusive and hard to comprehend, and as such

great benefit can be gained through a visualization of these interactions within a virtual or augmented reality

setting. In this paper, we present a framework that can be used for Material Science research and education

incorporating topics related to the dynamics in nanomaterials. We developed the proof of concept

implementation of this framework for Virtual Reality (VR) and Augmented Reality (AR) settings using the

Unity game engine. Throughout this paper, we discuss our framework as well as the related user experiences

and performance measurements we gathered when using our framework with the Google Daydream, HTC

Vive, and Microsoft HoloLens in introducing scientists to the use of AR and VR as a tool for nanocomposite

and molecular dynamics research.

1 INTRODUCTION

Carbon-nanocomposites have garnered more attention

over the years due to their special properties like low

density, high specific surface area, and thermal and

mechanical stability (Krolow, 2013). They are used in

various areas including batteries, biomedical

technology, and structural components (Baughman,

2002). Carbon-nanocomposites have gathered interest

due to a variety of reasons, including having a high

elastic modulus and tensile strength being just two

among many others (Arash, 2013). With this sudden

popularity, comes the need for proper Molecular

Dynamics (MD) simulation tools to aid in the research

of these nanocomposite materials. These tools are

important, as carbon nanotubes and nanocomposites in

general are extremely difficult to study due to their size

and their atomic nature. MD simulation tools help

researchers to better understand the properties of

carbon nanotubes (CNT) as well as how they will react

under certain circumstances. One such circumstance

that is of great interest to many researchers, is how

CNT react under various conditions when subjected to

the process of oxidation. Oxidation poses a challenge

towards the use of CNT because oxygen molecules

will bind to carbon molecules located within

imperfections in the carbon nanotube structure and

cause the nanotube to slowly decay through a process

of fraying and decomposition (Zhang, 2003). These

imperfections can be difficult to detect and are

typically common at the edges of a carbon nanotube.

In order to bring greater flexibility to the study of these

CNT in virtual and augmented reality, we developed a

framework to better understand how carbon-

nanocomposites react to oxidation. Researchers may

upload data related to reactions they may be currently

researching, and view them within virtual or

augmented reality settings on a variety of different

devices, e.g. Google Daydream, Microsoft HoloLens,

as well as view them concurrently with multiple other

researchers. This paper aims to present our generic

framework for implementing this system of

nanocomposites visualization and to increase the

flexibility of the evaluation of nanocomposites.

The rest of the article is organized as follows:

Previous initiatives and tools to visualize Molecular

Dynamics simulation in VR/AR environments is

highlighted in Section-2. Our proposed framework and

relevant implementation details are provided in

Section-3. Sections 4 and 5 contain the device-specific

performance of our proof-of-concept framework

implementation and end user survey results. Finally,

we highlight our observations based on data collected

from performance and survey results and the future

direction of our ongoing work in Section-6.

Iqbal, R., Kuttenkuler, T., Brewer, C. and Sakidja, R.

A Framework for Visualizing the Dynamic Events of Carbon Nanocomposites using Virtual and Augmented Reality Tools.

DOI: 10.5220/0007933903310336

In Proceedings of the 16th International Joint Conference on e-Business and Telecommunications (ICETE 2019), pages 331-336

ISBN: 978-989-758-378-0

331

2 RELATED WORKS

Research into applications of VR in relation to MD

simulation has been undertaken in the past. In 1998

(Ali, 1998), researchers created their own set of VR

hardware, labelled as the Cave Automatic Virtual

Environments (CAVE), as no set of VR tools existed

at the time that were commercially available and

allowed for an easy development process. They then

proceeded to create a software application which

served as the MD simulation framework which was

only compatible with the CAVE system.

More current research has also been undertaken in

an attempt to realize a VR software framework for MD

simulation which is compatible with modern, more

commercially available VR tools and hardware

systems. In one such study published in 2018

(O’Connor, 2018), researchers created a MD

simulation framework for use with the HTC Vive

which allowed for the real time simulation of atomic

structures undergoing MD. This research, while

limited to only one set of VR tools (the HTC Vive),

demonstrates the incredible potential of MD

simulations within a VR Environment when paired

with powerful computational assets made available

through modern technologies, like cloud computing.

Along with research studies regarding MD

simulations, commercial applications also exist with

the aim of providing an educational framework for the

study of atomic molecules as well as to improve

research capabilities.

MEL Science (melscience.com) is company that

specialize in educating children on the topic of

chemistry. The company has created a VR application

called the MEL Chemistry app which allows users to

view chemical models. Although the application

presents atomic structures to the user, these structures

are static and do not move. The application is heavily

catered towards students at the primary education

grade level and does not contain robust tools for the

study of more detailed chemical models for

researchers.

EduChemVR is a start-up company based out of

Sweden that also specializes in molecular and atomic

education. The company has recently embarked on an

endeavour to create applications to educate children on

these topics and has, as such, created a VR application

for students called Learning Carbons VR. This

application presents users with static models of various

carbon-based atomic structures. The application was

developed using the Unity Game Engine and utilizes a

Bluetooth gamepad controller for movement and

Google Cardboard for displaying the atomic structures.

Like MEL Chemistry, Learning Carbons VR is also

focused towards primary education grade level

students and does not contain robust tools for the study

of more detailed molecular models (educhem-vr.com).

LAMMPS is a fully realized MD simulation engine

package created by Sandia National Laboratories

(Plimpton, 1995). The program will produce atomic

trajectory information documenting the movement of

the individual atoms within the simulation box. Each

trajectory records the movements for every 2 femto-

seconds. The format of the LAMMPS results however

is typically not compatible with VR devices, even

though it does possess the ability to simulate complex,

large scale chemical reactions involving thousands of

individual atoms. We utilize LAMMPS to generate the

data necessary for the visualization of atoms and

molecular structures within our framework.

While the use of VR tools to simulate MD has

maintained high levels of interest throughout the years,

it is still very much a burgeoning field, especially when

dynamic processes are of interest. As a side effect,

access to frameworks for viewing MD simulations

within VR has remained somewhat stagnant and

researchers are often limited to a narrow scope of

hardware from which to choose from. Taking notice of

this issue, our framework aims to expand the range of

VR tools available to researchers hoping to utilize said

tools for MD simulations. Compared to the existing

works cited in the literature, our framework has also

expanded MD simulations to the realm of AR with the

same initiative to enable a broad set of tools and

hardware sets for researchers.

3 PROPOSED FRAMEWORK

AND IMPLEMENTATION

In Figure-1, we present our proposed framework for a

streamlined system to study MD simulations. The

framework’s ease of use and ability to be easily

integrated into a research pipeline is enhanced by the

fact that the application is compatible with a wide

range of different VR and AR platforms. This enables

researchers to be able to view a simulation in a variety

of different VR or AR settings and does not restrain the

researcher to a single device in which they must view

their simulations. On top of this, the framework

enables researchers to view any simulation

concurrently with other researchers and makes this

experience available across platforms. Therefore,

researchers are less limited by the devices they have

available and are able to take a more team centred

approach towards their research if they so desire.

SIGMAP 2019 - 16th International Conference on Signal Processing and Multimedia Applications

332

Figure 1: Our framework for CNT VR/AR simulation.

The proposed framework takes two inputs -

simulation data created using LAMMPS and user

interaction input collected via input devices associated

with the various VR and AR hardware. The user input

information fed into the system is used to allow a user

to interact with any given User Interface (UI) and to

allow for a user to control the camera when viewing a

simulation.

The framework accepts simulation data from both

cloud hosted databases and local file systems. In a

single user setting, a user may upload their simulation

data to a repository and then select the simulation from

the list of available data. In a multi-user setting, users

need the host-server (a user hosting the simulation) to

select and run the simulation they would like to view

and then other participants can join the same

simulation remotely.

The simulation data is fed into a file parser which

interprets the data for the given simulation and

generates atoms based on this data, allocating the

information gathered from the data to each atom’s

control script which then dictates the behaviour of the

atom throughout the simulation.

Figure 2: Molecular simulation data format.

In Figure-2, we show a snippet of one of our sample

data sets of a molecular simulation generated by

LAMMPS. The data set indicates the type of atoms

present within the simulation as letters located at the

top of the simulation file. C and O are present within

this simulation standing for Carbon and Oxygen. The

data set also shows the simulation matrix specifications

located just below the atom types present in the

simulation and indicates below the matrix, the number

of atoms of each type that are in the simulation. The

number of atoms present, in this case 190 and 8, are

written in the same order as the letter types at the top

of the simulation file. In other words, 190 carbon atoms

will be generated within the simulation along with 8

oxygen atoms. After this information is displayed in

the data set, the word “Direct” is used to indicate the

positional data of each atom present within the scene

for a single frame. The atom data is always listed in the

same order and is mapped to the atoms as they are

listed in the atom types and numbers section of the data

set. For example, in this data the first 190 sets of

positional data correspond to carbon atoms whereas the

last 8 correspond to oxygen atoms, following the

ordering of the listed atom types. Lastly, the data set

repeats this format for every frame that was simulated

within LAMMPS, only changing the positional data

within the “Direct” section of the data for each frame.

Figure 3: Simulation of carbon nanotube undergoing

oxidation. Left: Initial stage, Right: Structural fraying.

Once the data is processed, the user is then able to

control all aspects of a simulation through the UI and

to control the camera positioning through the camera

controller. After the initial set up, the camera data will

be used to render the simulation, atoms, and UI. The

proof-of-concept implementation of our proposed

framework in Unity (unity3d.com) goes as follows:

1. Upon entering the application, the user will be

presented with a menu and will utilize the various

input devices associated with each of the Virtual

or AR tool sets they are using to send data

regarding their input to the system.

2. Input data sent into to the system will then be

interpreted to allow for the user to interact with the

given User Interface (UI) as well as to control the

camera within the application once the simulation

starts.

3. The user will then be able to interact with the UI

to select whether or not they would like to view a

simulation by themselves or with other users.

A Framework for Visualizing the Dynamic Events of Carbon Nanocomposites using Virtual and Augmented Reality Tools

333

4. After making this selection, if the user chooses to

view a simulation by himself then the user will be

presented with a list of the available simulation

data sets from which to choose from. If the user

chooses to view a simulation with other users then

they will be connected to a dedicated server and

be able to view whichever simulation is currently

running on this server.

5. Users also have the ability to upload a simulation

data set generated by LAMMPS to a public

website repository so that the simulation will be

made available to be viewed within the application

or to request that the dedicated server begin

running a specific simulation dataset that the user

would like to view with other users.

6. Once a simulation is selected to be viewed, the

simulation data will be retrieved from the website

repository and subsequently parsed and

interpreted by the application. This process takes

place on the user’s system only in the case of a

single user simulation due to the fact that the initial

processing for a multi-user simulation is handled

by the host-server.

7. Once a simulation dataset has been parsed and

interpreted, data pertaining to atom behaviour is

allocated to each individual atom object and the

simulation begins running. If a user has joined a

multi-user session, the server simply sends this

atom behaviour to the client session of the user.

8. During the simulation, the user’s camera renders

the simulation (see Figure 3) and the user is given

control of the camera to enable them to move

around the simulation. In a multi-user setting, the

only user allowed to manipulate the playback of

the simulation is the user who has been assigned

the role of “Lead” upon connecting to the server.

9. When a user is finished viewing a simulation, they

can utilize the UI to exit the simulation they are

currently viewing and will be returned to the start

menu wherein the process will proceed from step

4 SYSTEM PERFORMANCE

Device performance when using our framework was

gathered via the frame rates measured while each

device simulated different molecular processes

containing different numbers of atoms (see Figure-4).

Due to the Google Daydream’s utilization of mobile

devices and smartphones, it is limited in the amount of

processing power as it has to perform calculations on

large amounts of data. This was also an issue present

within the Microsoft HoloLens as its onboard

processor is not powerful enough to render large data

sets when not tethered to a computer. Because of this,

for datasets with a number of atoms larger than 260,

the Google Daydream and Microsoft HoloLens were

unable to be utilized. For these larger datasets the HTC

Vive was utilized which must remain tethered to a

computer for processing. This allowed for the

simulation of much larger data sets. The range of data

sets that were tested for the various devices ranged

from 198 atoms to 10,000 atoms.

Figure 4: Framerate performance by number of atoms.

4.1 Google Daydream

The testing and development phases for the mobile and

smartphone version of our application were performed

on the Google Pixel 2. The Google Pixel 2 performed

best when the data set contained trajectory data for 300

atoms or less. For data sets that contain more than 300

atoms, the Google Pixel 2 could take up to a minute or

longer to transition into the scene containing the

simulation. At 1000 atoms or more, the Google Pixel 2

consistently crashed and would not enter the

simulation. This may be due to a failsafe mechanism

on the phone that will not allow a program to lock up

for more than a handful of minutes. It is also worth

noting that even if the Google Pixel 2 could have

rendered the simulation scene, it may not have been

able to simulate it in real time without freezing or

having a very low frame rate. Despite these limitations,

the Google Daydream implementation has the unique

advantage of making the program portable and any

user should be able to use the application provided they

have the hardware.

4.2 HTC Vive

For the HTC Vive to run the data sets, a GPU that is as

powerful or better than the Nvidia GTX 1060 and have

an Intel i5-4590 CPU or better is needed. All of our

available datasets were able to run on the HTC Vive.

We were unable to find an absolute cap wherein the

SIGMAP 2019 - 16th International Conference on Signal Processing and Multimedia Applications

334

HTC Vive simply could not handle the number of

atoms present within the simulation, however

framerate would continue to decline as the number of

atoms grew as can be seen in Figure-4.

4.3 Microsoft HoloLens

The Microsoft HoloLens makes use of a custom-built

onboard processor and GPU in order to project 3D

models into an AR environment. While the processor

and GPU contained within the HoloLens are fairly

powerful for a portable device of its kind, most of their

processing power is spent computing various

calculations to render and display 3D models within

the environment as well as keep track of their position

in world space. Therefore, the amount of processing

power made available to the HoloLens for other tasks

such as the calculations needed for our simulation was

limited and performance issues began appearing in

simulations with a number of atoms ranging as little as

260. It would appear that in this regard, the Microsoft

HoloLens performed less optimally than the Google

Daydream utilizing the Google Pixel 2. However, the

Microsoft HoloLens provides the same advantages as

the Google Daydream in that it is portable and able to

be used from any location as long as an internet

connection is established, as well as providing the

unique added ability to project a simulation into the

real world to be viewed in an AR setting.

5 SURVEY

A survey was composed to gain feedback on the

effectiveness of the framework to be used in the

development of VR and AR applications for MD

simulation. The survey also provided us with

information to better understand which device users

found more user friendly and which device enabled

them to better understand the interactions within the

MD simulations. A total of 18 adult participants (not

including the authors) were asked to participate in the

survey. Any learning or fatigue effect was mitigated by

randomizing which device users started with. There

were no strenuous tasks for the participants to perform,

except to interact with the user interface and move

around their respective simulation.

5.1 Survey Questions

Questions used for the survey are as follows:

Participants were given dichotomous, yes or no

questions asking if they enjoyed their experience and if

they have ever used a VR or AR device before. A

multiple answer question was given to determine past

user experience with any of the devices used in the

study, where users could choose any or all of the

devices they had used before, the choices were HTC

Vive, Oculus Rift, Smartphone VR, Microsoft

HoloLens, or other.

A multiple-choice question was used to determine

which device users enjoyed more and they were asked

to provide a short answer for their choice. Participants

were given Likert scale questions to gauge their ability

to quickly learn and utilize the controls for each of the

devices, where 1 represents having difficulty

understand the controls and 5 represents having no

difficulty understanding the controls.

Another Likert question was used to determine if

the experience improved participant understanding of

carbon nanocomposites, where 1 represents they

disagree that the experience improved their

understanding and 5 represents they agree the

experience improved their understanding. One more

Likert question was used to determine if the

participants found the experience to be educational at

all, where 1 represents not education and 5 represents

very educational.

Finally, participants were asked short answer

questions for what they enjoyed most about the

experience, what they disliked, and for any other

feedback in regards to the application.

5.2 Survey Results

The majority of participants (12 out of 18) have had

experience with VR or AR devices before. When asked

which devices the participants had used before, 7

answered HTC Vive, 7 answered Microsoft HoloLens,

1 answered Google Daydream, and 8 answered other.

For the question covering which device users

enjoyed the most, 10 (55.6%) of the participants

answered HTC Vive while the Microsoft HoloLens

received 5 (27.8%) votes. Users reported the HTC Vive

as the "most responsive" and "more immersive" than

the Daydream or HoloLens.

As for the questions pertaining to the controls and

ease of use of the devices, the Vive and Daydream had

similar responses where 10 (55.6%) of the participants

strongly agreed that the controls were easy to

understand. For the HoloLens, 6 (33.3%) strongly

agreed the controls were easy to understand, 5 (27.8%)

agreed, 6 (33.3%) were in the middle, and 1 (5.6%)

disagreed that the controls were easy to understand.

Next, participants were asked about whether or not the

experience was educational and whether or not the

experience improved their understanding of carbon

nano-composites. Overall, most of the participants

A Framework for Visualizing the Dynamic Events of Carbon Nanocomposites using Virtual and Augmented Reality Tools

335

found the experience to be educational and that it

improved their understanding of the structure of carbon

nanocomposites. Eight (44.4%) strongly agreed that

the experience was educational and that it helped

improve their understanding of carbon

nanocomposites, 6 (33.3%) agreed that it improved

their understanding of carbon nanocomposites, 7

(38.9%) agreed that the experience was educational,

and no one answered disagree or strongly disagree on

either question. Participants gave suggestions for

longer simulations, labels for the atoms, and more

information in the scenes for what is being viewed.

Lastly, participants were asked to give short

answers in regards to what they enjoyed about the

experience, what they disliked, and for other comments

that may be helpful to the developers. All but 1

participant, who responded “N/A”, answered that they

mostly enjoyed using the different devices. One

participant stated, “I enjoyed being able to observe a

complex phenomenon in a way that made it simple to

understand.” In regards to what users disliked about the

experience, 4 reported motion sickness with the Vive

and 5 reported general usability issues with the

HoloLens.

6 CONCLUSIONS

In this paper, a framework for researching molecular

dynamic processes within virtual and augmented

reality settings was presented. We demonstrated that

our framework provides researchers an easily

accessible and robust tool for visualizing the

interactions within VR and/or AR settings. Based on

the survey results, it is evident that the participants

were able to make use of our application to enhance

their understanding of the atomic interactions they

were witnessing along with the structure of carbon

nano-composites. It is worth noting that many of the

criticisms the survey participants mentioned or the

issues they were encountering were mostly due to the

limitations of the device they were using as opposed to

the framework we were showcasing to them. Despite

these issues, the survey demonstrated that the

framework we created is useable in all of the VR and

AR settings we presented to the participants. Another

significant finding of our survey was that many of the

users stated that they found the VR and AR aspects of

our framework when viewing the MD simulations

served to enrich the viewing experience. Many

participants mentioned that the ability to navigate a

MD simulation in a fully immersive 3D setting helped

them to better understand both the micro-level atomic

reactions and the macro-level molecular dynamic

events themselves. This illustrates the potential

benefits of our framework to enable researchers to gain

a more 3D understanding of molecular dynamic events

in AR/VR settings.

We aim to expand our current framework to allow

for the real time simulation of molecular dynamics

processes and are currently working on implementing

a system in which users can interact with a given

simulation even if all of the atomic events within the

simulation are not being calculated in real time. We

also plan on further developing the framework to allow

for greater detail in regards to the atomic reactions

taking place within any given simulation such as

showing the bonding between atoms, temperature,

atomic mass, and other relevant details of a CNT/MD

simulation.

REFERENCES

Krolow, M.Z., Hartwig, C.A., Link, G.C., Raubach, C.W.,

Pereira, J.S.F., Picoloto, R.S., Gonçalves, M.R.F.,

Carreño, N.L.V. and Mesko, M.F., 2013. Synthesis and

characterisation of carbon nanocomposites. In

NanoCarbon 2011 (pp. 33-47). Springer, Berlin,

Heidelberg.

Baughman, R.H., Zakhidov, A.A. and De Heer, W.A., 2002.

Carbon nanotubes--the route toward applications.

science, 297(5582), pp.787-792.

Arash, B., Wang, Q. and Varadan, V.K., 2014. Mechanical

properties of carbon nanotube/polymer composites.

Scientific reports, 4, p.6479.

Zhang, J., Zou, H., Qing, Q., Yang, Y., Li, Q., Liu, Z., Guo,

X. and Du, Z., 2003. Effect of chemical oxidation on the

structure of single-walled carbon nanotubes. The Journal

of Physical Chemistry B, 107(16), pp.3712-3718.

Ai, Z. and Fröhlich, T., 1998, August. Molecular dynamics

simulation in virtual environments. In Computer

Graphics Forum (Vol. 17, No. 3, pp. 267-273). Oxford,

UK and Boston, USA: Blackwell Publishers Ltd.

O’Connor, M., Deeks, H.M., Dawn, E., Metatla, O., Roudaut,

A., Sutton, M., Thomas, L.M., Glowacki, B.R., Sage, R.,

Tew, P. and Wonnacott, M., 2018. Sampling molecular

conformations and dynamics in a multiuser virtual reality

framework. Science advances, 4(6), p.eaat2731.

Plimpton, S., 1995. Fast parallel algorithms for short-range

molecular dynamics. Journal of computational physics,

117(1), pp.1-19.

SIGMAP 2019 - 16th International Conference on Signal Processing and Multimedia Applications

336