PHOTOREALISTIC RENDERING OF SCIENTIFIC DATA
Joel Ogden
1
, Jabari Jordan
1
, Chelsey Krol
2
, Tanya Papazian
3
, Hans-Peter Bischof
1
and Reynold Bailey
1
1
Department of Computer Science, Rochester Institute of Technology, Rochester, U.S.A.
2
Department of Computer Science, Mount Holyoke College, South Hadley, U.S.A.
3
Department of Computer Science, Montclair State University, Montclair, U.S.A.
Keywords:
Scientific Visualization, Maya, Spiegel.
Abstract:
Generating photorealistic images of astrophysical simulations can enhance the experience of watching galac-
tic visualizations for both the specialists who study the data and the average person who is simply interested
in outer space. Unfortunately, the astrophysicist who is creating the simulations typically lacks the expertise
required to generate photorealistic images. Likewise, a 3D artist may be unaware of the physics behind cer-
tain astrophysical events. We aim to use Spiegel, a user interface that controls the rendering of astrophysical
data and Maya, a high end 3D animation program, to allow a non-artist to easily create renders of photore-
alistic images. Spiegel provides a user-friendly interface for controlling the creation of potentially complex
rendering applications by individuals with little experience in computer programming. Since Spiegel’s basic
visualization capabilities are limited to simple primitives like points and lines, it was necessary to develop an
additional program for Spiegel to interface with Maya’s 3D rendering capabilities. This software interface is
called Miegel. Using Miegel, the astrophysicist now has access to Maya’s 3D rendering capabilities allowing
them to create stunning visualizations of astrophysical phenomena. In addition, new artistic effects can be
created with Maya in the form of presets, which can be integrated into the user’s visualization with minimal
knowledge of computer programming.
1 INTRODUCTION
Visualizations of scientific processes have always
been a crucial part of education and understanding
to the student and scientist alike. Being able to see
something as opposed to reading it from a textbook
allows for a multi-dimensional learning experience
and a more dynamic understanding of complex sub-
ject matter. Unfortunately not all visualizations are as
effective as they could be. This is because most sci-
entists do not have the training or skills in modern 3D
rendering techniques to produce compelling visuals.
Spiegel (Bischof et al., 2006) was developed to bridge
this gap by bringing tool that were previously only
available to a 3D artist into the hands of the scientist.
Spiegel is a software package that allows individuals
with little or no 3D rendering or programming ex-
perience to create simulations of scientific processes
based on data. The program in its basic form is how-
ever limited to how the data is displayed. Most visu-
alizations are displayed as simple points, which take
on the roles of stars or particles. Figure 2 shows an
Figure 1: Photorealistic rendering of a black-hole merger
created using our Spiegel-Maya interface called Miegel.
example rendering created using Spiegel’s basic ren-
dering capabilities. We created Miegel, a software in-
terface that allows Spiegel to take advantage of the
rendering and animation capabilities of Maya. By us-
ing our system, the user is able to create more realistic
and appealing visualizations such as the one shown in
Figure 1.
719
Ogden J., Jordan J., Krol C., Papazian T., Bischof H. and Bailey R..
PHOTOREALISTIC RENDERING OF SCIENTIFIC DATA.
DOI: 10.5220/0003851207190724
In Proceedings of the International Conference on Computer Graphics Theory and Applications (IVAPP-2012), pages 719-724
ISBN: 978-989-8565-02-0
Copyright
c
2012 SCITEPRESS (Science and Technology Publications, Lda.)
Figure 2: A visualization of a black-hole merger created
using Spiegel’s basic rendering capabilities. The stars are
displayed as 3D points.
Figure 3: Overview of the Spiegel framework. Data is ex-
tracted from one or more file servers and distributed to a
cluster of computers. Each processing unit in the cluster
generates one (or more) frames of the complete visualiza-
tion sequence in parallel. These frames are then combined
to create a video. Image courtesy of (Espinal et al., 2010).
2 SPIEGEL
Spiegel is a software package that allows users with
little experience in 3D digital graphics or computer
programming to create visualizations based on raw
simulation data. Spiegel makes this possible through
a graphical user interface that allows users to com-
bine components to create their desired visualization.
Each component represents an individual program de-
signed to perform a specific task. The Spiegel system
is designed to be extensible by allowing new compo-
nents and functionality to be added to the basic frame-
work. This is possible because Spiegel is designed
using a data flow architecture. Components have
communication endpoints, which can be connected
to form a complete visualization program. When the
Figure 4: Example of a program created in Spiegel that il-
lustrates the data flow architecture. Image courtesy of (Es-
pinal et al., 2010).
program is executed, data is passed from one compo-
nent to another. Each component performs specific
operations that contribute to the final result (Espinal
et al., 2010). Communication between components
is made possible by a variety of built-in data struc-
tures (Bischof and Dong, 2011). Data is taken in via
an extractor and placed in these structures which are
then passed between components as optional param-
eters. Figure 4 shows a simple visualization program
created using Spiegel’s graphical user interface.
3 PREVIOUS WORK
Spiegel’s basic rendering capabilities are provided us-
ing OpenGL primitives such as points and lines. As
can be seen in Figure 2, the resulting renderings are
useful but not necessarily compelling or realistic.
Several efforts have been undertaken to improve
the quality of the renderings produced by the Spiegel
system. One such effort expanded the visualization
capabilities of Spiegel by using Pixar’s PhotoRealis-
tic RenderMan
R
(Cassidy et al., 2011). By using the
shader library accessible through the RenderMan
R
program, the researchers were able to apply displace-
ment shaders to better represent stars and other cos-
mic forms. They also used noise patterns and color
palettes to develop realistic models of stars. A fur-
ther advantage of RenderMan
R
was its ability to in-
corporate realistic lighting effects, which proved to
produced far more compelling visualizations.
In addition, other work has been done to study
the gaze patterns of expert and novice astrophysicists
to gain insight into what makes a favorable or com-
pelling visualization (Arita et al., 2011). By hav-
ing participants view a number of images of galactic
events, both photographed and computer generated,
the researchers were able to gain a better understand-
ing of preferred camera positions and zoom settings.
This information was then used to modify existing
Spiegel visualization programs to create more favor-
able renderings.
Despite these advances, the improved Spiegel
framework had several major drawbacks. In partic-
ular, developing custom shaders for the RenderMan
R
component required significant programming exper-
tise. Furthermore, the system still lacked the capa-
IVAPP 2012 - International Conference on Information Visualization Theory and Applications
720
bility to handle complex animations and fluid simula-
tions.
To overcome these limitations, we decided to use
Maya because of its ability to produce compelling
visualizations with minimal programming. Further-
more, Maya has a robust fluid simulation system
and the ability to store artist created presets, which
novices can easily access. This fluid system proved
to be the ideal solution for simulating space dust and
gases.
4 MAYA
Maya is a high-end 3D animation program capa-
ble of producing stunning visual graphics. A major
shortcoming is that it takes a considerable amount of
time and training in order for an individual to pro-
duce these images. Through the Miegel (i.e. Maya-
Spiegel) interface that we created, the user can take
preset files created by an expert in the use of Maya
and insert them into the scene with relative ease. Us-
ing the controls provided in the Miegel interface, the
user can also customize the visualization by simply
adjusting various attributes.
In order to produce the visual effects of space
dust and gases, we utilized Maya’s fluid simulation
system. Through the careful application of noise at-
tributes and color gradients we were able to construct
cloud shapes that were much like those seen in galac-
tic imagery. These settings were then saved as presets,
which allow for faster render times and greater con-
trol. These types of presets can easily be loaded and
adjusted by a novice user to quickly create appealing
visualizations.
Another powerful feature that Maya provides is
the ability to map an image onto a fluid. This is done
by using the color values of the image to represent
specific densities in the fluid. Maya’s fluid simulator
can then use this information to produce the effect of
vortices and cloud formations (Brinsmead, 2007). We
were able to add further detail and realism to the fluid
simulation through the use of photographs of various
cloud formations made available through the National
Oceanic and Atmospheric Administration, as well as
other open source venues. This same technique can be
applied for any photographed astrophysical phenom-
ena, allowing for an added degree of dimensionality
to the visualization. By utilized Maya’s wide array of
physically accurate modifiers, we were able to manip-
ulate the fluid as if it were acted upon by forces found
in space.
5 MIEGEL
Miegel is a software interface designed to integrate
Maya’s rendering capabilities with Spiegel. Miegel
was created because the existing solutions provided
by OpenGL and RenderMan
R
were too complicated
for novices to use since they required significant pro-
gramming experience. The process of adding new
types of effects to the visualization pipeline, for ex-
ample, required many hours of implementation, test-
ing, and debugging. Miegel accomplished this by us-
ing the options available through Maya to produce
stunning visualizations, which in tern require less pro-
gramming hours. In addition, Miegel aimed to allow
for visuals produced by a 3D graphics artist to be eas-
ily integrated into the visualization. With Miegel, the
task of creating visual effects such as those seen in
professional productions as well as the process of in-
tegrating them into the visualization is greatly sim-
plified. This is done by the user interface in Miegel,
which allows for visuals produced by a 3D graphics
artist in Maya to be easily integrated into the visual-
ization through a drop-down menu. The applicable
attributes of the specific preset are then displayed for
the user to modify, giving the user full control over
the aesthetics of the finished rendered image.
Miegel works by producing a MEL (Maya Em-
bedded Language) script and a data cache PDC (Parti-
cle Disk Cache) and passing them to Maya. The MEL
script contains information about the objects Maya
should create, such as color and luminance values for
the given particles, as well as scripts to import addi-
tional scene files. The PDC is a raw file format used
by Maya as a way of caching information about the
particles. The cache maps attribute information about
particles to disk allowing for faster drawing of par-
ticles, versus having to re-calculate the positions for
each particle for each frame (Center, 2005). The PDC
contains the vector data for the particles that repre-
sent the stars in the simulation. Maya can then ren-
der this out into a batch of image files, which can be
at full HD resolution (in contrast with the previous
solution which could only produce standard resolu-
tion images). The big advantage with this system is
that Miegel automates most of the process and allows
users unfamiliar with Maya to still take advantage of
its powerful capabilities.
Figure 5 illustrates the Miegel architecture. A
MEL script is a language provided by Maya that
allows users to create macro programs to automate
many of Maya’s functions and operations. Because
the Maya GUI is itself written in MEL, any operation
that can be performed by a human using Maya can
also be programmed in MEL. This means that a pro-
PHOTOREALISTIC RENDERING OF SCIENTIFIC DATA
721
gram like Miegel that generates a MEL script can
perform any action in Maya that is available to a
user. The simplicity of MEL scripts mean that Miegel
can easily be expanded to utilize any of Maya’s fea-
tures (Autodesk, 2010).
An example of a MEL script is
particle n star p 0 0 0;
setAttr star.prt 8;
setAttr particleCloud1.incandescence
type double3 1 1 1;
setAttr particleCloud1.glowIntensity 1;
These few lines instruct Maya to place a single
particle named star at vector (0, 0, 0), convert the type
of the particle to a cloud, then change the incandes-
cence color to white with maximum glow. Once this
single particle is created, it can be attached to the PDC
that contains all of the particles (potentially tens of
thousands) and an image that matches the data in the
simulation can be rendered. Scripts such as the one
shown above are automatically generated by Miegel
and utilized by Maya to produce the resulting image.
Spiegel utilizes simulation files with the exten-
sion .sim. A sim file is a human readable file writ-
ten in plain text, which contains information such as
the number of stars in the simulation, the positions
of the stars for each frame, as well as their velocity,
acceleration and other information. Miegel creates a
disk cache first be parsing out information from the
sim file. For the purpose of the visualization, we only
cared for the positions and frame number of the stars.
To convert the .sim to PDC, Miegel makes two passes
over the .sim file. First, the positions of the stars and
their frame number are extracted, at the same time
filling any gaps in the data. This information is then
written as raw bytes into an .msf (Maya-Spiegel For-
mat) file. From here, Miegel can quickly extract the
data for any particular star in file from any frame. The
benefits of having the .msf file is that the .sim file
only has to be parsed once and any subsequent vi-
sualizations using the same data does not have to be
run through the converter again. For reference, a 7GB
.sim file took over an hour to be convert to msf. Other
benefits of the .msf format include a much smaller
file size and predictable data placement allowing for
close to real-time access of information, this allows
for frames deep in the simulation to be quickly ex-
tracted and used for the visualization.
After the MEL script is generated and the PDC
completed, Maya can read the script and output a
Maya scene file (.ma) which holds all the information
needed by the renderer. The .ma file is the standard
file format used by Maya that can be read and edited,
allowing Maya users to verify the visualization before
Miegel
Visualization
Attributes
Simulation Data
Interpreter
Maya Scene Files
Maya 3D
Maya Renderer
MEL Script
Generator
Particle Disk
Cache Creator
Maya Scene File
Spiegel
Visualization
Settings
.sim to .msf
Convertor
Maya Effects
Importer
Figure 5: Architecture of Miegel. The Simulation Data In-
terpreter takes the position data of the stars, and creates the
Particle Disk Cache, which is needed for Maya to map the
location of the stars to the particles objects. The visual-
ization attribute module takes in information from the user
about the desired appearance of the stars (including color,
size, and glow intensity) and produces a MEL script. The
script can also include instructions to integrate other Maya
files into the visualization. Once the script is generated, it
is executed by Maya to produce the individual frames of the
visualization.
rendering and import other Maya scene files. The .ma
file is read by the Maya renderer to produce the final
images. The render can be flagged with arguments to
change the attributes of the outputted file, such as res-
olution, and what rendering techniques it should use,
such as anti-aliasing. This provides other advantages
over the OpenGL and RenderMan
R
solutions, where
such features must be explicitly programmed.
Spiegel components were created to allow for easy
access to Miegel from Spiegel. A user familiar with
Spiegel can easily use the new Miegel component to
access Maya. The user can import the required simu-
lation file, then choose which frames in the simulation
to render. Furthermore, they can import as many 3rd
party Maya scenes that are available, having access to
IVAPP 2012 - International Conference on Information Visualization Theory and Applications
722
Miegel
Visualization
Attributes
Simulation Data
Interpreter
Maya Scene Files
Maya 3D
Maya Renderer
MEL Script
Generator
Particle Disk Cache
Creator
Maya Scene File
Maya 3D
Simulation.msf3
Extractor
Clock
Miegel
Star Attributes
(Colors, size, etc.)
Space Cloud
Effects.ma
Background
Stars.ma
Maya Renderer
Start, Stop, FPS
Rendering
Settings
Maya File
Importer
Figure 6: : The Spiegel components available to the user
and how they interact to produce visualization.
easy to use sliders and drop down boxes to choose at-
tributes, which change the look of the stars as well as
the image output format, location, resolution, as well
as other renderer attributes. Figure 6 illustrates how
the information flows through Spiegel into Miegel and
the renderer.
Figure 7: Maya viewport view of the simulation.
6 CONCLUSIONS AND FUTURE
WORK
We have created a highly customizable and robust
program called Miegel that is able to take simulation
data in its raw form from Spiegel and send it to Maya
to generate compelling visualizations. This allows the
user to utilize the full gambit of Maya’s capabilities
in Spiegel in order to further enhance their visual-
ization. With Miegel, Spiegel could be expanded to
use professional grade animation programs to better
its ability to create realistic and eye catching simula-
tions. This provides the ability to add numerous spe-
cial effects and/or programs to Spiegel in the future.
In future work it would be possible for the user to use
the integrated camera capabilities, allowing for nearly
seamless incorporation of stereoscopic cinematogra-
phy into scientific simulations. Maya also has soni-
fication capabilities, allowing the user to coordinate
sound with the aesthetics of the visualization, which
would greatly enhance the viewing experience. Maya
also allows the use of it’s diverse library of shaders
and the ability to create customizable shaders, giving
even greater customization to the aesthetics of scene.
In addition, Maya’s newest release has the capability
to render images using a GPU based rendering system
that has the capability to display these images in the
viewport in their rendered state. Figure 7 shows the
viewport view in Maya of a simulation that was gener-
ated from our program Miegel. It is entirely feasible
to integrate other animation and rendering programs
into Spiegel, which will better enhance the simula-
tions that are produced. The framework we have cho-
sen allows for the addition of any preset the user may
require, giving the program an endless potential for
expansion that is only limited to one’s imagination.
ACKNOWLEDGEMENTS
This material is based upon work supported by
the National Science Foundation under Award No.
0851743. Any opinions, findings, and conclusions or
recommendations expressed in this material are those
of the authors and not necessarily reflect the views of
the National Science Foundation.
REFERENCES
Arita, J., Feliz, J., Rodriguez, D., Bischof, H. P., Rege,
M., and Bailey, R. (2011). Creating audiencespecific
galactic simulations using eye-tracking technology. In
in proceedings of International Conference on Infor-
mation Visualization Theory and Applications.
Autodesk, I. (2010). Maya api reference.
http://download.autodesk.com/us/maya/2009help/API
/main.htm.
Bischof, H.-P., Dale, E., and Peterson, T. (2006). Spiegel
- a visualization framework for large and small scale
systems. In Arabnia, H. R., editor, Proceedings of
the 2006 International Conference on Modeling, Sim-
ulation and Visualization Methods, pages 199–205.
CSREA Press.
Bischof, H.-P. and Dong, A. (2011). Directing a visual-
ization ala kubrick. In Proceedings of The 2011 In-
ternational Conference on Modeling, Simulation and
Visualization Methods.
Brinsmead, D. (2007). Volume displace-
ment method for clouds using fluids.
http://area.autodesk.com/blogs/duncan/volume displ
acement method for clouds using fluids.
PHOTOREALISTIC RENDERING OF SCIENTIFIC DATA
723
Cassidy, P., Kilburn, T., Salemink, V., Bailey, R., and
Bischof, H.-P. (2011). Improving the visualization of
galactic events using pixars renderman. In Poster Pa-
pers Proceedings of 19th International Conference on
Computer Graphics, Visualization, and Computer Vi-
sion, 2011.
Center, S. D. S. (2005). Maya pdc format simplified.
http://www.sdsc.edu/us/visservices/software/maya/pd
c table.html.
Espinal, J., Allen, V., Amable, K., Bailey, R., and Bischof,
H.-P. (2010). Renderman’s power to visualization’s
rescue. In Communication Papers Proceedings of
18th International Conference on Computer Graph-
ics, pages 243–249.
IVAPP 2012 - International Conference on Information Visualization Theory and Applications
724