Programming Support for the Flexible Coupling of Distributed Software

Components for Scientiﬁc Simulations

Michael Hofmann

, Felix Ospald

, Hansjörg Schmidt

and Rolf Springer

Department of Computer Science, Chemnitz University of Technology, Chemnitz, Germany

Department of Mathematics, Chemnitz University of Technology, Chemnitz, Germany

Keywords:

Distributed Simulations, Data Coupling, Parallel Computing.

Abstract:

In this article, we investigate the ﬂexible coupling of distributed software components that are required for

an optimization process of lightweight structures built from hybrid materials. The software components in-

clude computationally intensive applications for the simulation of hybrid structures, control applications for

implementing the optimization process as well as data-oriented applications for the generation, management,

and visualization of simulation data. The participating software components and application programs are de-

scribed to demonstrate their strongly varying functionalities as well as the diversity of data exchange methods

that need to be considered for the data coupling. Furthermore, we present the design and usage of a soft-

ware library with transparent data coupling mechanisms for software components that are ﬂexibly distributed

among different computing resources.

1 INTRODUCTION

The coupling of scientiﬁc simulations is an increas-

ingly popular method for the simulation of com-

plex dynamic physical problems. Especially, strongly

inter-disciplinary areas, such as climate research or

mechanical engineering, often lead to simulation

models that are independently developed, but need to

be integrated in larger simulation environments later.

Technical approaches for the required model cou-

pling include dedicated toolkits (Larson et al., 2005),

domain-speciﬁc languages (Bulatewicz, 2006; Kohn

et al., 2000), and generalized interfaces (Gregersen

et al., 2007). By focusing on the data exchange,

model-independent software components, such as for

the generation, management, or visualization of data

can also be integrated. However, the variety of ap-

plications and their ﬂexible execution in a distributed

environment pose a challenge to the data coupling.

In this article, we investigate the requirements to-

wards an efﬁcient programming support for the data

coupling of scientiﬁc simulations that are executed in

a distributed computing environment. As an appli-

cation example, the optimized design of lightweight

structures within the research project MERGE

considered. The optimization process developed in-

cludes both the simulation of the manufacturing of

MERGE Technologies for Multifunctional Lightweight

Structures, http://www.tu-chemnitz.de/merge

hybrid structures and the calculation of their operat-

ing load cases. These computationally intensive ap-

plications are executed several times during the op-

timization process, thus requiring the usage of high-

performance computing platforms. Data coupling is

required between the simulation applications as well

as with additional applications that implement, for ex-

ample, the optimization method, an interactive user

control and a central data repository. Depending on

the optimization problem as well as on the required

computing resources, a ﬂexible execution and distri-

bution of the applications is required, thus leading to

different programming efforts for their data coupling.

For example, the applications have to be loosely cou-

pled (e. g., through network communications) when

executed in a distributed computing environment or

tightly coupled (e. g., through direct function calls)

when executed on a single compute node. We present

the design of a software library that supports a trans-

parent data coupling of such ﬂexible distributed ap-

plications and show a usage example to demonstrate

our approach.

The rest of this article is organized as follows.

Section 2 describes the distributed software compo-

nents for the optimization of hybrid structures. Sec-

tion 3 presents the design and usage of the software

library. Section 4 discusses related work and Sect. 5

concludes the article.

506

Hofmann M., Ospald F., Schmidt H. and Springer R..

Programming Support for the Flexible Coupling of Distributed Software Components for Scientiﬁc Simulations.

DOI: 10.5220/0005105405060511

In Proceedings of the 9th International Conference on Software Engineering and Applications (ICSOFT-EA-2014), pages 506-511

ISBN: 978-989-758-036-9

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

2 DISTRIBUTED SOFTWARE

COMPONENTS FOR

OPTIMIZING HYBRID

STRUCTURES

The distributed simulation and optimization of hy-

brid structures is used as an application example that

requires a ﬂexible coupling of a variety of software

components. In the following, the optimization pro-

cess and its software components are described.

2.1 Optimization of Hybrid Structures

The goal of the optimization process is the optimiza-

tion of manufacturing parameters for hybrid struc-

tures. Figure 1 gives an overview of the optimiza-

tion process. Conﬁguring and executing optimization

problems will be performing through a user interface,

which will also be used to manage the input and out-

put data of optimization runs. Each optimization run

starts with an optimization problem that speciﬁes the

geometry of the structure, the objective function and

constraints for the optimization, and parameters, for

example, about the materials, the manufacturing pro-

cess, and the operating load cases. The optimiza-

tion method is implemented within a separate soft-

ware component that executes an optimization loop.

In each iteration, the optimization method selects spe-

ciﬁc values for the parameters to be optimized, starts

the simulations, and evaluates the objective function

to decide whether the optimization is ﬁnished or not.

The manufacturing by injection molding is simulated

with a computational ﬂuid dynamics (CFD) applica-

tion (see Sect. 2.2). The cooling of the structure and

the operating load cases are simulated with a ﬁnite el-

ement method (FEM) application (see Sect. 2.3).

Figure 2 shows an injection molding simulation

where the material is injected into the lower left face

and the structure is about half ﬁlled. The test case rep-

resents an existing injection molding tool and will be

used to compare the simulation and optimization re-

sults with manufactured parts. Manufacturing param-

eters to be optimized include the position, diameter,

angle, and ﬁll rate for injecting the material.

2.2 CFD Simulation of the

Manufacturing Process

The hybrid structures will be manufactured by in-

jection molding, which represents one of the most

economically important processes for the mass-

production of plastic products. The parts are pro-

duced by injecting molten plastic into a mold, fol-

Figure 1: Overview of the optimization process.

Figure 2: Illustration of the injection molding simulation

with resulting ﬁber orientation details.

lowed by a cooling process. Fillers, such as glass and

carbon ﬁbers are mixed in to improve the mechanical

properties of the parts. The injection molding process

is simulated with a customized open-source CFD ap-

plication based on OpenFOAM, a C++ library imple-

menting the ﬁnite volume method (Jasak et al., 2007)

parallelized with MPI. However, our general opti-

mization process will also be capable of employing

alternative simulations, for example, with specialized

closed-source applications (Niedziela et al., 2013).

Our OpenFOAM solver features a compressible

two phase ﬂow with heat transfer and ﬁber orienta-

tion calculation. The change of the ﬁber distribution

caused by the ﬂow is modelled by the Folgar-Tucker

Equation (Tucker and Advani, 1994). Input data of

the simulation are the structure geometry, the mate-

rial properties, and the parameters of the injection

molding process. The simulation is complete when

the mold is ﬁlled. Simulation results are the ﬁber ori-

ProgrammingSupportfortheFlexibleCouplingofDistributedSoftwareComponentsforScientificSimulations

507

entation ﬁeld and the temperature ﬁeld, which are the

input data for the FEM simulation (see Sect. 2.3).

2.3 FEM Simulation of Cooling and

Load Cases

An FEM application simulates the cooling of the

structure and the operating load cases. The cooling

leads to residual stresses within the material and to a

shrinking of the structure. The load case simulation

results in displacements and stresses of the structure.

Both simulations are performed with an in-house

adaptive 3D FEM application (Beuchler et al., 2001)

parallelized with OpenMP. However, our general opti-

mization process will also be capable of employing al-

ternative simulations with commercial closed-source

FEM applications, such as ANSYS

or Abaqus

The adaptive FEM starts with a coarse mesh,

which is adaptively reﬁned based on residual type er-

ror indicators (Verfürth, 1999). This approach leads

to high precision results, but with reduced computa-

tional costs. Input data of the simulation are the struc-

ture geometry, the material properties, and the load

case to simulate. Additionally, the ﬁber orientation

and the temperature distribution within the structure

calculated by the CFD simulation are used. Simula-

tion results are the displacement ﬁeld and the result-

ing stresses, which will be used for evaluating the ob-

jective function of the optimization process.

2.4 Optimization Objectives

An iteration of the optimization loop can be very time

consuming. Furthermore, the results of the numerical

simulations are often discontinuous and noisy with

respect to the optimization parameters and deriva-

tives are usually not available. The optimization may

have multiple objectives regarding the manufactur-

ing process (e. g., minimization of production costs)

and structure properties (e. g., maximization of the

strength, stiffness, or durability). Thus, in general,

we will have to employ high-dimensional, derivative-

free, multi-objective optimization algorithms to deter-

mine Pareto optimal solutions. These algorithms will

either be implemented separately in Python or em-

ployed through existing software frameworks, such as

OpenMDAO (Heath and Gray, 2012).

In each iteration of the optimization loop, manu-

facturing and load case simulations have to be per-

formed for several different settings of the parameters

to be optimized. The simulation runs for these param-

eter settings are independent from each other and can

http://www.ansys.com/

http://www.simulia.com/

therefore be distributed, for example, on various com-

puting resources. The number of independent simu-

lation runs depends strongly on the number of opti-

mization parameters and on the optimization method.

2.5 User Control and Data Management

In addition to the simulation applications, several

software components will be required to handle the

data involved in the optimization process. Mesh gen-

erators, such as Gmsh (Geuzaine and Remacle, 2009),

are required to model the structure geometries and to

generate the meshes for the simulations. Data reposi-

tories store the simulation data and provide a project-

oriented storage for simulation and optimization pa-

rameters. Applications, such as ParaView (Henderson

Squillacote, 2008) or VTK (Schroeder et al., 2006),

are used for the visualization of simulation results.

The usage of the optimization process will be sup-

ported by a web interface, that allows an user-friendly

conﬁguration and execution of optimization runs.

3 SIMULATION COMPONENT

AND DATA COUPLING (SCDC)

LIBRARY

The implementation of the optimization process will

be support by a communication library that eases the

coupling of different software components. In the fol-

lowing, the communication library, its programming

interface and an usage example are presented.

3.1 Requirements and Solutions

The various applications involved in the optimization

process described in Sect. 2 lead to different require-

ments towards a suitable programming support:

Variety of Software Components: The considered

software components differ strongly with regard to

their functionalities as well as their speciﬁc imple-

mentations as application programs. To unify the gen-

eral coupling of the software components, the SCDC

library enforces the following usage model: Each

software component can act as an SCDC service that

provides access to datasets. Each software compo-

nent can act as an SCDC client to access the datasets

of other software components acting as services. An

URI-based addressing scheme is used to identify sin-

gle services and datasets. The datasets of a service

are made available by data providers, which deﬁne

the speciﬁc content and functionality of a dataset.

ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications

508

This includes data providers with predeﬁned pur-

poses (e. g., for accessing the local ﬁle system or a

MySQL database) as well as generic data providers

that are conﬁgured with hook functions. Each service

can contain several data providers, which are distin-

guished through individual base paths within the URI-

based addressing scheme.

Flexible execution on Distributed Resources: The

execution of the software components should be ﬂex-

ibly distributed on various computing resources. This

means that the software components of the optimiza-

tion process should be executed, for example, either

entirely on a single compute node or separately on

dedicated computing resources that ﬁt to the different

applications (e. g., storage server, HPC cluster, web

server). The SCDC library omits the need for ex-

pensive adaptations of each software component to

a speciﬁc execution scenarios. A service uses the

same library functions to provide access to its dataset

through different data coupling mechanisms (e. g., di-

rect memory buffer access, inter-process or network

communication). Similarly, the access from a client

to a service uses the same library functions, indepen-

dent from the data coupling mechanism. Accessing

different services requires only to change the URI ad-

dress that identiﬁes the target service. All adaptations

necessary for exploiting different data coupling mech-

anisms are encapsulated in the library.

Diversity of Data Access Methods: The different

software components and their alternative realizations

(e. g., different data storage components or alternative

simulation applications) lead to a heterogeneous land-

scape of application programs with a large diversity

of supported data access methods. The SCDC library

provides a C and Python interface for the integration

into application programs which allow modiﬁcations

of their source code. With these interfaces, the data

access can be performed directly through memory

buffers provided to the library functions while at the

same time the underlying data exchange is mapped,

for example, to network communication. Addition-

ally, the Python interface is used for wrapping closed-

source (e. g., commercial) application programs that

employ usually a ﬁle-based data access. To provide

such an application as a service, a dedicated Python

program will be used for setting up the service with

the SCDC library, executing the speciﬁc application

program, and managing its input and output ﬁles.

3.2 Programming Interface and Usage

The programming interface of the SCDC library con-

sists of service and client functions:

Service Functions: A software component that

acts as a service has to specify the data coupling

mechanisms available for client accesses. Direct

coupling is enabled as default and connects all

commands executed by a client to direct function

calls within a service. Further connection-oriented

coupling (e. g., with communication based on Unix

Domain sockets or TCP sockets) can be enabled with

the scdc_nodeport_open/close/start/stop

functions. Each data coupling mechanism

has to be opened and closed separately and

can be started and stopped temporarily. The

scdc_dataprov_open/close functions are used to

add data providers to a service.

Client Functions: A software component that acts

as a client uses the scdc_dataset_open/close/cmd

functions to access the datasets provided by a ser-

vice. The following URI-based addressing schemes

are currently used to identify a dataset to be opened:

• scdc:///<path> to access datasets of the same

software component through direct coupling.

• scdc+uds://<socketname>/<path> to access

datasets of software components executed on the

same compute node through inter-process com-

munication with Unix Domain sockets. The

socket to be used is speciﬁed with <socketname>.

• scdc+tcp://<hostname>/<path> to access

datasets of software components executed on

different compute nodes through network com-

munication with TCP sockets. The compute node

of the service is speciﬁed with <hostname>.

The speciﬁcation of <path> for identifying a spe-

ciﬁc dataset is the same for all three schemes. A ﬁrst

part of <path> usually represents the base path that

selects a speciﬁc data provider of a service and the re-

maining seconds part of <path> identiﬁes a speciﬁc

dataset of this data provider. However, the speciﬁc

kind of data represented by a dataset depends on the

data provider. For example, a dataset represents a ﬁle

system directory for the data provider accessing the

local ﬁle system while it represents a database table

for the data provider accessing a MySQL database.

Usage example: Figure 3 shows an usage example

for the SCDC library with two software components

acting as a storage service A (top) and as a client with

integrated storage service B (bottom). The storage

service A is executed on a compute node hostA and

stores the data in its local ﬁle system. In lines 3–4,

a data provider of type “fs” is created that can be

accessed with the base path “store” and uses the

speciﬁed storage directory. In line 6, access to the

storage service is opened through connection-oriented

ProgrammingSupportfortheFlexibleCouplingofDistributedSoftwareComponentsforScientificSimulations

509

1 \ centering

2 /

storage service A on hostA

3 dp = d a t a prov_op e n ( " store " , " fs" ,

4 " p a t h_t o _ s tora g e _ dir e c t ory " );

6 np = n o d e port_op e n ( " tcp " );

7 node p o r t _ s tart ( np , 0);

9 /

keep the service running

10 w hile ( g e t c h ar () != ’ q ’);

12 nodep o r t _ s t o p ( np );

13 node p o r t _ c lose ( np );

15 data p r o v _ c lose ( dp );

14 /

client & db storage service B

15 dp = d a t a p rov_ope n ( " st o re " , " mysq l " ,

16 " dbserver : d buser : dbpasswd : d bname " );

18 da t a s e t_in p u t _ cre a t e (& in , " b u ffer " ,

19 buf , buf_size );

21 if ( db s t o re ) u ri = " sc dc :/// s t ore " ;

22 else uri = " sc dc + tc p :// h ostA / sto r e " ;

24 /

open dataset & store data ’XY’

25 ds = d a t a s e t_open ( uri );

26 dataset_ c m d ( ds , " put XY " , &in , NU L L );

27 datas e t _ c l o s e ( ds );

29 da t a s e t_i n p u t _des t r o y (& in );

31 data p r o v _ c lose ( dp );

Figure 3: Usage example for the SCDC library with two

software components, i. e. storage service A (top) and client

with integrated storage service B (bottom). The preﬁx

scdc_ of library functions is omitted due to space reasons.

coupling with TCP sockets. The corresponding TCP

server is started in a non-blocking way (line 7). Thus,

it is necessary to keep the main thread of the service

running and to provide an appropriate mechanism for

terminating the service (line 10). Finally, the TCP

server and the data provider are stopped (lines 12–15).

The client with integrated storage service B shown

in Fig. 3 (bottom) stores the data within a MySQL

database. Thus, in lines 15–16, a data provider of

type “mysql” is created by specifying the required

database access credentials. The client stores data

from a memory buffer either with storage service A

or B. In lines 18–19, an input data object in is pre-

pared such that it reads from the memory buffer buf.

The selection between storage service A and B is

achieved by a differing URI address for the dataset of

the storage operation (lines 21–22). The storage op-

eration itself is performed by opening the correspond-

ing dataset (line 25) and performing a put command

to store the data from the input data object (line 26).

Since there is no output expected for the command,

the usage of an output data object is omitted. The

handle ds of the opened dataset might be used to per-

form several commands before it is closed (line 27).

Finally, the input data object is destroyed and the

data provider representing the MySQL database is

stopped. This usage example demonstrates how a

software component can access various alternative

software components without adaptations to its im-

plementation. Furthermore, the different data cou-

pling mechanisms (i. e., direct memory buffer access

or network communication with TCP sockets) are

transparently hidden within the SCDC library.

4 RELATED WORK

Environmental research is one of the most prominent

areas for the coupling of simulation models, as it in-

volves a variety of models from disciplines, such as

atmospheric sciences, hydrology, geology, chemistry,

and ecology. The resulting need for interoperable

software components has led to a large number of ar-

chitectures, frameworks, and toolkits speciﬁcally de-

signed to support the coupling of simulation models.

Established approaches for high performance com-

puting are, for example, the Earth System Modelling

Framework (Hill et al., 2004), the Common Compo-

nent Architecture (Bernholdt et al., 2006), and the

Model Coupling Toolkit (Larson et al., 2005). How-

ever, a proper characterization of the approaches and

their roles for model coupling is very hard due to their

broad functionalities, various implementations, and

supportive tools. A more detailed overview of the ap-

proaches and their features is given by Jagers (Jagers,

2010) and Dunlap et al. (Dunlap et al., 2013).

Coupling via an I/O infrastructure represents a

low-level alternative to model coupling. Specialized

communication libraries, such as PSMILe from the

OASIS framework (Redler et al., 2010) or the parallel

coupler PALM (Piacentini et al., 2011), provide func-

tions for the data exchange between parallel software

components. Since these functions are mapped to

MPI operations, the data exchange is restricted to MPI

programs in a uniform HPC software environment.

The Typed Data Transfer (TDT) library (Linstead,

2004) circumvents this limitation by using ﬁles and

sockets as alternative data exchange methods next to

MPI. The SCDC library provides a similar encapsu-

lation of different data exchange methods. However,

the focus on ﬂexible distributed software components

requires to support intra-process, inter-process, and

ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications

510

network communication. Furthermore, the existing

approaches usually provide two-sided communica-

tion operations that need to be actively invoked by

each software component. Instead, the SCDC library

provides an infrastructure for implementing clients

and services where commands, such as put and get,

represent remote data accesses.

5 CONCLUSION

We have described the distributed software compo-

nents required for the simulation and optimization of

lightweight structures. Especially the data coupling

between HPC simulation codes and data management

applications has to be improved to achieve an au-

tomated optimization process. The software library

proposed will support the development by replacing

the common ﬁle-based data exchange of these ap-

plications. By encapsulating different data exchange

mechanisms into the library, a ﬂexible distribution of

the software components among different computing

resources will be achieved without requiring expen-

sive adaptations to the application codes.

ACKNOWLEDGEMENT

This work was performed within the Federal Clus-

ter of Excellence EXC 1075 “MERGE Technologies

for Multifunctional Lightweight Structures” and sup-

ported by the German Research Foundation (DFG).

Financial support is gratefully acknowledged.

REFERENCES

Bernholdt, D., Allan, B., Armstrong, R., Bertrand, F., Chiu,

K., Dahlgren, T., Damevski, K., Elwasif, W., Ep-

perly, T., Govindaraju, M., Katz, D., Kohl, J., Krish-

nan, M., Kumfert, G., Larson, J., Lefantzi, S., Lewis,

M., Malony, A., Mclnnes, L., Nieplocha, J., Norris,

B., Parker, S., Ray, J., Shende, S., Windus, T., and

Zhou, S. (2006). A component architecture for high-

performance scientiﬁc computing. Int. J. High Perfor-

mance Computing Applications, 20(2):163–202.

Beuchler, S., Meyer, A., and Pester, M. (2001). SPC-

PM3AdH v1.0 - Programmer’s manual. Preprint

SFB/393 01-08, TU-Chemnitz.

Bulatewicz, T. (2006). A domain-speciﬁc language for

model coupling. In Proc. of the Winter Simulation

Conf., pages 1091–1100. IEEE.

Dunlap, R., Rugaber, S., and Mark, L. (2013). A feature

model of coupling technologies for earth system mod-

els. Computers & Geosciences, 53:13–20.

Geuzaine, C. and Remacle, J.-F. (2009). Gmsh: A 3-D

ﬁnite element mesh generator with built-in pre- and

post-processing facilities. Int. J. Numerical Methods

in Engineering, 79(11):1309–1331.

Gregersen, J., Gijsbers, P., and Westen, S. (2007). OpenMI:

Open modelling interface. J. Hydroinformatics,

9(3):175–191.

Heath, C. and Gray, J. (2012). OpenMDAO: Framework

for ﬂexible multidisciplinary design, analysis and op-

timization methods. In Proc. of the 8th AIAA Multi-

disciplinary Design Optimization Specialist Conf.

Henderson Squillacote, A. (2008). The ParaView guide: A

parallel visualization application. Kitware.

Hill, C., DeLuca, C., Balaji, V., Suarez, M., and da Silva,

A. (2004). The architecture of the earth system model-

ing framework. Computing in Science & Engineering,

6(1):18–28.

Jagers, H. (2010). Linking data, models and tools: An

overview. In Proc. of the Int. Congress on Environ-

mental Modelling and Software (iEMSs’10).

Jasak, H., Jemcov, A., and Tukovic, Z. (2007). Open-

FOAM: A C++ library for complex physics simula-

tions. In Proc. of the Int. Workshop on Coupled Meth-

ods in Numerical Dynamics (CMND’07).

Kohn, S., Kumfert, G., Painter, J., and Ribbens, C. (2000).

Divorcing language dependencies from a scientiﬁc

software library. In Proc. of the 10th SIAM Conf. on

Parallel Processing for Scientiﬁc Computing. SIAM.

Larson, J., Jacob, R., and Ong, E. (2005). The Model Cou-

pling Toolkit: A new Fortran90 toolkit for building

multiphysics parallel coupled models. Int. J. High Per-

formance Computing Applications, 19(3):277–292.

Linstead, C. (2004). Typed Data Transfer (TDT) user’s

guide.

Niedziela, D., Tröltzsch, J., Latz, A., and Kroll, L. (2013).

On the numerical simulation of injection molding pro-

cesses with integrated textile ﬁber reinforcements. J.

Thermoplastic Composite Materials, 26(1):74–90.

Piacentini, A., Morel, T., Thévenin, A., and Duchaine, F.

(2011). O-PALM: An open source dynamic parallel

coupler. In Proc. of the IV Int. Conf. on Computa-

tional Methods for Coupled Problems in Science and

Engineering.

Redler, R., Valcke, S., and Ritzdorf, H. (2010). OASIS4

– A coupling software for next generation earth sys-

tem modelling. Geoscientiﬁc Model Development,

3(1):87–104.

Schroeder, W., Martin, K., and Lorensen, B. (2006). The

Visualization Toolkit: An Object-oriented Approach

to 3D Graphics. Kitware.

Tucker, C. and Advani, S. (1994). Processing of short-ﬁber

systems. Flow and Rheology in Polymer Composites

Manufacturing, pages 147–147.

Verfürth, R. (1999). A review of a posteriori error es-

timation techniques for elasticity problems. Com-

puter Methods in Applied Mechanics and Engineer-

ing, 176(1–4):419–440.

ProgrammingSupportfortheFlexibleCouplingofDistributedSoftwareComponentsforScientificSimulations

511