REAL TIME SIMULATION
OF DEFORMABLE OBJECTS WITH FORCE FEEDBACK
Application to surgery simulation
Moulay Brahim El Khalil Ghembaza, Karim Djouani and Yacine Amirat
LISSI Laboratory, Paris 12 University, 120-122 rue Paul Armangot, 94400 Vitry-sur-Seine, France
Keywords: Medical simulators, Physically-based models, Deformable objects, Virtual reality, Continuous collision
detection, Haptic interaction.
Abstract: This paper presents some issues in the simulation of deformable objects with force feedback. It presents an
overview of our approach for the conception of a virtual reality medical simulator. We describe a new base
finite element method (Extended Tensor-Mass Model) suitable for soft tissue simulation under real time
constraints. Our approach allows fast computation of non-linear and viscoelastic mechanical deformations
and forces. As far as real-time interactions are concerned, we present our work on collision detection and
haptic interaction. Thus, for contact management, a continuous collision detection method based on cubic
spline parametric approximation is proposed. Finally, interactive endovascular simulator is described.
1 INTRODUCTION
The variety and the complexity of medicine have
made it for a long time force of progress for many
scientific and technical fields. The medical domain
is among the main application areas for numerical
imaging and vision since their beginnings. In
parallel, the graphical tools, computer science and
robotics have become central for modern medicine.
These assistance tools are part of what is called
Surgetics (a new generation of Computer- and
Robot-Assisted Surgery systems). The surgery
simulation, which constitutes actually an active
research field, concerns some practical tool designs
that allow to offer experts at the same time the
possibility to practise intensive training for operative
gestures, unrestricted by ethical problems, and the
ability to plan with precision some interventions and
surgical procedures.
During the 1990s, a great interest for medical
procedures simulation has been developed. The
earlier simulators had been developed for navigation
within 3d-anatomical data bases and found many
applications in education and training. These
simulators used only geometrical models for the
anatomical structures, without taking into account
their physical reality.
Therefore, new simulators have been proposed in
order to overcome the drawbacks cited above, by
using more realistic physical modelling of the
various anatomical structures and their interactions
(Moline, 1997), (Gross, 1999). Taking into account
the physical phenomena should not only allow to
improve quality of the medical simulators, but also
to widen considerably their field of application.
The main components of an advanced surgery
simulation environment can be summarised by the
following elements:
1. High precision of 3d-data acquisition is
accomplished by medical imaging and/or vision
systems. A pre-treatment extracts the anatomical
structures and creates geometrical models;
2. Accuracy of deformation modelling of the
biological tissues and surgical tools, allowing the
surgeon to modify geometry and topology of
various Virtual Environment “VE” objects for
incisions simulating, tissues repositioning,
transplantation, cutting, perforation, etc;
3. Continuous collision detection algorithm
between deformable and/or rigid VE objects and
contact-friction management;
4. Realism by haptic rendering synthesis (contact
efforts of medical tools and biological tissues),
and real-time insuring (Parallel computing),
using adequate interaction devices to feel the
haptic sensations met in a conventional surgery
are required;
5. Specific problems for some interventions:
assistance by Virtual Fixtures during navigation
310
Brahim El Khalil Ghembaza M., Djouani K. and Amirat Y. (2005).
REAL TIME SIMULATION OF DEFORMABLE OBJECTS WITH FORCE FEEDBACK - Application to surgery simulation.
In Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Robotics and Automation, pages 310-315
DOI: 10.5220/0001185203100315
Copyright
c
 SciTePress

of tools in hollow bodies, in order to improve the
precision and the security of the operative
gesture, particularly at the time of guidance and
placement of the surgical tools.
Thus, the design of a surgery simulation
environment requires finding a compromise between
complexity of the adopted models and fast
calculation of the algorithms.
The search for an efficient method for real-time
computation of nonlinear elastic mechanical
deformations is only at its beginning. We can
distinguish two main communities. On the one hand
the biomechanics community is interested into the
precise characterization of the behaviour laws of
certain biological tissues, but without being
concerned with computation time. On the other
hand, the computer graphics expert’s community is
committed to the development of deformable object
simulators for biomedical applications, but adopts
very simple mechanical models, in generally linear
elastic ones, and without being concerned with
matching these mechanical models to the
experimental behaviour of biological tissues.
Recently, some approaches have been proposed
(Liu et al., 2004), (Schwartz et al., 2004) in order to
take into account the requirements of the two
communities cited above: biomechanical modelling
of soft tissues under real-time constraint. Our work
joints the same point of view while aiming at
proposing an approach which is sufficiently simple
and quick to be compatible with real-time
applications. We try to reproduce as exactly as
possible the real biological tissue behaviour obtained
by biomechanical experimentation.
The present paper is organized as follows. In
section 2, dynamic simulation of biological tissues
has been addressed and a new approach is presented
for deformation and forces modelling. In section 3,
we lay out how we handle some aspects of real-time
interaction that are inevitable in current medical
simulators such as collision detection and haptic
interaction. In section 4, interactive endovascular
procedures are described and our simulator
workbench is presented. Finally, some conclusions
and perspectives are given.
2 DYNAMIC SIMULATION OF
SOFT TISSUE
Modelling soft tissues consists of formulating
constitutive equations related to their deformation. A
survey of deformable object modelling was done by
(Gibson and Mirtich, 1997). In brief, they divided
the work done on deformable objects into two parts:
non-physically based models and physically based
models. Physically based models can further be
divided into discrete object models and other models
based on continuum mechanics. The latter is usually
solved using the finite element method (FEM).
In advanced simulation environments, accurate
process modelling at the geometrical, physical and
physiological level is required. The biological
tissue’s mechanical deformation modelling
constitutes an essential part of surgery simulation
(Delingette, 1998). It runs up against two
fundamental and antagonistic constraints: on one
hand the realism of numerical modelling and, on the
other hand, the computation speed.
2.1 Linear Tensor-Mass Model
The most promising approach towards real-time
computation of nonlinear viscoelastic deformation
appears to be the tensor-mass model introduced by
(Cotin et al., 2000). The tensor-mass algorithm for
linear elasticity is both time-efficient and physically
accurate. It also allows local topological changes of
mesh elements, so that simulation of cutting or
perforation is possible.
The force
)( jT
i
F applied to a summit
)( jT
i
P of
tetrahedron T
i
is defined as follow:
[]
∑
=
⋅=
3
0
)(
0
)()(
k
kTkT
T
jkjT
ii
i
i
PPKF
(1)
where
0
)(kT
i
P are the rest positions of the four
vertices of tetrahedron T
i
,
)(kT
i
P are the current
positions of the vertices, and
[
]
i
T
jk
K are 3×3 stiffness
tensors depending only on the rest geometry of
tetrahedron T
i
and on the Lamé coefficients. These
tensors can be pre-computed; therefore computation
at run-time is restricted to matrix-vector
multiplications and matrix summations. Given a
complete mesh, the total elastic force F
i
applied on a
vertex P
i
is obtained by summing the forces
contributed by all adjacent tetrahedrons of T
i
:
[]
[
]
∑
∈
⋅+⋅=
)(
00
i
PNj
jjijiiiii
PPKPPKF (2)
where
[]
ii
K is the sum of tensors
[
]
k
T
ii
K
associated with the tetrahedron adjacent to
P
i
,
[
]
ij
K is the sum of tensors
[
]
k
T
ij
K associated with
the tetrahedron adjacent to edge (i, j), and N(P
i
) is
REAL TIME SIMULATION OF DEFORMABLE OBJECTS WITH FORCE FEEDBACK - Application to surgery
simulation
311

the neighbourhood of vertex P
i
. The resulting system
has to be solved dynamically.
Based on FEM theory, we have proposed a new
approach for physically-based deformable modelling
as an extension of the linear elastic tensor-mass
method. Our approach allows fast computation of
non-linear and viscoelastic mechanical deformations
and forces. Its principle consists of pre-computing a
certain number of tensors depending on the
geometrical and mechanical characteristics of each
finite element, which are combined dynamically
during the simulation phase. The proposed method is
sufficiently generic to be applied to a large variety of
behaviours and objects, as soft biological tissue
deformation under real-time computation
requirements.
2.2 Non-linear viscoelastic Tensor-
Mass Model
We now present the non-linear and viscoelastic
Tensor-Mass Model, (Ghembaza et al., 2005), for
soft tissue simulation.
As a first step we show that adequate real-time
correction of linear elasticity parameters allows to
model different types of non-linear elastic
deformations. In our model, expression of
force
)( jT
i
F applied on vertex
)( jT
i
P within a
tetrahedral mesh element Ti is:
[] [] [] ()
∑
=
⋅++=
3
0
)(
0
)()(
)()(
k
kTkT
T
jki
T
jki
T
jkjT
ii
iii
i
PPBTATKF δµδλ
(3)
where
[
]
i
T
jk
K ,
[
]
i
T
jk
A and
[
]
i
T
jk
B are 3×3 tensors, λ
i
and µ
i
are the Lamé coefficients of the material, and
δλ
i
and δµ
i
are non-linear corrections. Tensors only
depend on the geometry at rest so that pre-
computation is possible.
The linear tensor-mass method can typically deal
with meshes of a few thousand nodes in real-time.
The non-linear extension increases computing time
by a factor 4. However the method remains suitable
for real-time applications, due to the fact that the
non-linear computational overhead is restricted to a
limited number of mesh elements where the highest
deformation rates occur (Ghembaza et al., 2005).
The tissue’s mechanical properties are defined
locally for every finite element by a stiffness tensor
associated with this element. Thus, it is possible to
build inhomogeneous models, composed of various
structures with different mechanical properties.
Viscosity can easily be introduced into the
tensor-mass model, under assumption that the
behaviour is restricted to a simple linear viscous
relation. We have introduced a viscous force that is
proportional to the deformation speed and to a
viscosity coefficient η. After discretisation into a
tetrahedral mesh, the expression obtained is very
similar to (1), except that deformation speed
replaces deformation, and a viscosity tensor replaces
the stiffness tensor. Expression of the viscous
force
v
jT
i
F
)(
applied to a vertex
)( jT
i
P of tetrahedron
T
i
is given by:
[]
∑
=
⋅=
3
0
)(
0
)(
)(
)(
k
kTkT
v
T
jk
v
jT
ii
i
i
PP
dt
d
KF
(4)
where
[]
)(v
T
jk
i
K are 3×3 tensors depending only
on the rest geometry of tetrahedron T
i
and on the
viscosity coefficient η.
We apply the complete algorithm presented in
(Ghembaza et al., 2005) to simulate dynamically the
different behaviours.
3 REAL-TIME INTERACTIONS
The surgeon’s assistance for preoperative, as well as
for intra-operative gestures, requires both a real-time
realistic visualization (soft tissue deformation
modelling) and force feeling (haptic rendering
interface).
The goal of a medical simulator is to allow real-
time interactions with realistic modelling. It is well
known that during a simulation, given any physical
model, the most difficult aspects, in terms of
computational time or updating data structures, are
collision detection, the different rates for haptic
interaction between graphical updates and physical
simulation and topology modification during
specific surgical procedures. We address these
problems in the following sections.
3.1 Contact Management
The collision detection is the most critical step in
dynamic simulation, because it requires a very
significant calculating time compared to that
necessary to the calculation of movement and
deformations of the objects. Figure 1 shows the
contact’s management procedure.
When dealing with contact characterization, the
goal is to detect if, when, and where objects collide.
To deal with this computationally demanding
problem in our simulation, we use an inspired
ICINCO 2005 - ROBOTICS AND AUTOMATION
312

continuous collision detection method (i.e. collision
detection must be able to return the first collision
time) from (Redon et al., 2002) without taking into
account of friction between a deformable and a rigid
bodies.
Figure 1: Algorithm to detect and manage all contact
elements
Indeed, one of the major drawbacks of the
discrete collision detection methods is that it can
"miss" the collisions if the speed of object is very
high. Moreover, discrete collision detection requires
backtracking methods to compute the first contact
time, which is necessary in constraint-based
analytical dynamics simulations. Continuous
collision detection methods overcome such a
problem, because they interpolate the trajectory of
every element (triangle in our case) between two
sampling time and thus calculate the minimal time
corresponding to the first collision. Very few
continuous methods have been proposed in the
literature. The approach developed by (Redon et al.
2002) is well adapted to treat the collisions between
rigid objects. We propose an extension of this
approach to treat collisions between deformable
objects, using parametric approximation method
(Cubic Spline representative the deformation
trajectory) to interpolate the mesh elements (figure
2).
An "Accelerating Hierarchy" approach
(bounding volume technique) is implemented in
order to decrease the number of "triangle-triangle
tests" and thus to increase the speed of the
algorithm. Thus, a recursive division of the space
containing the whole of the triangles, based on a
"bounding-box" algorithm (Axis-Aligned Bounding
Boxes: AABB) (van den Bergen, 1997) has been
implemented.
However, computing the collision response
requires us to evaluate the involved local
deformation of the colliding objects, using a non-
linear penalty method (Deguet et al., 1998) (Moore
et al., 1988) to ensure the separation of the objects in
collision. This can be done by determining the
fictitious interpenetration of the objects. Figure 3
shows these stages.
Figure 2: Cubic Spline based deformation trajectory
3.2 Haptic Rendering
Haptic systems gives people the sensation of
touching objects in virtual environments or
teleoperation applications. Including haptic
technology improves the perception of a surgeon
leading to a deeper sense of immersion. Many
problems arise in haptic applications especially in
the case of deformable objects manipulation, for
instance computational time, numerical instability in
the integration of the body dynamics, time delays,
etc. Lengthy computations are forbidden in haptic
systems which need high simulation rates (about 300
Hz to 1 KHz) to obtain realistic force feedback. The
update rates of the physical objects being simulated
is normally of the order of 25 to 60 Hz. This
difference in simulation rates can cause an
oscillatory behaviour in the haptic device that can
become highly unstable (
Adams et al., 1999). Several
numerical approaches (Cavusoglu et al., 2000) have
been proposed to solve the difference rate problem.
For our purpose, the objective is to develop
robust and rapid algorithms which allow haptics
feedback for deformable objects. The reaction
calculation is ensured by a compliance method
(interaction between a flexible or rigid body, the
surgical tool for instance, and deformable body as
soft biological tissues).
The reaction force (
c
F
r
) is calculated using the
minimal distance dist between the local model (soft
biological tissue) and the haptic tool.
Thus, the force vector
c
F
r
is given by:
t
Spline
A
0
C
0
B
1
B
0
A
1
C
1
REAL TIME SIMULATION OF DEFORMABLE OBJECTS WITH FORCE FEEDBACK - Application to surgery
simulation
313

⎩
⎨
⎧
<⋅⋅⋅−⋅⋅−
=
o
therwi
se
distifnnvbndistk
F
c
0
0)(
r
r
r
r
r
r
(5)
Where k is the rigidity coefficient, b is a
damping coefficient, dist is the penetration depth
between the two bodies,
v
r
is the relative linear
velocity of these two objects at collision.
n
r
is the
normal direction of contact.
4 APPLICATION
The application concerns endovascular procedure
simulation with physical deformation modelling of
the Abdominal Aorta and Aneurysm (AAA). The
latter is an arterial wall pathology involving a
permanent dilation of the abdominal aorta which can
be life-threatening. The endovascular procedure is
mainly used for the treatment of AAA. The goal of
the intervention is therefore to repair the swelling
and prevent the rupture of the aneurysm. The
prosthesis is hooked from inside the aorta into its
wall with a stent. Figure 3 shows the prosthesis
deployment process in an AAA endovascular
procedure.
Figure 3: Prosthesis deployment procedure
The endovascular treatment is a complex surgery
(Hausegger et al., 2001) which has not been deeply
investigated and no simulators are available at the
moment. The currently used catheters are passive.
They cause important frictions with the aorta,
leading to risk of damage, making the prosthesis
delivering difficult and may even lead to failure.
This phenomenon is also amplified by the lack of
tactile sensation. In order to overcome these
drawbacks, an active compliant micro-mechanism,
(figure 4), is used to help the surgeon (Djouani et al.
2002). Therefore it is interesting to build a simulator
allowing the surgeon to practice such a technique
which offers a rich environment to practice a variety
of aneurysms models and to manage different
possible complications.
Figure 4: Experimental device in scale 1
The dynamic simulator and the haptic interface
are designed as independent processes and they are
connected via a local model. Figure 5 shows the
basic visual-haptic platform. It has been
implemented using C++/OpenGL and it provides
force feedback through means of an haptic interface
of type 6 dof PHANToM. The visual-haptic
platform uses a PC - 2.8 GHz Pentium IV Intel
processor with 512 MB of RAM.
Figure 5: Haptic interaction environment
The dynamic simulator carries out the physical
simulation, collision detection and the graphical
rendering of different deformable objects in the
virtual environment. It receives the haptic position
from the PHANToM (and eventually, the distance
between the haptic point and the local model) and
sends to the haptic process the different parameters
(i.e. set of colliding facets between the deformable
virtual object and the tool) to update the local model.
This update process is repeated at a rate of 30 Hz.
For the soft biological tissues modelling, we
must deal with mechanical deformation of the aortic
aneurysm tissues under contact with the catheter.
These tissues present complex behaviours, showing
viscoelasticity and anisotropy among other things. In
(Watton et al., 2004), the authors provide rare
experimental data on the aneurysm behaviour.
As the objective is to adapt the shape of the
active catheter to the geometry of inspected vessel,
ICINCO 2005 - ROBOTICS AND AUTOMATION
314

we implement the algorithm proposed in section 3
for contact management. When an effort is sensed,
an order is reported to the surgeon so as to decrease
the interaction effort. This method is particularly
well adapted in inspections of human vascular
networks and allows, even if there are contact zones,
to strongly limit the importance of the interaction
efforts.
5 CONCLUSION
We have addressed some important issues in the
conception of a realistic virtual medical simulator.
We have presented some theoretical aspects in order
to ensure real-time computation with realistic
biomechanical modelling. Thus, we have described
two main computational aspects to deal with
deformable virtual objects simulation. The first
aspect concerns the formulation of the deformation
model that meets both fast graphics/haptics
rendering rates and actual physical law accuracy.
The second aspect concerns the contact management
in the case of deformable and/or rigid object
interaction. For continuous collision detection we
use bounding volume techniques which we believe
to be suitable for deformable objects like virtual
organs in medical simulators. Finally, for haptic
rendering, a non-linear penalty method has been
used for the reaction force computation. Based on
our approach, finally our endovascular simulator has
been presented. Complexity analysis, serial and
parallel algorithms are under study for the soft
biological tissues simulation.
REFERENCES
Adams, R., and Hannaford, B., 1999. Stable haptic
interaction with virtual environments. In IEEE
Transactions on Robotics & Automation 99.
Cavusoglu, M., and Tendick, F., 2000. Multirate
simulation for high fidelity haptic interaction with
deformable objects in virtual environments. In Int.
Conf. on Robotics and Automation ’00.
Cotin, S., Delingette, H., and Ayache, N., 2000. A Hybrid
Elastic Model for Real-Time Cutting, Deformations,
and Force Feedback for Surgery Training and
Simulation. Visual Computer, Vol. 16, No. 8, pp. 437-
452.
Deguet, A., Joukhadar, A., and Laugier, C., 1998. Models
and algorithms for the collision of rigid and
deformable bodies. In Robotics: the algorithm
perspective, Proc. of the Workshop on the Algorithmic
Foundations of Robotics. Houston, USA.
Delingette, H., 1998. Toward realistic soft tissue modeling
in medical simulation. Proc. of the IEEE Special Issue
in on Virtual and Augmented Reality in Medicine, Vol.
86, No. 3, pp. 512-523.
Djouani, K., Bailly, Y., Amirat, Y., and Francois, C.,
2002. A micro device design and control for
endovascular stent-grafts delivering in aortic aneurysm
treatments. CARS’02 Computer Assisted Radiology
and Surgery.
Ghembaza, M.B.K., Amirat, Y., Djouani, K., and Daachi,
D., 2005. Deformation Modelling for Surgery
Simulation. IEEE SMC’05 International Conference
on Systems, Man and Cybernetics. Hawaii, USA, to
appear.
Gibson, F., and Mirtich, B., 1997. A survey of deformable
models in computer graphics. Tech. Rep. MERL,
Cambridge, MA, http://www.merl.com/reports/TR97-
19/index.html.
Gross, M.H., 1999. Surgery simulation - a challenge for
graphics and vision. VMV '99 (Vision, Modeling and
Visualization Workshop). Erlangen, Germany.
Hausegger, K.A., Schedlbauer, P., Deutschmann H.A., and
Tiesenhausen, K., 2001. Complications in endoluminal
repair of abdominal aortic aneurysms, European
Journal of Radiology. 39 (1), pp. 22-33.
Liu, Y., Kerdok, A.E., and Howe, R.D., 2004. A nonlinear
finite element model of soft tissue indentation. 2
nd
International Symposium on Medical Simulation.
Springer Verlag, Boston, MA, pp. 67-76.
May, J., White, G.H., and Harris, J.P., 2001. Endoluminal
repair of abdominal aortic aneurysms - state of the art.
European Journal of Radiology. 39 (1), pp. 16-21.
Moline, J., 1997. Virtual reality for health care: a survey.
In Virtual Reality in Neuro-Psycho-Physiology. G.
Riva Editor, Ios Press, Amsterdam.
Moore, M., and Wilhems, J., 1988. Collision detection and
response for computer animation. In Proceedings of
SIGGRAPH.
Redon, S., Kheddar, A., and Coquillart, S., 2002. Fast
Continuous Collision Detection between Rigid Bodies.
EUROGRAPHICS, Vol. 21, No. 3.
Schwartz, J.M., Deniger, M., Rancourt, D., Moisan, C.,
and Laurendeau, D., 2004. Modeling Liver Tissue
Properties Using a Non-Linear Visco-Elastic Model
for Surgery Simulation. Medical Image Analysis
Journal.
van den Bergen, G., 1997. Efficient Collision Detection of
Complex Deformable Models using AABB Trees.
Journal of Graphics Tools. 2 (4), 1-13.
Watton, P.N., Hill, N.A., and Heil, M., 2004. A
Mathematical Model for the Growth of the Abdominal
Aortic Aneurysm. Biomechanics and Modeling in
Mechanobiology, Vol. 3, pp. 98-113.
REAL TIME SIMULATION OF DEFORMABLE OBJECTS WITH FORCE FEEDBACK - Application to surgery
simulation
315