Fadi Yaacoub
, Yskandar Hamam
and Antoine Abche
Université Paris-Est, ESIEE-Paris, Laboratoire A²SI, Cité Descartes, 93162 Noisy Le Grand, France
Université de Versailles SQ, Laboratoire LISV, 10/12 Avenue de l'Europe, 78140 Velizy, France
F’SATIE -Tshwane University of Technology, Private Bag X680, Pretoria 0001, South Africa
University of Balamand, Dept. of Electrical Engineering, P.O.BOX 100, Tripoli, Lebanon
Keywords: Virtual Reality, Surgical Simulation, Convex Hull, Collision Detection, Healthcare Technology, Wrist
Arthroscopy, Haptic Feedback, 3D Modeling and Visualization.
Abstract: The minimally invasive approach of arthroscopy means less pain and faster recovery time for patients
compared to open surgery. However, it implies a high difficulty of performance. In this paper, a functional
prototype of a virtual reality simulator for training wrist arthroscopic surgery is introduced. This simulator
allows medical students as well as surgeons to interact with anatomical structures by modeling and
operating on virtual objects displayed on the computer screen. A 3-D virtual representation of the bones
constituting the wrist of a patient is shown. Also, algorithms that model objects using the convex hull
approaches and simulate real time collision detection between virtual objects during the training on the
operation are presented. In addition, a force feedback device is used as a haptic interface with the computer
simulation system. This leads in the development of a low cost system that is used by trainees with the same
benefits as professional devices. In this regard, the wrist arthroscopy can be simulated and medical students
can easily acquire the system and can learn the basic skills required with safety, flexibility and less cost.
Virtual Environment (VE) provides a new
dimension of graphical simulation (Goebel, 1993). It
is described as an application that allows users to
navigate and interact with a computer-generated
three dimensional (3-D) space in real time. In this
context, Virtual Reality (VR) is not only a hardware
system but also an emerging technology that
changes the way individuals interact with computers.
Recently, medicine has entered a period of intense
technological transition driven by the need to
provide improved care at lower cost. Since, the
outcome of surgical procedures is closely related to
the skills of the surgeon, the latter should remain at a
high level of technical and professional expertise.
These skills are being developed over years of
surgical training on animals, cadavers and patients.
For surgical trainees to reach a high level, new and
alternative ways of performing surgical training are
required. In addition, the low availability and high
cost of cadaver and animal specimens for traditional
medical training and the public concern with the
inhuman treatment of animals have become another
impetus for surgeons and medical students to use
new technology in their education and their training
to gain valuable information and experience.
VR technology has opened new realms in the
practice of medicine. The graphics capabilities of
VR tools, particularly in modeling and displaying
medical data can be of great assistance in teaching,
learning, training and experimenting surgeries.
Furthermore, researchers on medical education
depend heavily on VR simulators that have become
one of the main components of changing radically
the traditional medical training and the surgical
certification scenarios (Immersion Corporation).
They allow the process of iterative learning through
assessment, evaluation, decision making and error
correction which create a much stronger learning
An important application is the arthroscopic
surgery simulation. In arthroscopy, the object is
visualized and accessed through small portals. An
optical endoscope equipped with a video camera
allows the visualization of the procedure through
one of the portals, while surgical probes and other
instruments are inserted through additional portals.
Yaacoub F., Hamam Y. and Abche A. (2009).
In Proceedings of the International Conference on Health Informatics, pages 74-81
DOI: 10.5220/0001552500740081
Arthroscopy decreases soft tissue disruption which
leads to less pain and less chance for infection.
However, it implies a high difficulty of performance
and necessitates the surgeon to acquire psychomotor
skills which are essential to become expert. On the
other hand, arthroscopy is increasingly being used in
the treatment of the hand. Wrist arthroscopy, in
particular, has proven to be extremely valuable in
both diagnosis and therapy. It is an important skill
for all hand surgeons (
Haisman et al., 2005), in exactly
the same way as shoulder and knee arthroscopy.
The skills required for arthroscopy are taught
through hands-on clinical experience. As
arthroscopy becomes a more common procedure, it
is now obvious that special trainings are necessary to
master surgical operations and guarantee
qualification of the surgeons. Different research
groups have shown significant advantage of using
medical simulation systems over existing
conventional methods that use live patients. Hence, a
VR training system to simulate wrist arthroscopic
procedures in VE is proposed in this paper. Two
main issues are addressed: the 3-D reconstruction
process and the 3-D interaction. The proposed
system provides a VE with realistic representation of
the region of interest. Based on a sequence of CT
images a realistic representation of the wrist joint is
obtained suitable for the computer simulation. Two
main components of the computer-based system
interface are illustrated: the 3-D interaction to guide
the surgical instruments and the user interface for
haptic feedback. In this context, algorithms that
model objects using the Convex Hull (CH)
approaches and simulate real time exact Collision
Detection (CD) between virtual objects during the
training on the surgical operation are presented.
Also, a force feedback device is used as a haptic
interface with the computer simulation system.
The rest of the paper is structured as follows:
section 2 reviews some of the previous surgical
simulators. The design criteria of the proposed VR
system are presented in section 3. Section 4 shows
the segmentation of the CT images and the
generation of the 3-D virtual wrist model. In section
5, algorithm to construct the CH is implemented,
then the CD problem is formulated and a linear
programming solution is obtained to test whether a
collision exists or not. A force feedback device that
is used as a haptic interface with the computer
simulation system is presented in section 6. Section
7 shows a virtual simulation of dorsal percutaneous
scaphoid fixation. Finally, conclusions are given in
section 8.
VR Surgical simulators have been developed for a
wide range of procedures. However, they are often
associated with specific involvements. The VR
simulators presented are classified based on their
applications and their relation to the organs or areas
they treat.
Many simulators are associated with laparoscopy
such as LapSim (Surgical Science). The LapSim
simulator focuses on implanting basic skills that
would be needed by the trainee towards performing
bigger procedures. The Lapmentor (Simbionix) is a
force feedback enabled laparoscopic training
simulator. Medical students can train on either the
basic skills or perform full procedures. This system
offers as well the opportunity to perform a complete
surgery. Moreover, The LASSO project (G. Szekely
et al., 2000) is an integrated development effort to
construct a laparoscopic simulation platform. The
abdominal cavity is modeled using data from the
Visible Human. Organ surface features are generated
using a combination of texture analysis/synthesis. In
addition, MIST is an endoscopic simulator where
trainees are guided through a series of exercises of
progressive complexity, enabling them to develop
the skills essential for good clinical practice and
VIST is a simulator for catheter based procedures
for angiography and interventional procedures
(Mentice). With VIST, trainees are able to practice
on many operations such as carotid, coronary, renal
and vena cava. Furthermore, VR simulations of
cystoscopy and ureteroscopy procedures are done
using the UroMentor (Simbionix). The UroMentor
has a mass of practice modules and patient profiles
that can be used to perform safe surgical procedures.
Besides, the (Simbionix) GI Mentor II simulator,
associated with colonoscopy, is an interactive
computerized simulator that provides hands-on
training in endoscopic procedures. Also, Bro-
Nielsen et al. (1999) described a PC-based
bronchoscopy simulator. In addition to realistic
visual effects, this system uses a haptic interface
designed to provide realistic force feedback during
scope insertion. The system has been expanded to
include colonoscopy and flexible sigmoidoscopy.
However, most simulators described above are
expensive to acquire and need maintenance.
Regarding arthroscopy simulators, most
developments have been for knee training (Heng et
al., 2004), the second case of arthroscopy that was
treated is the shoulder arthroscopy simulations
(Bayonat et al. 2006) and very little work has been
done for wrist arthroscopy even though the wrist is a
very important joint in the body and it handles many
activities. Thus, the problem of building an
inexpensive and practical simulator for training
medical students and treat the issue of the wrist
arthroscopy remained.
The design of the proposed computer-based
arthroscopy simulator was based on a trade-off
between medical professor’s needs and VR
limitations. Wrist arthroscopy is selected due to
several reasons:
1- Wrist arthroscopy is a frequent pathology
(study of essential nature of disease) that has been
less studied and practiced than knee and shoulder
2- A varied type of involvements and specific
surgeries can be covered by wrist arthroscopy
simulation such as: dorsal percutaneous scaphoid
fixation, volar percutaneous scaphoid fixation,
capitolunate arthrodesis ...
Two major aims are addressed:
1- Applying VR and physical simulation
techniques to generate 3-D models and to simulate
operations with fidelity and realism.
2- Trying to cover different requirements for the
apprentice learning process and providing the user
with tools to facilitate teaching, learning and training
on several experiments.
Therefore, medical images are processed to
generate volumetric object models. These 3-D
models are presented both visually via rendering on
the computer monitor and haptically with a force
feedback device. Visual parameters such as
viewpoint, zooming, color and lighting effects can
be interactively controlled and object models can be
manipulated with force feedback to change relative
probe and object positions, and to simulate many
surgical procedures. The interaction between the
haptic device and the computer closes the feedback
loop between the user and the simulator, offering a
better understanding of the anatomical structures.
Figure 1 outlines the main components of the
proposed VR simulation system.
Figure 1: A Flowchart of the Simulation System.
Segmentation subdivides an image into its
constituent parts. The watershed segmentation
(Couprie et al., 2005) has proven to be a powerful
and fast technique for both contour detection and
region based segmentation. This method allows,
from a gradient image, to find a thin separation
between the components of a given set of points
called markers. Figure 2 shows the original image
and figure 3 shows the gradient with the markers.
Figure 2: Original Image.
Figure 3: Gradient of Original Image with Markers.
HEALTHINF 2009 - International Conference on Health Informatics
The watershed algorithm is implemented by
region growing based on the set of markers to avoid
over-segmentation. At the end of the process all
minima are completely separated by dams, called
watershed lines. The watershed result is shown in
figure 4.
Figure 4: Watershed Result.
The final result of segmenting a set of CT images
is a volumetric image that represents the labeled
bones. Figure 5 shows the final 3-D image of the
Figure 5: 3-D Image of the Wrist.
After the segmentation of the CT images, the
Marching Cube algorithm is used to construct the
boundary of the objects in the scene. The algorithm
finds the appropriate surface patch in a look-up table
and builds this patch, interpolated according to the
values of the eight corners of this unit cube. The
union of all these patches constitutes the
approximated iso-surface and a list of facets is
generated (Daragon et al., 2003). Figure 6 shows a
high resolution 3-D virtual representation of the
bones constituting the wrist of a patient. This
representation provides the surgeon with precise and
detailed information of the region of interest that he
will be working on.
Figure 6: 3-D Virtual Model of the Wrist Bones.
The goal of a medical simulator is to support
medical students during training and practicing on
surgeries with high precision. In this regard, medical
objects are modeled with a tightness fit i.e. each
object is modeled by its corresponding CH. This will
give the simulator a high degree of precision but at
the same time an increase in the cost of the
complexity and the computational time for collision
check. Therefore, by taking advantages of the speed
and robustness of Linear Programming (LP)
techniques the problem of CD is formulated and
solved (Yaacoub et al, 2007). In addition, convex
objects allow the LP algorithm to converge quickly
and detect the collision if it exists. Thus, the CH of
each object is reconstructed. Then, the CD problem
is formulated as an optimization problem based on
convex objects and solved using linear programming
(simplex method).
5.1 Convex Hull Algorithm
Most exact collision detection systems work almost
exclusively with convex objects because they allow
CD algorithms to converge quickly. Moreover,
convex envelopes have less contact points than real
objects. This leads to a decrease in the size of the
system of equations needed to calculate the
collision. A new hybrid CH technique is developed
to construct the convex envelope of a 3-D medical
object (Yaacoub et al., 2006). The corresponding
pseudo-code is shown as follow:
Algorithm 1: The Convex Hull Approach.
1: find an initial plane from the min and max abscise and
the max distance with respect to (x
, x
2: construct a polyhedron from the initial plane and the
max distance to this plane
3: for each facet F of the polyhedra do
4: for each unassigned point p do
5: if p is above F then
6: assign p to F's outside set
7: end if
8: end for
9: end for
10: Discard all points inside the polyhedron forming a
input set n
11: find a starting edge (a, b) using the 2D Gift
algorithm on the XY projection
12: for i = 1 ... n
13: find point p
corresponding to min angle between
plane P in XY containing (a, b) and plane T = (a, b, p
14: replace c p
15: save (a, b, c) into Q
16: wrap the edge (a, c)
17: if facet has been explored then
18: wrap the edge (b, c)
19: if facet has been explored then
20: return
21: end if
22: end if
23: end for
As a result of applying the CH hybrid technique,
figure 7 shows different bones constituting the 3-D
wrist model: 1
Metacarpal (a), 2
Metacarpal (b),
Metacarpal (c), Scaphoid (d), Capitate (e),
Hamate (f), Radius (g) and Ulna (h). Each bone is
covered with its corresponding convex envelope.
Figure 7: Bones from the 3-D wrist model enclosed by
their corresponding CHs.
5.2 Linear Programming Solution
To formulate the problem, each facet i of the convex
envelope is represented by the plane inequality in
the form of:
Any point lying on the object must satisfy the
inequalities of the planes constituting the object.
These equations form the constraints of the collision
problem and represent the facets that separate two
regions in space. Therefore, if a point satisfies two
sets of inequalities simultaneously, it belongs to the
corresponding convex objects. Thus, a collision is
detected at that point between these two objects.
The problem is reduced to maximize an objective
function in the form of (x + y + z). It is formulated
as follows:
X c
subject to:
x y zwhere X =
, c. . . d dd b 111
The coefficients of the matrices A and b are
calculated using the facets obtained from the CHs
reconstructed by the approach presented in the
previous subsection. Using the duality property, the
problem becomes:
π b
subject to:
Having formulated the problem, the dual system is
solved using a linear programming algorithm. If the
system is bounded, a feasible solution exists and
consequently, a collision is detected. Otherwise,
there is no collision.
HEALTHINF 2009 - International Conference on Health Informatics
Force feedback is a very interesting technology in
the context of human machine interface. It is used
as a haptic interface in order to make 3-D models
and simulations accessible to users and
participants. In this work, a 3-DOF force feedback
device is used. This will enhance the surgical
performance by guiding the (surgeon, student ...)
and give him a sense of touch and resistance when
collision is detected.
When the user moves the haptic device, the
position of the probe changes allowing dynamic
interactions with the virtual environment. That is,
the position of the medical probe is updated at
every step and the CD is checked by applying the
proposed algorithm on the updated matrices that
formulate the collision problem, i.e. solving the
system of equations at every step change. If
collision is detected, a force is applied against the
motion of the user of the haptic device. Therefore,
the user can feel the resistance of the applied force
against his hand’s motion, i.e. against the force
applied by the user to move the haptic device. This
force-reflecting device enables medical students
during the training to experience the real feeling of
touch. Touching virtual objects rather than seeing
them enhances the capability of the computer-
based system and gives the user the feeling of so
called “Immersion”. Figure 8 shows the flowchart
of the haptic feedback algorithm.
Figure 8: The Haptic Feedback Algorithm.
Figure 9 shows the haptic feedback system
designed, implemented and tested with the
computer-based simulation system.
Figure 9: The Haptic Feedback System.
Techniques of performing wrist arthroscopy have
been developed to evaluate and treat various wrist
disorders, such as scaphoid fractures. For
example, the dorsal percutaneous approach is a
very efficient way in treating displaced proximal
pole scaphoid fractures in many clinical and
operating rooms. This technique allows for faster
rehabilitation without restriction once CT scan
confirms a solid union.
Percutaneous arthroscopically assisted internal
fixation by a dorsal approach may be considered
in all acute scaphoid fractures selected for
surgical fixation (Rettig and Raskin, 1999). The
dorsal guide wire permits dorsal and volar
implantation of a cannulated screw along the
central axis of the scaphoid (Wozasek and Moser,
1991). The surgical technique described in (Slade
and Jaskwhich, 2001) uses the Standard Acutrak
screw. This screw is a headless, cannulated,
tapered screw with a graduated thread pitch to
provide inter-fragmentary compression without
hardware protrusion. This technique permits the
percutaneous reduction and rigid internal fixation
of proximal pole fractures.
First, the wrist is flexed and pronated for the
scaphoid to appear as a cylinder. The center of the
cylinder is the location for guide wire placement.
Then, the guide wire is driven dorsal to volar
through the center of the scaphoid. The wire exits
at the base of the thumb. Figure 10 (a) shows the
real placement of the guide wire during the
surgery. Real figures of the operation are taken
from a real surgery done by Dr. Joseph F. Slade
and distributed by ACUMED. On the other hand,
figure 10 (b) shows the same process done
virtually using the proposed VR simulation
Figure 10 (a): Real Operation.
Figure 10 (b): Virtual Operation.
After this step, a hand-held cannulated reamer
is placed over the guide wire and is used to
prepare the scaphoid. The scaphoid is reamed to
fit the length of the screw. Then the screw is
selected and is advanced with a cannulated driver
to the level of the reamed scaphoid. Figure 11 (a)
shows the real insertion of the screw in the
scaphoid while figure 11 (b) shows the virtual
operation of the same process.
Figure 11 (a): Real Operation.
Figure 11 (b): Virtual Operation.
VR surgical simulators play a very important role in
the practice of surgery for medical education and
training. An innovative application is the hand
surgery, especially wrist arthroscopy, which has
proven to be an extremely valuable tool in both
diagnosis and therapy. This paper presents a
functional prototype of a VR training system for
simulating wrist arthroscopy. Segmentation of CT
images and 3-D virtual model of the wrist of a
patient are shown. Algorithms that model objects
using the CH approaches and simulate real time
exact CD for solid objects are presented. Also, a
force feedback device coupled with a haptic
simulation algorithm is incorporated with the
system. Finally, a virtual simulation of dorsal
percutaneous scaphoid fixation is shown. This leads
in the development of a system that is used to
simulate wrist arthroscopic surgery procedures in a
VE. Medical students can learn the required basic
skills and then perform the training procedure on
real patients. This low cost system is safe, flexible
and can provide the students with precise and
detailed information for training and educational
HEALTHINF 2009 - International Conference on Health Informatics
purposes with the same benefits as professional
This research is funded by a grant from the Lebanese
National Council of Scientific Research (CNRS-L).
Goebel, M., 1993. (Ed.): Virtual Reality, Computers and
Graphics, Special Issue. vol.17, 6.
Medical Simulation System, Immersion Corporation
[online]. www.immersion.com/medical/products
Haisman, J., Bush, M., Wolfe, S., 2005. “Wrist
Arthroscopy: Standard Portals and Arthroscopic
Anatomy”, Journal of the American Society for
Surgery of the Hand, vol. 5, Issue 3, pp. 175-181.
LapSIM, Surgical Science Products, [online].
LapMentor, Simbionix Company Products, [online].
Szekely, G., Brechbuhler, C., Dual, J., et al. 2000. “Virtual
Reality-Based Simulation of Endoscopic Surgery”.
Teleoperators and Virtual Environments, 9(3), pp.
Procedicus VIST, Procedicus MIST, Mentice Products,
[online]. http://www.mentice.com
UroMentor, Simbionix Company Products, [online].
GI Mentor II, Simbionix Products [online].
Bro-Nielsen, M., Tasto, J., Cunningham, R., and Merril,
G., 1999. “PreOp endoscopic simulator: A PC-Based
Immersive Training System for Bronchoscopy”,
Medicine meets virtual reality, MMVR, vol. 7, pp. 76-
Heng, P., Cheng, Ch., Wong, T., Xu, Y., Chui, Y., Chan
K., and Tso, S., 2004. “A Virtual-Reality Training
System for Knee Arthroscopic Surgery”, IEEE
Transactions on Information Technology in
Biomedicine, Vol. 8, No. 2, pp. 217-227.
Bayonat, S., Garcia, M., Mendoza, C., and Fernandez,
J.M., 2006. “Shoulder Arthroscopy Training System
with Force Feedback”, IEEE conference on Medical
Information Visualization, MedVis, pp. 71-76.
Couprie, M., Najman, L., and Bertrand, G., 2005. “Quasi-
linear algorithms for the topological watershed”.
Journal of Mathematical Imaging and Vision, Special
Issue on Mathematical Morphology. Vol. 22, Issue 2 -
3, pp. 231-249.
Daragon, X., Couprie, M., and Bertrand, G., 2003.
“Discrete Frontiers”, Discrete geometry for computer
imagery, LNCS, Springer Verlag, Vol. 2886, pp. 236-
Yaacoub, F., Hamam, Y., Abche, A., and Fares, C., 2006.
“Convex Hull in Medical Simulations: A New Hybrid
Approach”, 32
Annual Conference of IEEE
Industrial Electronics Society, IECON'06, pp. 3308–
Yaacoub, F., Hamam, Y., and Abche, A., 2007. “Collision
Detection in Computer Simulation for Wrist
Arthroscopy Surgery Training”, IEEE International
Conference on Computer as a Tool, EUROCON'07,
pp. 2088–2095.
Slade, J., Jaskwhich, D., 2001. “Percutaneous Fixation of
Scaphoid Fractures”, Hand clinics, vol. 17(4), pp. 553-
Rettig, M., Raskin, K., 1999. “Retrograde Compression
Screw Fixation of Acute Proximal Pole Scaphoid
Fractures”, Journal of Hand Surgery, vol. 24 pp.
Wozasek, G., Moser, K., 1991. “Percutaneous Screw
Fixation of Fractures of The Scaphoid”, Journal of
Bone Joint Surgery, vol. 73, pp. 138-142.
ACUMED, Innovative Orthopedic Implants and
Accessories, [online]. http://www.acumed.net