![](bg1.png)
Visual Servoing-based Registration of Multimodal Images
M. Ourak
1
, B. Tamadazte
1
, N. Andreff
1
and E. Marchand
2
1
FEMTO-ST Institute, AS2M Department, Universit
´
e de Franche-Comt
´
e/CNRS/ENSMM, 24 rue Savary, Besano¸n, France
2
Universit
´
e de Rennes 1, IRISA, Rennes, France
Keywords:
Visual Servoing, Mutual Information, Nelder-Mead Simplex.
Abstract:
This paper deals with mutual information-based numerical and physical registration of white light images vs.
fluorescence images for microrobotic laser microphonosurgery of the vocal folds. More precisely, it presents
two techniques: a numerical registration of multimodal images and a vision feedback control for positioning
an endoscope with regards to a preoperative image (fluorescence image). Nelder-Mead Simplex for nonlinear
optimization is used to minimize the cost-function. The proposed methods are successfully validated in an
experimental set-up using preoperative fluorescence images and real-time white light images of the vocal
folds.
1 INTRODUCTION
Direct visualization of the larynx and the trachea is
often used for the diagnosis but also in surgical in-
tervention (Jackel et al., 2013). The most successful
robotic system for the vocal folds surgery is certainly
the suspension lryngoscope (Figure 1(a)). It consists
of a straight-rigid laryngoscope, a stereomicroscope,
a laser source, and a controlled device based on a foot-
pedal activating the laser (Eckel et al., 2003). This
system is largely deployed in hospitals but it features
many drawbacks: i) extreme extension of the patient’s
neck; ii) poor ergonomics of the operating setup; iii)
considerable skills and expertise required for the clin-
ician; and iv) lack of precision.
Alternative approaches are under investiga-
tion: the use of the HARP (Highly Articulated
Robotic Probe) highly flexible robot for conventional
surgery (Degani et al., 2006) or the use of an endo-
scopic laser micromanipulator (Tamadazte and Andr-
eff, 2014) (Figure 1(b)). In both cases, surgery can
be preceded by a diagnosis using fluorescence imag-
ing (Sevick-Muraca, 2004). For this, the fluorescence
based diagnosis is done a few days before surgery in-
tervention. Therefore, during a surgical intervention
the fluorescence diagnosis image must be registered
to the real-time white light images grabbed by the en-
doscopic system in order to define the incision path
pour the laser ablation or resection. The Registration
can be done either numerically or by physically mov-
ing the endoscope to the place where the fluorescence
image was grabbed few days ago.
In this paper, our aim is to control a robot based
on direct visual servoing, using image information
coming from light white and fluorescence sensors.
However, this control needs to be done without a pri-
ory model of the robot and the camera. Indeed, ap-
proaches have been implemented which are mainly
based on the use of the image global information
(gradient (Marchand and Collewet, 2010), photome-
try (Collewet and Marchand, 2011) or mutual infor-
mation (Dame and Marchand, 2011)). The use of mu-
tual information (MI) in visual servoing problems has
proved to be especially effective in the case of mul-
timodal and less contrasted images (Dame and Marc-
hand, 2009).In fact, these control techniques assume
that the kinematic model of the robot and the cam-
era intrinsic parameters are at least partially known,
but would fail if the system parameters were fully un-
known. In addition to the constraint that the initial po-
sition cannot be very distant from the desired position
to totally ensure convergence. Therefore, it was pro-
posed to use the Simplex method (Nelder and Mead,
1965) instead of gradient (which requires at least a
rough calibration of the camera and a computation of
the camera/robot transformation) as in (Miura et al.,
2005) where the geometrical visual features are used
to design the controller.
Furthermore, in the surgical robotics context, it
is preferable to free ourselves from any calibration
procedure (camera, robot or robot/camera system) for
several reasons:
44
Ourak M., Tamadazte B., Andreff N. and Marchand E..
Visual Servoing-based Registration of Multimodal Images.
DOI: 10.5220/0005528900440051
In Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO-2015), pages 44-51
ISBN: 978-989-758-123-6
Copyright
c
2015 SCITEPRESS (Science and Technology Publications, Lda.)