FUZZY MODEL BASED CONTROL APPLIED TO IMAGE-BASED

VISUAL SERVOING

Paulo J. Sequeira Gonc¸alves

Escola Superior de Tecnologia de Castelo Branco

Av. Empres

´

ario, 6000-767 Castelo Branco, Portugal

L.F. Mendonc¸a, J. M. Sousa, J. R. Caldas Pinto

Technical University of Lisbon, IDMEC / IST

Av. Rovisco Pais, 1049-001 Lisboa, Portugal

Keywords:

visual servoing, robotic manipulators, inverse fuzzy control, fuzzy compensation, fuzzy modeling.

Abstract:

A new approach to eye-in-hand image-based visual servoing based on fuzzy modeling and control is proposed

in this paper. Fuzzy modeling is applied to obtain an inverse model of the mapping between image features

velocities and joints velocities, avoiding the necessity of inverting the Jacobian. An inverse model is identiﬁed

for each trajectory using measurements data of a robotic manipulator, and it is directly used as a controller.

As the inversion is not exact, steady-state errors must be compensated. This paper proposes the use of a

fuzzy compensator to deal with this problem. The control scheme contains an inverse fuzzy model and a

fuzzy compensator, which are applied to a robotic manipulator performing visual servoing, for a given proﬁle

of image features velocities. The obtained results show the effectiveness of the proposed control scheme:

the fuzzy controller can follow a point-to-point pre-deﬁned trajectory faster (or smoother) than the classic

approach.

1 INTRODUCTION

In eye-in-hand image-based visual servoing, the Jaco-

bian plays a decisive role in the convergence of the

control, due to its analytical model dependency on

the selected image features. Moreover, the Jacobian

must be inverted on-line, at each iteration of the con-

trol scheme. Nowadays, the research community tries

to ﬁnd the right image features to obtain a diagonal

Jacobian (Tahri and Chaumette, 2003). The obtained

results only guarantee the decoupling from the posi-

tion and the orientation of the velocity screw. This

is still a hot research topic, as stated very recently in

(Tahri and Chaumette, 2003).

In this paper, the previous related problems in the

Jacobian are addressed using fuzzy techniques, to ob-

tain a controller capable to control the system. First, a

fuzzy model to derive the inverse model of the robot is

used to compute the joints and end-effector velocities

in a straightforward manner. Second, the control ac-

tion obtained from the inverse model is compensated

to nullify a possible steady-state error by using a fuzzy

compensation.

A two degrees of freedom planar robotic manipu-

lator is controlled, based on eye-in-hand image-based

visual servoing using fuzzy control systems.

The paper is organized as follows. Section 2 de-

scribes brieﬂy the concept of image-based visual ser-

voing. The problem statement for the control prob-

lem tackled in this paper is presented in Section 3.

Section 4 presents fuzzy modeling and identiﬁca-

tion. Fuzzy compensation of steady-state errors is de-

scribed in Section 5. Section 6 presents the experi-

mental setup. The obtained results are presented in

Section 7. Finally, Section 8 present the conclusions

and the possible future research.

2 IMAGE-BASED VISUAL

SERVOING

In image-based visual servoing (Hutchinson et al.,

1996), the controlled variables are the image features,

extracted from the image containing the object. The

choice of different image features induces different

control laws, and its number depends also on the num-

ber of degrees of freedom (DOF) of the robotic ma-

nipulator under control. The robotic manipulator used

as test-bed in this paper is depicted in Fig. 1, and it has

2 DOF. Thus, only two features are needed to perform

the control. The image features, s, used are the coor-

dinates x and y of one image point.

143

Sequeira Gonçalves P., Mendonça L., Sousa J. and Caldas Pinto J. (2004).

FUZZY MODEL BASED CONTROL APPLIED TO IMAGE-BASED VISUAL SERVOING.

In Proceedings of the First International Conference on Informatics in Control, Automation and Robotics, pages 143-150

DOI: 10.5220/0001133201430150

Copyright

c

SciTePress

Z

0

X

0

Y

0

q

1

-q

2

Z

c

X

c

Y

c

Figure 1: Planar robotic manipulator with eye-in-hand,

camera looking down, with joint coordinates, and world and

camera frames.

2.1 Modeling the Image-Based

Visual Servoing System

In this paper, the image-based visual servoing sys-

tem used is the eye-in-hand system, where the camera

is ﬁxed at the robotic manipulator end-effector. The

kinematic modeling of the transformation between

the image features velocities and the joints velocities

is deﬁned as follows. Let

e

P be the pose of the end-

effector (translation and rotation), and

c

P be the pose

of the camera. Both depend on the robot joint vari-

ables q. Thus, the transformation from the camera

velocities and the end-effector velocities (Gonc¸alves

and Pinto, 2003a) is given by:

c

˙

P =

c

W

e

·

e

˙

P, (1)

where

c

W

e

=

c

R

e

S (

c

t

e

) ·

c

R

e

0

3×3

c

R

e

(2)

and S(

c

t

e

), is the skew-symmetric matrix associated

with the translation vector

c

t

e

, and

c

R

e

is the rotation

matrix between the camera and end-effector frames

needed to be measured.

The joint and end-effector velocities are related in

the end-effector frame by:

e

˙

P =

e

J

R

(q) · ˙q (3)

where

e

J

R

is the robot Jacobian for the planar robotic

manipulator (Gonc¸alves and Pinto, 2003a), and is

given by:

e

J

R

(q)=

l

1

· sin(q

2

) l

2

+ l

1

· cos(q

2

)0001

0 l

2

0001

T

(4)

and l

i

, with i =1, 2 are the lengths of the robot links.

The image features velocity, ˙s and the camera veloc-

ity,

c

˙

P are related by:

˙s = J

i

(x, y, Z) ·

c

˙

P (5)

where J

i

(x, y, Z) is the image Jacobian, which is

derived using the pin-hole camera model and a sin-

gle image point as the image feature (Gonc¸alves and

Figure 2: Control loop of image-based visual servoing, us-

ing a PD control law.

Pinto, 2003a), s , and is deﬁned as

J

i

(x, y, Z)=

−

1

Z

0

x

Z

x · y −

1+x

2

y

0 −

1

Z

y

Z

1+y

2

−x · y −x

(6)

where Z is the distance between the camera and the

object frames.

The relation between the image features velocity

and the joints velocities can now be easily derived

from (1), (3) and (5):

˙s = J(x, y, Z, q) · ˙q, (7)

where J is the total Jacobian, deﬁned as:

J(x, y, Z, q)=J

i

(x, y, Z) ·

c

W

e

·

e

J

R

(q) (8)

2.2 Controlling the Image-Based

Visual Servoing System

One of the classic control scheme of robotic manip-

ulators using information from the vision system, is

presented in (Espiau et al., 1992). The global control

architecture is shown in Fig. 2, where the block Robot

inner loop law is a PD control law.

The robot joint velocities ˙q to move the robot to a

predeﬁned point in the image, s

∗

are derived using

the Visual control law, (Gonc¸alves and Pinto, 2003a),

where an exponential decayment of the image fea-

tures error is speciﬁed:

˙q = −K

p

·

ˆ

J

−1

(x, y, Z, q) · (s − s

∗

) . (9)

K

p

is a positive gain, that is used to increase or de-

crease the decayment of the error velocity.

3 PROBLEM STATEMENT

To derive an accurate global Jacobian, J, a perfect

modeling of the camera, the image features, the posi-

tion of the camera related to the end-effector, and the

depth of the target related to the camera frame must

be accurately determined.

Even when a perfect model of the Jacobian is avail-

able, it can contain singularities, which hampers the

application of a control law. To overcome these difﬁ-

culties, a new type of differential relationship between

the features and camera velocities was proposed in

ICINCO 2004 - INTELLIGENT CONTROL SYSTEMS AND OPTIMIZATION

144

(Suh and Kim, 1994). This approach estimates the

variation of the image features, when an increment in

the camera position is given, by using a relation G.

This relation is divided into G

1

which relates the po-

sition of the camera and the image features, and F

1

which relates their respective variation:

s + δs = G(

c

P + δ

c

P )=G

1

(

c

P )+F

1

(

c

P, δ

c

P )

(10)

The image features velocity, ˙s, depends on the camera

velocity screw,

c

˙

P , and the previous position of the

image features, s, as shown in Section 2. Considering

only the variations in (10):

δs = F

1

(

c

P, δ

c

P ), (11)

let the relation between the camera position variation

δ

c

P , the joint position variation, δq and the previous

position of the robot q be given by:

δ

c

P = F

2

(δq, q). (12)

The equations (11) and (12) can be inverted if a one-

to-one mapping can be guaranteed. Assuming that

this inversion is possible, the inverted models are

given by:

δ

c

P = F

−1

1

(δs,

c

P ) (13)

and

δq = F

−1

2

(δ

c

P, q) (14)

The two previous equations can be composed be-

cause the camera is rigidly attached to the robot end-

effector, i.e., knowing q,

c

P can easily be obtained

from the robot direct kinematics,

c

R

e

and

c

t

e

. Thus,

an inverse function F

−1

is given by:

δq = F

−1

(δs, q) , (15)

and it states that the joint velocities depends on the

image features velocities and the previous position

of the robot manipulator. Equation (15) can be dis-

cretized as

δq(k)=F

−1

k

(δs(k +1),q(k)). (16)

In image-based visual servoing, the goal is to obtain a

joint velocity, δq(k), capable of driving the robot ac-

cording to a desired image feature position, s(k +1),

with an also desired image feature velocity, δs(k+1),

from any position in the joint spaces. This goal can be

accomplished by modeling the inverse function F

−1

k

,

using inverse fuzzy modeling as presented in section

4. This new approach to image-based visual servoing

allows to overcome the problems stated previously re-

garding the Jacobian inverse, the Jacobian singulari-

ties and the depth estimation, Z.

4 INVERSE FUZZY MODELING

4.1 Fuzzy Modeling

Fuzzy modeling often follows the approach of encod-

ing expert knowledge expressed in a verbal form in

a collection of if–then rules, creating an initial model

structure. Parameters in this structure can be adapted

using input-output data. When no prior knowledge

about the system is available, a fuzzy model can be

constructed entirely on the basis of systems measure-

ments. In the following, we consider data-driven

modeling based on fuzzy clustering (Sousa et al.,

2003).

We consider rule-based models of the Takagi-

Sugeno (TS) type. It consists of fuzzy rules which

each describe a local input-output relation, typically

in a linear form:

R

i

: If x

1

is A

i1

and ...and x

n

is A

in

then y

i

= a

i

x + b

i

,i=1, 2,...,K. (17)

Here R

i

is the ith rule, x =[x

1

,...,x

n

]

T

are the in-

put (antecedent) variable, A

i1

,...,A

in

are fuzzy sets

deﬁned in the antecedent space, and y

i

is the rule out-

put variable. K denotes the number of rules in the

rule base, and the aggregated output of the model, ˆy,

is calculated by taking the weighted average of the

rule consequents:

ˆy =

K

i=1

β

i

y

i

K

i=1

w

i

β

i

, (18)

where β

i

is the degree of activation of the ith rule:

β

i

=Π

n

j=1

µ

A

ij

(x

j

),i=1, 2,...,K, and

µ

A

ij

(x

j

):R → [0, 1] is the membership function

of the fuzzy set A

ij

in the antecedent of R

i

.

To identify the model in (17), the regression ma-

trix X and an output vector y are constructed from

the available data: X

T

=[x

1

,...,x

N

], y

T

=

[y

1

,...,y

N

], where N n is the number of sam-

ples used for identiﬁcation. The number of rules,

K, the antecedent fuzzy sets, A

ij

, and the conse-

quent parameters, a

i

,b

i

are determined by means of

fuzzy clustering in the product space of the inputs

and the outputs (Babu

ˇ

ska, 1998). Hence, the data

set Z to be clustered is composed from X and y:

Z

T

=[X, y].GivenZ and an estimated number of

clusters K, the Gustafson-Kessel fuzzy clustering al-

gorithm (Gustafson and Kessel, 1979) is applied to

compute the fuzzy partition matrix U.

The fuzzy sets in the antecedent of the rules are

obtained from the partition matrix U, whose ikth el-

ement µ

ik

∈ [0, 1] is the membership degree of the

data object z

k

in cluster i. One-dimensional fuzzy

sets A

ij

are obtained from the multidimensional fuzzy

sets deﬁned point-wise in the ith row of the partition

matrix by projections onto the space of the input vari-

ables x

j

. The point-wise deﬁned fuzzy sets A

ij

are

approximated by suitable parametric functions in or-

der to compute µ

A

ij

(x

j

) for any value of x

j

.

The consequent parameters for each rule are ob-

tained as a weighted ordinary least-square estimate.

Let θ

T

i

=

a

T

i

; b

i

, let X

e

denote the matrix [X; 1]

FUZZY MODEL BASED CONTROL APPLIED TO IMAGE-BASED VISUAL SERVOING

145

Figure 3: Robot-camera conﬁguration for model identiﬁca-

tion.

and let W

i

denote a diagonal matrix in R

N×N

hav-

ing the degree of activation, β

i

(x

k

), as its kth diag-

onal element. Assuming that the columns of X

e

are

linearly independent and β

i

(x

k

) > 0 for 1 ≤ k ≤ N ,

the weighted least-squares solution of y =X

e

θ +

becomes

θ

i

=

X

T

e

W

i

X

e

−1

X

T

e

W

i

y . (19)

4.2 Inverse modeling

For the robotic application in this paper, the inverse

model is identiﬁed using input-output data from the

inputs ˙q(k), outputs δs(k +1)and the state of the

system q(k). A commonly used procedure in robotics

is to learn the trajectory that must be followed by the

robot. From an initial position, deﬁned by the joint

positions, the robotic manipulator moves to the prede-

ﬁned end position, following an also predeﬁned tra-

jectory, by means of a PID joint position controller.

This specialized procedure has the drawback of re-

quiring the identiﬁcation of a new model for each

new trajectory. However, this procedure revealed to

be quite simple and fast. Moreover, this specialized

identiﬁcation procedure is able to alleviate in a large

scale the problems derived from the close-loop identi-

ﬁcation procedure. The identiﬁcation data is obtained

using the robot-camera conﬁguration shown in Fig. 3.

Note that we are interested in the identiﬁcation of

the inverse model in (16). Fuzzy modeling is used

to identify an inverse model, as e.g. in (Sousa et al.,

2003). In this technique, only one of the states of

the original model, ˙q(k), becomes an output of the

inverted model and the other state, q(k), together

with the original output, δs(k +1), are the inputs of

the inverted model. This model is then used as the

main controller in the visual servoing control scheme.

Therefore, the inverse model must be able to ﬁnd a

joint velocity, ˙q(k), capable to drive the robot fol-

lowing a desired image feature velocity in the image

space, δs(k +1), departing from previous joint posi-

tions, q(k).

5 FUZZY COMPENSATION OF

STEADY-STATE ERRORS

Control techniques based on a nonlinear process

model such as Model Based Predictive Control or

control based on the inversion of a model, (Sousa

et al., 2003), can effectively cope with the dynam-

ics of nonlinear systems. However, steady-state er-

rors may occur at the output of the system as a result

of disturbances or a model-plant mismatch. A scheme

is needed to compensate for these steady-state errors.

A classical approach is the use of an integral control

action. However, this type of action is not suitable

for highly nonlinear systems because it needs differ-

ent parameters for different operating regions of the

system.

Therefore, this paper develops a new solution,

called fuzzy compensation. The fuzzy compensator

intends to compensate steady-state errors based on the

information contained in the model of the system and

allows the compensation action to change smoothly

from an active to an inactive state. Taking the local

derivative of the model with respect to the control ac-

tion, it is possible to achieve compensation with only

one parameter to be tuned (similar to the integral gain

of a PID controller). For the sake of simplicity, the

method is presented for nonlinear discrete-time SISO

systems, but it is easily extended to MIMO systems.

Note that this is the case of the 2-DOF robotic system

in this paper.

5.1 Derivation of fuzzy

compensation

In this section it is convenient to delay one step the

model for notation simplicity. The discrete-time SISO

regression model of the system under control is then

given by:

y(k)=f(x(k − 1)) , (20)

where x(k − 1) is the state containing the lagged

model outputs, inputs and states of the system. Fuzzy

compensation uses a correction action called u

c

(k),

which is added to the action derived from an inverse

model-based controller, u

m

(k), as shown in Fig. 4.

The total control signal applied to the process is thus

given by,

u(k)=u

m

(k)+u

c

(k). (21)

Note that the controller in Fig. 4 is an inverse model-

based controller for the robotic application in this pa-

per, but it could be any controller able to control the

system, such as a predictive controller.

Taking into account the noise and a (small) off-

set error, a fuzzy set SS deﬁnes the region where

the compensation is active, see Fig. 5. The error

ICINCO 2004 - INTELLIGENT CONTROL SYSTEMS AND OPTIMIZATION

146

Figure 4: Fuzzy model-based compensation scheme.

Figure 5: Deﬁnition of the fuzzy boundary SS where fuzzy

compensation is active.

is deﬁned as e(k)=r(k) − y(k), and the mem-

bership function µ

SS

(e(k)) is designed to allow for

steady-state error compensation whenever the support

of µ

SS

(e(k)) is not zero. The value of B that deter-

mines the width of the core(µ

SS

) should be an upper

limit of the absolute value of the possible steady-state

errors. Fuzzy compensation is fully active in the in-

terval [−B,B]. The support of µ

SS

(e(k)) should be

chosen such that it allows for a smooth transition from

enabled to disabled compensation. This smoothness

of µ

SS

induces smoothness on the fuzzy compensa-

tion action u

c

(k), and avoids abrupt changes in the

control action u(k).

The compensation action u

c

(k) at time k is given

by

u

c

(k)=µ

SS

(e(k))

k−1

i=0

u

c

(i)+Ke(k) f

−1

u

,

(22)

where µ

SS

(e(k)) is the error membership degree at

time k, K

c

is a constant parameter to be determined

and

f

u

=

∂f

∂u(k − 1)

x(k−1)

(23)

is the partial derivative of the function f in (20) with

respect to the control action u(k − 1), for the present

state of the system x(k − 1).

6 EXPERIMENTAL SETUP

The experimental setup can be divided in two subsys-

tems, the vision system and the robotic manipulator

system. The vision system acquires and process im-

ages at 50Hz, and sends image features, in pixels, to

the robotic manipulator system via a RS-232 serial

port, also at 50 Hz. The robotic manipulator system

controls the 2 dof planar robotic manipulator, Fig. 6

and Fig. 1 using the image data sent from the vision

system.

6.1 Vision System

The vision system performs image acquisition and

processing under software developed in Visual

C++

TM

and running in an Intel Pentium IV

TM

, at 1.7

GHz, using a Matrox Meteor

TM

frame-grabber. The

CCD video camera, Costar, is ﬁxed in the robot end-

effector looking up and with its optical axis perpen-

dicular to the plane of the robotic manipulator. The

planar target is parallel to the plane of the robot, and

is above it. This conﬁguration allows deﬁning the Z

variable as pre-measured constant in the image Jaco-

bian (6) calculation, Z =1. The target consists of one

light emitting diode (LED), in order to ease the image

processing and consequently diminish its processing

time, because we are mainly concerned in control al-

gorithms.

Following the work in (Reis et al., 2000) the im-

age acquisition is performed at 50 Hz. It is well

known that the PAL video signal has a frame rate of

25 frames/second. However, the signal is interlaced,

i.e., odd lines and even lines of the image are codiﬁed

in two separate blocks, known as ﬁelds. These ﬁelds

are sampled immediately before each one is sent to

the video signal stream. This means that each ﬁeld

is an image with half the vertical resolution, and the

application can work with a rate of 50 frames/second.

The two image ﬁelds were considered as separate im-

ages thus providing a visual sampling rate of 50 Hz.

When performing the centroid evaluation at 50 Hz,

an error on its vertical coordinate will arise, due to

the use of only half of the lines at each sample time.

Multiplying the vertical coordinate by two, we obtain

a simple approximation of this coordinate.

The implemented image processing routines ex-

tract the centroid of the led image. Heuristics to track

this centroid, can be applied very easily (Gonc¸alves

and Pinto, 2003b). These techniques allow us to cal-

culate the image features vector s, i.e., the two im-

age coordinates of the centroid. The image process-

ing routines and the sending commands via the RS-

232 requires less than 20ms to perform in our system.

The RS-232 serial port is set to transmit at 115200

bits/second. When a new image is acquired, the pre-

FUZZY MODEL BASED CONTROL APPLIED TO IMAGE-BASED VISUAL SERVOING

147

Figure 6: : Experimental Setup, Planar Robotic Manipula-

tor with eye-in-hand, camera looking up.

vious one was already processed and the image data

sent via RS-232 to the robotic manipulator system.

6.2 Robotic Manipulator System

The robot system consists of a 2 dof planar robotic

manipulator (Baptista et al., 2001) moving in a hor-

izontal plane, the power ampliﬁers and an Intel

Pentium

TM

200MHz, with a ServoToGo

TM

I/O card.

The planar robotic manipulator has two revolute joints

driven by Harmonic Drive Actuators - HDSH-14.

Each actuator has an electric d.c. motor, a harmonic

drive, an encoder and a tachogenerator. The power

ampliﬁers are conﬁgured to operate as current ampli-

ﬁers. In this functioning mode, the input control sig-

nal is a voltage in the range ±10V with current ratings

in the interval [−2; 2]V . The signals are processed

through a low cost ISA-bus I/O card from ServoToGo,

INC. The I/O board device drivers were developed at

the Mechanical Engineering Department from Insti-

tuto Superior T

´

ecnico. The referred PC is called in

our overall system as the Target PC. It receives the im-

age data from the vision system via RS-232, each 20

ms, and performs, during the 20 ms of the visual sam-

ple time, the visual servoing algorithms developed in

the Host-PC.

It is worth to be noted that the robot manipulator

joint limits are: q ∈ [−

π

2

;

π

2

].

6.3 Systems Integration

The control algorithms for the robotic manipulator

are developed in a third PC, called Host-PC. An exe-

cutable ﬁle containing the control algorithms is then

created for running in the Target-PC. The executable

ﬁle created, containing the control algorithm, is then

downloaded, via TCP/IP, to the Target-PC, where all

the control is performed. After a pre-deﬁned time for

execution, all the results can be uploaded to the Host-

PC for analysis. During execution the vision system

sends to the Target PC the actual image features as a

visual feedback for the visual servoing control algo-

rithm, using the RS-232 serial port.

7 RESULTS

This section presents the simulation results obtained

for the robotic manipulator, presented in Section 6.

First, the identiﬁcation of the inverse fuzzy model of

the robot is described. Then, the control results using

the fuzzy model based controller introduced in this

paper, i.e. the combination of inverse fuzzy control

and fuzzy compensation, are presented.

7.1 Inverse Fuzzy Modeling

In order to apply the controller described in this paper,

ﬁrst an inverse fuzzy controller must be identiﬁed.

Recall that a model must be identiﬁed for each trajec-

tory. This trajectory is presented in Fig. 10. The pro-

ﬁle chosen for the image features velocity moves the

robot from the initial joints position q =[−1.5; 0.3]

to the ﬁnal position q =[−1.51; 1.52], in one sec-

ond, starting and ending with zero velocity. An in-

verse fuzzy model (16) for this trajectory is identiﬁed

using the fuzzy modeling procedure described in Sec-

tion 4.1. The measurements data is obtained from a

simulation of the planar robotic manipulator eye-in-

hand system.

0 50 100 150 200 250

−2

−1

0

1

2

No samples [20 ms]

Joint Positions [rad]

Inverse Fuzzy Model Input Data

0 50 100 150 200 250

−0.1

−0.05

0

0.05

0.1

No samples [20 ms]

Image features velocity [m/s]

Figure 7: Input data for fuzzy identiﬁcation. Top: joint po-

sitions, q. Bottom: image feature velocity, δs.

The set of identiﬁcation data used to build the in-

verse fuzzy model contains 250 samples, with a sam-

ple time of 20ms. Figure 7 presents the input data,

which are the joint positions q

1

(k) and q

2

(k), and the

image features velocities δs

x

(k) and δs

y

(k), used for

identiﬁcation.

ICINCO 2004 - INTELLIGENT CONTROL SYSTEMS AND OPTIMIZATION

148

Note that to identify the inverse model, one cannot

simply feed the inputs as outputs and the outputs as

inputs. Since the inverse model (16) is a non-causal

model, the output of the inverse model must be shifted

one step, see (Sousa et al., 2003). The validation of

the inverse fuzzy model is shown in Fig. 8, where the

joint velocities δq are depicted. Note that two fuzzy

models are identiﬁed, one for each velocity. It is clear

that the model is quite good. Considering, e.g. the

performance criteria variance accounted for (VAF),

the models have the VAFs of 70.2% and 99.7%. When

a perfect match occur, this measure has the value of

100%. Then, the inverse model for the joint velocity

δq

2

is very accurate, but the inverse model for δq

1

is

not so good. This was expectable as the joint velocity

δq

1

varies much more than δq

2

. However, this model

will be sufﬁcient to obtain an accurate controller, as is

shown in Section 7.2.

0 50 100 150 200 25

0

−0.05

0

0.05

No samples [20 ms]

Joint 1 Velocity [rad/s]

Inverse Fuzzy Model Validation

0 50 100 150 200 25

0

−2

−1

0

1

2

No samples [20 ms]

Joint 2 Velocity [rad/s]

Figure 8: Validation of the inverse fuzzy model (joint veloc-

ities δq). Solid – real output data, and dash-dotted – output

of the inverse fuzzy model.

In terms of parameters, four rules (clusters) re-

vealed to be sufﬁcient for each output, and thus the

inverse fuzzy model has 8 rules, 4 for each output, δq

1

and δq

2

. The clusters are projected into the product-

space of the space variables and the fuzzy sets A

ij

are

determined.

7.2 Control results

This section presents the obtained control results, us-

ing the classical image-based visual servoing pre-

sented in Section 2, and the fuzzy model-based con-

trol scheme combining inverse model control pre-

sented in Section 4, and fuzzy compensation de-

scribed in Section 5. The implementation was devel-

oped in a simulation of the planar robotic manipula-

tor eye-in-hand system. This simulation was devel-

oped and validated in real experiments, using clas-

sic visual servoing techniques, by the authors. Re-

0 10 20 30 40 50 60 70 80 9

0

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

No samples [20 ms]

Image Features Position [m]

Comparison of Image Features Position versus Trajectory

Figure 9: Image features trajectory, s. Solid – fuzzy visual

servo control; dashed – classical visual servo control, and

dash-dotted – desired trajectory.

−20 −10 0 10 20 30 40 50 6

0

−60

−50

−40

−30

−20

−10

0

10

20

30

40

Comparison of Image Features Trajectory

u [pixel]

v [pixel]

Figure 10: Image features trajectory s, in the image plane.

Solid – inverse fuzzy model control, and dash-dotted – clas-

sical visual servo control.

call that the chosen proﬁle for the image features ve-

locity moves the robot from the initial joints position

q =[−1.5; 0.3] to the ﬁnal position q =[−1.51; 1.52]

in one second, starting and ending with zero velocity.

The comparison of the image features trajectory for

both the classic and the fuzzy visual servoing con-

trollers is presented in Fig. 9. In this ﬁgure, it is shown

that the classical controller follows the trajectory with

a better accuracy than the fuzzy controller. However,

the fuzzy controller is slightly faster, and reaches the

vicinity of the target position before the classical con-

troller. The image features trajectory in the image

plane is presented in Fig. 10, which shows that both

controllers can achieve the goal position (the times,

×, sign in the image) with a very small error. This

ﬁgure shows also that the trajectory obtained with the

inverse fuzzy model controller is smoother. There-

fore, it is necessary to check the joint velocities in

order to check their smoothness. Thus the joint veloc-

FUZZY MODEL BASED CONTROL APPLIED TO IMAGE-BASED VISUAL SERVOING

149

0 10 20 30 40 50 60 70 80 9

0

−2

−1

0

1

2

3

4

No samples [20 ms]

Joint Velocity [rad/s]

Comparison of Joint Velocities

Figure 11: Joint velocities, δq. Solid – fuzzy visual servo,

and dashed – classical visual servo.

ities are depicted in Fig. 11, where it is clear that the

classical controller presents undesirable and danger-

ous oscillations in the joint velocities. This fact is due

to the high proportional gain that the classical con-

troller must have to follow the trajectory, which de-

mands high velocity. This effect is easily removed by

slowing down the classical controller. But in this case,

the fuzzy controller clearly outperforms the classical

controller. The classical controller can only follow

the trajectory without oscillations in the joint veloc-

ities if the robot takes 1.5s to move from one point

to the other. In this case, the classical controller is

about 50% slower than the fuzzy model-based con-

troller proposed in this paper.

8 CONCLUSIONS

This paper introduces an eye-in-hand image-based vi-

sual servoing scheme based on fuzzy modeling and

control. The fuzzy modeling approach was applied

to obtain an inverse model of the mapping between

image features velocities and joints velocities. This

inverse model is added to a fuzzy compensator to be

directly used as controller of a robotic manipulator

performing visual servoing, for a given image features

velocity proﬁle.

The obtained results showed that both the classi-

cal and the fuzzy controllers can follow the image

features velocity proﬁle. However, the proportional

gain of the classic visual servoing must be very high.

This fact justiﬁes the oscillations veriﬁed in the joint

velocities, which are completely undesirable in robot

control. For that reason, the inverse fuzzy control pro-

posed in this paper performs better.

As future work, the proposed fuzzy model based

control scheme will be implemented in the experi-

mental test-bed. Note that an off-line identiﬁcation

of the inverse fuzzy model must ﬁrst be performed.

The complete automation of this identiﬁcation step is

also under study.

ACKNOWLEDGEMENT

This work is supported by the “Programa de Financia-

mento Plurianual de Unidades de I&D (POCTI) do Quadro

Comunit

´

ario de Apoio III”, by program FEDER, by the

FCT project POCTI/EME/39946/2001, and by the “Pro-

grama do FSE-UE, PRODEP III, acc¸

˜

ao 5.3, no

ˆ

ambito do

III Quadro Comunit

´

ario de apoio”.

REFERENCES

Babu

ˇ

ska, R. (1998). Fuzzy Modeling for Control. Kluwer

Academic Publishers, Boston.

Baptista, L., Sousa, J. M., and da Costa, J. S. (2001). Fuzzy

predictive algorithms applied to real-time force con-

trol. Control Engineering Practice, pages 411–423.

Espiau, B., Chaumette, F., and Rives, P. (1992). A new

approach to visual servoing in robotics. IEEE Trans-

actions on Robotics and Automation, 8(3):313–326.

Gonc¸alves, P. S. and Pinto, J. C. (2003a). Camera con-

ﬁgurations of a visual servoing setup, fora2dof

planar robot. In Proceedings of the 7th Interna-

tional IFAC Symposium on Robot Control, Wroclaw,

Poland., pages 181–187, Wroclaw, Poland.

Gonc¸alves, P. S. and Pinto, J. C. (2003b). An experi-

mental testbed for visual servo control of robotic ma-

nipulators. In Proceedings of the IEEE Conference

on Emerging Technologies and Factory Automation,

pages 377–382, Lisbon, Portugal.

Gustafson, D. E. and Kessel, W. C. (1979). Fuzzy clustering

with a fuzzy covariance matrix. In Proceedings IEEE

CDC, pages 761–766, San Diego, USA.

Hutchinson, S., Hager, G., and Corke, P. (1996). A tuto-

rial on visual servo control. IEEE Transactions on

Robotics and Automation, 12(5):651–670.

Reis, J., Ramalho, M., Pinto, J. C., and S

´

a da Costa, J.

(2000). Dynamical characterization of ﬂexible robot

links using machine vision. In Proceedings of the 5th

Ibero American Symposium on Pattern Recognition,

pages 487–492, Lisbon, Portugal.

Sousa, J. M., Silva, C., and S

´

a da Costa, J. (2003). Fuzzy

active noise modeling and control. International Jour-

nal of Approximate Reasoning, 33:51–70.

Suh, I. and Kim, T. (1994). Fuzzy membership function

based neural networks with applications to the visual

servoing of robot manipulators. IEEE Transactions on

Fuzzy Systems, 2(3):203–220.

Tahri, O. and Chaumette, F. (2003). Application of moment

invariants to visual servoing. In Proceedings of the

IEEE Int. Conf. on Robotics and Automation, pages

4276–4281, Taipeh, Taiwan.

ICINCO 2004 - INTELLIGENT CONTROL SYSTEMS AND OPTIMIZATION

150