VISION-BASED AUTONOMOUS APPROACH AND LANDING FOR

AN AIRCRAFT USING A DIRECT VISUAL TRACKING METHOD

Tiago F. Gonc¸alves, Jos´e R. Azinheira

IDMEC, IST/TULisbon, Av. Rovisco Pais N.1, 1049-001 Lisboa, Portugal

Patrick Rives

INRIA-Sophia Antipolis, 2004 Route des Lucioles, BP93, 06902 Sophia-Antipolis, France

Keywords:

Aircraft autonomous approach and landing, Vision-based control, Linear optimal control, Dense visual track-

ing.

Abstract:

This paper presents a feasibility study of a vision-based autonomous approach and landing for an aircraft using

a direct visual tracking method. Auto-landing systems based on the Instrument Landing System (ILS) have

already proven their importance through decades but general aviation stills without cost-effective solutions

for such conditions. However, vision-based systems have shown to have the adequate characteristics for the

positioning relatively to the landing runway. In the present paper, rather than points, lines or other features

susceptible of extraction and matching errors, dense information is tracked in the sequence of captured images

using an Efﬁcient Second-Order Minimization (ESM) method. Robust under arbitrary illumination changes

and with real-time capability, the proposed visual tracker suits all conditions to use images from standard

CCD/CMOS to Infrared (IR) and radar imagery sensors. An optimal control design is then proposed using the

homography matrix as visual information in two distinct approaches: reconstructing the position and attitude

(pose) of the aircraft from the visual signals and applying the visual signals directly into the control loop. To

demonstrate the proposed concept, simulation results under realistic atmospheric disturbances are presented.

1 INTRODUCTION

Approach and landing are known to be the most de-

manding ﬂight phase in ﬁxed-wing ﬂight operations.

Due to the altitudes involved in ﬂight and the con-

sequent nonexisting depth perception, pilots must in-

terpret position, attitude and distance to the runway

using only two-dimensional cues like perspective, an-

gular size and movement of the runway. At the same

time, all six degrees of freedom of the aircraft must be

controlled and coordinated in order to meet and track

the correct glidepath till the touchdown.

In poor visibility conditions and degraded visual

references, landing aids must be considered. The In-

strument Landing System (ILS) is widely used in most

of the international airports around the world allow-

ing pilots to establish on the approach and follow the

ILS, in autopilot or not, until the decision height is

reached. At this point, the pilot must have visual con-

tact with the runway to continue the approach and

proceed to the ﬂare manoeuvre or, if it is not the

case, to abort. This procedure has proven its relia-

bility through decades but landing aids systems that

require onboard equipment are still not cost-effective

for most of the general airports. However, in the

last years, the Enhanced Visual Systems (EVS) based

on Infrared (IR) allowed the capability to proceed to

non-precision approaches and obstacle detection for

all weather conditions. The vision-based control sys-

tem proposed in the present paper intends then to take

advantage of these emergent vision sensors in order to

allow precision approaches and autonomous landing.

The intention of using vision systems for au-

tonomous landings or simply estimate the aircraft po-

sition and attitude (pose) is not new. Flight tests of

a vision-based autonomous landing relying on fea-

ture points on the runway were already referred by

(Dickmanns and Schell, 1992) whilst (Chatterji et al.,

1998) present a feasibility study on pose determina-

tion for an aircraft night landing based on a model

of the Approach Lighting System (ALS). Many oth-

ers have followed in using vision-based control on

F. Gonçalves T., R. Azinheira J. and Rives P. (2009).

VISION-BASED AUTONOMOUS APPROACH AND LANDING FOR AN AIRCRAFT USING A DIRECT VISUAL TRACKING METHOD.

In Proceedings of the 6th International Conference on Informatics in Control, Automation and Robotics - Robotics and Automation, pages 94-101

DOI: 10.5220/0002212900940101

 SciTePress

ﬁxed/rotary wings aircrafts, and even airships, in sev-

eral goals since autonomous aerial refueling ((Kim-

mett et al., 2002), (Mati et al., 2006)), stabiliza-

tion with respect to a target ((Hamel and Mahony,

2002), (Azinheira et al., 2002)), linear structures

following ((Silveira et al., 2003), (Rives and Azin-

heira, 2004), (Mahony and Hamel, 2005)) and, obvi-

ously, automatic landing ((Sharp et al., 2002), (Rives

and Azinheira, 2002), (Proctor and Johnson, 2004),

(Bourquardez and Chaumette, 2007a), (Bourquardez

and Chaumette, 2007b)). In these problems, differ-

ent types of visual features were considered including

geometric model of the target, points, corners of the

runway, binormalized Plucker coordinates, the three

parallel lines of the runway (at left, center and right

sides) and the two parallel lines of the runway along

with the horizon line and the vanishing point. Due to

the standard geometry of the landing runway and the

decoupling capabilities, the last two approaches have

been preferred in problems of autonomous landing.

In contrast with feature extraction methods, direct

or dense methods are known by their accuracy be-

cause all the information in the image is used with-

out intermediate processes, reducing then the sources

of errors. The usual disadvantage of such method is

the computational consuming of the sum-of-squared-

differences minimization due to the computation of

the Hessian matrix. The Efﬁcient Second-order Mini-

mization (ESM) (Malis, 2004) (Behimane and Malis,

2004) (Malis, 2007) method does not require the

computation of the Hessian matrix maintaining how-

ever the high convergence rate characteristic of the

second-order minimizations as the Newton method.

Robust under arbitrary illumination changes (Silveira

and Malis, 2007) and with real-time capability, the

ESM suits all the requirements to use images from

the common CCD/CMOS to IR sensors.

In vision-based or visual servoing problems, a pla-

nar scene plays an important role since it simpliﬁes

the computation of the projective transformation be-

tween two images of the scene: the planar homogra-

phy. The Euclidean homography, computed with the

knowledge of the calibration matrix of the imagery

sensor, is here considered as the visual signal to use

into the control loop in two distinct schemes. The

position-based visual servoing (PBVS) uses the re-

covered pose of the aircraft from the estimated projec-

tive transformation whilst the image-based visual ser-

voing (IBVS) uses the visual signal directly into the

control loop by means of the interaction matrix. The

controller gains, from standard output error LQR de-

sign with a PI structure, are common for both schemes

whose results will be then compared with a sensor-

based scheme where precise measures are considered.

The present paper is organized as follows: In the

Section 2, some useful notations in computer vision

are presented, using as example the rigid-body motion

equation, along with an introduction of the consid-

ered frames and a description of the aircraft dynamics

and the pinhole camera models. In the same section,

the two-view geometry is introduced as the basis for

the IBVS control law. The PBVS and IBVS control

laws are then presented in the Section 3 as well as the

optimal controller design. The results are shown in

Section 4 while the ﬁnal conclusions are presented in

Section 5.

2 THEORETICAL BACKGROUND

2.1 Notations

The rigid-body motion of the frame b with respect to

frame a by

∈ SO(3) and

∈ R

, the rotation

matrix and the translation vector respectively, can be

expressed as

X =

X+

(1)

where,

X ∈ R

denotes the coordinates of a 3D point

in the frame a or, in a similar manner,in homogeneous

coordinates as

X =



0 1





(2)

where,

∈ SE(3), 0 denotes a matrix of zeros with

the appropriate dimensions and

X ∈ R

are the ho-

mogeneous coordinates of the point

X. In the same

way, the Coriolis theorem applied to 3D points can

also be expressed in homogenous coordinates, as a re-

sult of the derivative of the rigid-body motion relation

in Eq. (2),

X =

−1

X = (3)



ω v

0 0



X =

X ,

∈ R

4×4

where, the angular velocity tensor

ω ∈ R

3×3

is the

skew-symmetric matrix of the angular velocity vec-

tor ω such that ω × X =

ωX and the vector

[v,ω]

⊤

∈ R

denotes the velocity screw and indicates

the velocity of the frame b moving relative to a and

viewed from the frame a. Also important in the

present paper is the deﬁnition of stacked matrix, de-

noted by the superscript ”

”, where each column is

rearranged into a single column vector.

2.2 Aircraft Dynamic Model

Let F

be the earth frame, also called NED for North-

East-Down, whose origin coincides with the desirable

VISION-BASED AUTONOMOUS APPROACH AND LANDING FOR AN AIRCRAFT USING A DIRECT VISUAL

TRACKING METHOD

touchdown point in the runway. The latter, unless ex-

plicitly indicated and without loss of generality, will

be considered aligned with North. The aircraft lin-

ear velocity v = [u, v, w] ∈ R

, as well as its angular

velocity ω = [p,q,r] ∈ R

, is expressed in the air-

craft body frame F

whose origin is at the center of

gravity where u is deﬁned towards the aircraft nose,

v towards the right wing and w downwards. The at-

titude, or orientation, of the aircraft with respect to

the earth frame F

is stated in terms of Euler angles

Φ = [φ,θ,ψ] ⊂ R

, the roll, pitch and yaw angles re-

spectively.

The aircraft motion in atmospheric ﬂight is usu-

ally deduced using Newton’s second law and con-

sidering the motion of the aircraft in the earth frame

, assumed as an inertial frame, under the inﬂuence

of forces and torques due to gravity, propulsion and

aerodynamics. As mentioned above, both linear and

angular velocities of the aircraft are expressed in the

body frame F

as well asfor the considered forces and

moments. As a consequence, the Coriolis theorem

must be invoked and the kinematic equations appear

naturally relating the angular velocity rate ω with the

time derivative of the Euler angles

Φ = R

−1

ω and the

instantaneous linear velocity v with the time deriva-

tive of the NED position





⊤

= S

⊤

In order to simplify the controller design, it is

common to linearize the non-linear model around an

given equilibrium ﬂight condition, usually a func-

tion of airspeed V

and altitude h

. This equilib-

rium or trim ﬂight is frequently chosen to be a steady

wings-level ﬂight, without presence of wind distur-

bances, also justiﬁed here since non-straight landing

approaches are not considered in the present paper.

The resultant linear model is then function of the per-

turbation in the state vector x and in the input vector

u as



˙x





0 A











(4)

describing the dynamics of the two resultant decou-

pled, lateral and longitudinal, motions. The longitu-

dinal, or vertical, state vector is x

= [u, w,q,θ]

⊤

∈ R

and the respective input vector u

= [δ

,δ

]

⊤

∈ R

(elevator and throttle) while, in the lateral case, the

state vector is x

= [v, p,r,φ]

⊤

∈ R

and the respec-

tive input vector u

= [δ

,δ

]

⊤

∈ R

(ailerons and

rudder). Because the equilibrium ﬂight condition is

slowly varying for manoeuvres as the landing phase,

the linearized model in Eq. (4) can then be considered

constant along all the glidepath.

2.3 Pinhole Camera Model

The onboard camera frame F

, rigidly attached to the

aircraft, has its origin at the center of projection of

the camera, also called pinhole. The corresponding z-

axis, perpendicular to the image plane, lies on the op-

tical axis while the x- and y- axis are deﬁned towards

right and down, respectively. Note that the camera

frame F

is not in agreement with those usually de-

ﬁned in ﬂight mechanics.

Let us consider a 3D point P whose coordinates in

the camera frame F

are

X = [X,Y, Z]

⊤

. This point

is perspectively projected onto the normalized image

plane I

∈ R

, distant one-meter from the center of

projection, at the point m = [x,y,1]

⊤

∈ R

such that

m =

X. (5)

Note that, computing the projected point m know-

ing coordinates X of the 3D point is a straightforward

problems but the inverse is not true because Z is one

of the unknowns. As a consequence, the coordinates

of the point X could only be computed up to a scale

factor, resulting on the so-called lost of depth percep-

tion.

When a digital camera is considered, the same

point P is projected onto the image plane I , whose

distance to the center of projection is deﬁned by the

focal length f ∈ R

, at the pixel p = [p

, p

,1] ⊂ R

p = Km (6)

where, K ∈ R

3×3

is the camera intrinsical parameters,

or calibration matrix, deﬁned as follows

K =





fs p

0 f

0 0 1





(7)

The coordinates p

= [p

, p

,1]

⊤

∈ R

deﬁne the

principal point, corresponding to the intersection be-

tween the image plane and the optical axis. The pa-

rameter s, zero for most of the cameras, is the skew

factor which characterizes the afﬁne pixel distortion

and, ﬁnally, f

and f

are the focal lengths in the both

directions such that when f

= f

the camera sensor

presents square pixels.

2.4 Two-views Geometry

Let us consider a 3D point P whose coordinates

X in

the current camera frame are related with those

X in

the earth frame by the rigid-body motion in Eq. (1) of

with respect to F

X =

X+

. (8)

ICINCO 2009 - 6th International Conference on Informatics in Control, Automation and Robotics

Figure 1: Perspective projection induced by a plane.

Let us now consider a second camera frame denoted

reference camera frame F

∗

in which the coordinates

of the same point P , in a similar manner as before,

are

∗

X =

∗

X+

∗

. (9)

By using Eq. (8) and Eq. (9), it is possible to relate the

coordinates of the same point P between reference F

∗

and current F

camera frames as

X =

∗

⊤

∗

X+

−

∗

⊤

∗

= (10)

∗

X+

∗

However, considering that P lies on a plane Π, the

plane equation applied to the coordinates of the same

point in the reference camera frame gives us

∗

⊤∗

X = d

∗

⇔

∗

⊤∗

X = 1 (11)

where,

∗

⊤

= [n

]

⊤

∈ R

is the unit normal

vector of the plane Π with respect to F

∗

and d

∗

∈ R

the distance from the plane Π to the optical center of

same frame. Thus, substituting Eq. (11) into Eq. (10)

results on

X =



∗

⊤



∗

X =

∗

X (12)

where,

∗

∈ R

3×3

is the so-called Euclidean homog-

raphy matrix. Applying the perspective projection

from Eq. (5) along with the Eq. (6) into the planar

homography mapping deﬁned in Eq. (12), the rela-

tion between pixels coordinates p and

∗

p illustrated

in Figure 1 is obtained as follows

p ∝ K

∗

−1∗

p ∝

∗

p (13)

where, G ∈ R

3×3

is the projectivehomography matrix

and ” ∝ ” denotes proportionality.

3 VISION-BASED AUTONOMOUS

APPROACH AND LANDING

3.1 Visual Tracking

The visual tracking is achieved by directly estimat-

ing the projective transformation between the image

taken from the airborne camera and a given reference

image. The reference images are then the key to re-

late the motion of the aircraft F

, through its airborne

camera F

, with respect to the earth frame F

. For

the PBVS scheme, it is the known pose of the refer-

ence camera with respect to the earth frame that will

allows us to reconstruct the aircraft position with re-

spect to the same frame. What concerns the IBVS,

where the aim is to reach a certain conﬁguration ex-

pressed in terms of the considered feature, the path

planning is then an implicity need of such scheme.

For example, if lines are considered as features, the

path planning is deﬁned as a function of the parame-

ters which deﬁne those lines. In the present case, the

path planning shall be deﬁned by images because it is

the dense information that is used in order to estimate

the projective homography

∗

3.2 Visual Servoing

3.2.1 Linear Controller

The standard LQR optimal control technique was

chosen for the controller design, based on the lin-

earized models of both longitudinal and lateral mo-

tions in Eq. (4). Since not all the states are expected

to be driven to zero but to a given reference, the con-

trol law is more conveniently expressed as an opti-

mal output error feedback. The objective of the fol-

lowing vision-based control approaches is then to ex-

press the respective control laws as a function of the

visual information, which is directly or indirectly re-

lated with the pose of the aircraft. As a consequence,

the pose state vector P = [n,e,d,φ,θ,ψ]

⊤

∈ R

, in

agreement to the type of vision-based control ap-

proach, is given differently from the velocity screw

V = [u,v,w, p,q,r]

⊤

∈ R

, which could be provided

from an existent Inertial Navigation System (INS) or

from some ﬁltering method based on the estimated

pose. Thus, the following vision-based control laws

are more correctly expressed as

u = −k

(P− P

∗

) − k

(V− V

∗

) (14)

where, k

and k

are the controller gains relative to

the pose and velocity states, respectively.

3.2.2 Position-based Visual Servoing

In the position-based, or 3D, visual servoing (PBVS)

the control law is expressed in the Cartesian space

and, as a consequence, the visual information com-

puted into the form of planar homography is used to

reconstruct explicitly the pose (position and attitude).

The airborne camera will be then considered as only

VISION-BASED AUTONOMOUS APPROACH AND LANDING FOR AN AIRCRAFT USING A DIRECT VISUAL

TRACKING METHOD

another sensor that provides a measure of the aircraft

pose.

In the same way that, knowing the relative pose

between the two cameras, R and t, and the planar

scene parameters, n and d, it is possible to compute

the planar homography matrix H it is also possible to

recover the pose from the decomposition of the esti-

mated projective homography G, with the additional

knowledge of the calibration matrix K. The decom-

position of H can be performed by singular value de-

composition (Faugeras, 1993) or, more recently, by an

analytical method (Vargas and Malis, 2007). These

methods result into four different solutions but only

two are physically admissible. The knowledge of the

normal vector n, which deﬁnes the planar scene Π,

allows us then to choose the correct solution.

Therefore, from the decomposition of the esti-

mated Euclidean homography

∗

= K

−1c

∗

K, (15)

both

∗

and

∗

are recovered being respectively,

the rotation matrix and normalized translation vector.

With the knowledge of the distance d

∗

, it is then pos-

sible to compute the estimated rigid-body relation of

the aircraft frame F

with respect to the inertial one

∗



∗



−1



0 1



(16)

where,

corresponds to the pose of the airborne

camera frame F

with respect to the aircraft body

frame F

and

∗

to the pose of the reference camera

frame F

∗

with respect to the earth frame F

. Finally,

without further considerations, the estimated pose

obtained from

and

could then be applied to

the control law in Eq. (17) as

u = −k

(

P− P

∗

) − k

(V− V

∗

) (17)

3.2.3 Image-based Visual Servoing

In the image-based, or 2D, visual servoing (IBVS)

the control law is expressed directly in the image

space. Then, in contrast with the previous approach,

the IBVS does not need the explicit aircraft pose rel-

ative to the earth frame. Instead, the estimated planar

homography

H is used directly into the control law

as some kind of pose information such that reach-

ing a certain reference conﬁguration H

∗

the aircraft

presents the intended pose. This is the reason why an

IBVS scheme needs implicitly for path planning ex-

pressed in terms of the considered features.

In IBVS schemes, an important deﬁnition is that

of interaction matrix which is the responsible to relate

the time derivative of the visual signal vector s ∈ R

with the camera velocity screw

c∗

∈ R

s = L

c∗

(18)

where, L

∈ R

k×6

is the interaction matrix, or the fea-

ture jacobian. Let us consider, for a moment, that

the visual signal vector s is a matrix and equal to the

Euclidean homography matrix

∗

, the visual feature

considered in the present paper. Thus, the time deriva-

tive of s, admitting the vector

∗

n/d

∗

as slowly vary-

ing, is

s =

∗

⊤

(19)

Now, it is known that both

∗

and

∗

are related with

the velocity screw

c∗

, which could be determined

using Eq. (3), as follows

c∗

∗

−1

∗

= (20)



∗

⊤

∗

−

∗

⊤

∗

0 1



from where,

∗

and

∗

. By

using such results back in Eq. (19) results on

∗



∗

⊤



∗

⊤

∗

⊤

(21)

Hereafter, in order to obtain the visual signal vector,

the stacked version of the homography matrix

∗

must be considered and, as a result, the interaction

matrix is given by

s =

∗





I(3)

∗

−

∗1

I(3)

∗

−

∗2

I(3)

∗

−

∗3













(22)

where, I(3) is the 3 × 3 identity matrix and H

the ith column of the matrix as well as n

is the

ith element of the vector. Note that,

ωH is the ex-

ternal product of ω with all the columns of H and

ω× H

= −H

× ω = −

ω.

However, the velocity screw in Eq. (18), as well

as in Eq. (22), denotes the velocity of the reference

frame F

∗

with respect to the airborne camera frame

and viewed from F

which is not in agreement

with the aircraft velocity screw that must be applied

into the control law in Eq. (17). Instead, the veloc-

ity screw shall be expressed with respect to the refer-

ence camera frame F

∗

and viewed from aircraft body

frame F

, where the control law is effectively applied.

In this manner, and knowing that the velocity tensor

c∗

is a skew-symmetric matrix, then

c∗

= −

⊤

c∗

= −

∗c

(23)

ICINCO 2009 - 6th International Conference on Informatics in Control, Automation and Robotics

Now, assuming the airborne camera frame F

rigidly

attached to the aircraft body frame F

, to change the

velocity screw from the aircraft body to the airborne

camera frame, the adjoint map must be applied as

V =

−1

= (24)



⊤

0 0



from where,

⊤

ω and

⊤

−

⊤

ω and, as a result, the following velocity

transformation

∈ R

6×6

is obtained

V =



⊤

−

⊤





(25)

Using the Eq. (25) into the Eq. (22) along with the

result from Eq. (23) results as follows

s =

∗

= −L

∗c

(26)

Finally, let us consider the linearized version of the

previous result as

s− s

∗

− H

∗s

= −L

(P− P

∗

) (27)

where,



0 R



(28)

are the kinematic and navigation equations, respec-

tively, linearized for the same trim point as for the air-

craft linear model [φ,θ,ψ]

⊤

= [0,θ

,0]

⊤

. It is then

possible to relate the pose error P− P

∗

of the air-

craft with the Euclidean homography error

∗

−H

∗s

For the present purpose, the reference conﬁguration is

∗

= I(3) which corresponds to match exactly both

current I and reference I

∗

images. The proposed

homography-based IBVS visual control law is then

expressed as

u = −k

)

†



∗

− H

∗s



− k

(V− V

∗

)

(29)

where, A

†



⊤



−1

⊤

is the Moore-Penrose

pseudo-inverse matrix.

4 RESULTS

The vision-based control schemes proposed above

have been developed and tested in an simulation

framework where the non-linear aircraft model is im-

plemented in Matlab/Simulink along with the control

aspects, the image processing algorithms in C/C++

and the simulated image is generated by the Flight-

Gear ﬂight simulator. The aircraft model considered

Figure 2: Screenshot from the dense visual tracking soft-

ware. The delimited zone (left) corresponds to the bottom-

right image, warped to match with the top-right image. The

warp transformation corresponds to the estimated homogra-

phy matrix.

corresponds to a generic category B business jet air-

craft with 50m/s of stall speed, 265m/s of maximum

speed and 20m wing span. This simulation framework

has also the capability to generate atmospheric condi-

tion effects like fog and rain as well as vision sensors

effects like pixels spread function, noise, colorimetry,

distortion and vibration of different types and levels.

The chosen airport scenario was the Marseille-

Marignane Airport with an nominal initial position

deﬁned by an altitude of 450m and a longitudinal

distance to the runway of 9500m, resulting into a

3 degrees descent for an airspeed of 60m/s. In order

to have illustrative results and to verify the robustness

of the proposed control schemes, two sets of simula-

tions results are presented. The ﬁrst with an initial

lateral error of 50m, an altitude error of 30m and a

steady wind composed by 10m/s headwind and 1m/s

of crosswind. The latter, with a different initial lateral

error of 75m, considers in addition the presence of tur-

bulence. What concerns the visual tracking aspects, a

database of 200m equidistant images along the run-

way axis till the 100m height, and 50m after that,

was considered and the following atmospheric con-

ditions imposed: fog and rain densities of 0.4 and 0.8

([0,1]). The airborne camera is considered rigidly at-

tached to the aircraft and presents the following pose

= [4m, 0m, 0.1m,0,−8 degrees,0]

⊤

. The simula-

tion framework operates with a 50ms, or 20Hz, sam-

pling rate.

For all the following ﬁgures, the results of the two

simulations are presented simultaneously and iden-

tiﬁed in agreement with the legend in Figure 3(a).

When available, the corresponding references are pre-

sented in black dashed lines. Unless speciﬁed, the

detailed discussion of results is referred to the case

without turbulence.

Let us start with the longitudinal trajectory illustrated

in Figure 3(a) where it is possible to verify immedi-

ately that the PBVS results are almost coincident with

the ones where the sensor measurements were consid-

ered ideal (Sensors). Indeed, because the same con-

VISION-BASED AUTONOMOUS APPROACH AND LANDING FOR AN AIRCRAFT USING A DIRECT VISUAL

TRACKING METHOD

trol law is used for these two approaches, the results

differ only due to the pose estimation errors from the

visual tracking software. For the IBVS approach, the

ﬁrst observation goes to the convergence of the air-

craft trajectory with respect to the reference descent

that occurs later than for the other approaches. This

fact is a consequence not only of the limited validity

of the interaction matrix in Eq (27), computed for a

stabilized descent ﬂight, but also of the importance of

the camera orientation over the position, for high al-

titudes, when the objective is to match two images.

In more detail, the altitude error correction in Fig-

ure 3(b) shows then the IBVS with the slowest re-

sponse and, in addition, a static error not greater than

2m as a cause of the wind disturbance. In fact, the

path planning does not contemplates the presence of

the wind, from which the aircraft attitude is depen-

dent, leading to the presence of static errors. These

same aspects are veriﬁed in the presence of turbulence

but now with a global altitude error not greater than

8m, after stabilization. The increasing altitude error

at the distance of 650m before the touchdown corre-

sponds to the natural loss of altitude when proceeding

to the pitch-up, or ﬂare, manoeuvre (see Figure 3(c))

in order to reduce the vertical speed and correctly land

the aircraft. What concerns the touchdown distances,

both Sensors and PBVS results are very close and at

a distance around 330m after the threshold line while,

for the IBVS, this distance is of approximately 100m.

In the presence of turbulence, these distances became

shorter mostly due to the oscillations in the altitude

control during the ﬂare manoeuvre. Again, Sensors

and PBVS touchdown points are very close and about

180m from the runway threshold line while, for the

IBVS, this distance is about 70m.

The lateral trajectory illustrated in Figure 3(e) shows a

smooth lateral error correction for all the three control

schemes, where both visual control laws maintain an

error below the 2m after convergence. Once more, the

oscillations around the reference are a consequence of

pose estimation errors from visual tracking software,

which become more important near the Earth surface

due to the high displacement of the pixels in the im-

age and the violation of the planar assumption of the

region around the runway. The consequence of these

effects are perceptible in the ﬁnal part not only in the

lateral error correction but also in the yaw angle of the

aircraft in Figure 3(f). For the latter, the static error is

also an inﬂuence of the wind disturbance which im-

poses an error of 1 degree with respect to the runway

orientation of exactly 134.8 degrees North.

In the presence of turbulence, Sensors and PBVS con-

trol schemes present a different behavior during the

−10000 −8000 −6000 −4000 −2000 0 2000

−100

100

200

300

400

500

Distance to Touchdown − [m]

Altitude − [m]

Reference

Sensors

PBVS

IBVS

Sensors w/ turb.

PBVS w/ turb.

IBVS w/ turb.

(a) Longitudinal trajectory

−10000 −8000 −6000 −4000 −2000 0 2000

−35

−30

−25

−20

−15

−10

−5

Distance to Touchdown − [m]

Altitude Error −[m]

(b) Altitude error

−10000 −8000 −6000 −4000 −2000 0 2000

−2

Distance to Touchdown − [m]

Pitch angle − [deg]

−10000 −8000 −6000 −4000 −2000 0 2000

Distance to Touchdown − [m]

True airspeed − [m/s]

(d) Airspeed

−10000 −8000 −6000 −4000 −2000 0 2000

−60

−40

−20

Distance to Touchdown − [m]

Lateral Error − [m]

(e) Lateral trajectory

−10000 −8000 −6000 −4000 −2000 0 2000

120

122

124

126

128

130

132

134

136

138

140

Distance to Touchdown − [m]

Yaw angle − [deg]

(f) Yaw angle

Figure 3: Results from the vision-based control schemes

(PBVS and IBVS) in comparison with the ideal situation of

precise measurements (Sensors).

lateral error correction manoeuvre. Indeed, due to the

important bank angle and the high pitch induced by

the simultaneous altitude error correction manoeuvre,

the visual tracking algorithm lost information on the

near-ﬁeld of the camera essential for the precision of

the estimated translation. The resultant lateral error

estimative, greater than it really is, forces the lateral

controller to react earlier in order to minimize such

error. As for the longitudinal case, the IBVS presents

a slower response on position error corrections result-

ing into a lateral error not greater than 8m which con-

trasts with the 4m from the other two approaches.

It should be noted the precision of the dense visual

tracking software. Indeed, the attitude estimation er-

rors are often below 1 degree for transient responses

and below 0.1 degrees in steady state. Depending

on the quantity of information available in the near

ﬁeld of the camera, the translation error could vary

between the 1m and 4m for both lateral and altitude

errors and between 10m and 70m for the longitudinal

distance. The latter is usually less precise due to its

alignment with the optical axis of the camera.

ICINCO 2009 - 6th International Conference on Informatics in Control, Automation and Robotics

100

5 CONCLUSIONS

In the present paper, two vison-based control schemes

for an autonomous approach and landing of an aircraft

using a direct visual tracking method are proposed.

For the PBVS solution, where the vision system is

nothing more than a sensor providing position and at-

titude measures, the results are naturally very similar

with the ideal case. The IBVS approach based on a

path planning deﬁned by a sequence of images shown

clearly to be able to correct an initial pose error and

land the aircraft under windy conditions. Despite the

inherent sensitivity of the vision tracking algorithm to

the non-planarity of the scene and the high pixels dis-

placement in the image for low altitudes, a shorter dis-

tance between the images of reference was enough to

deal with potential problems. The inexistence of a ﬁl-

tering method, as the Kalman ﬁlter, is the proof of the

robustness of the proposed control schemes and the

reliability of the dense visual tracking. This clearly

justify further studies to complete the validation and

the eventual implementation of this system on a real

aircraft.

ACKNOWLEDGEMENTS

This work is funded by the FP6 3rd Call European

Commission Research Program under grant Project

N.30839 - PEGASE.

REFERENCES

Azinheira, J., Rives, P., Carvalho, J., Silveira, G., de Paiva,

E.C., and Bueno, S. (2002). Visual servo control for

the hovering of an outdoor robotic airship. In IEEE In-

ternational Conference on Robotics and Automation,

volume 3, pages 2787–2792.

Behimane, S. and Malis, E. (2004). Real-time image-based

tracking of planes using efﬁcient second-order mini-

mization. In IEEE International Conference on Intel-

ligent Robot and Systems, volume 1, pages 943–948.

Bourquardez, O. and Chaumette, F. (2007a). Visual servo-

ing of an airplane for alignment with respect to a run-

way. In IEEE International Conference on Robotics

and Automation, pages 1330–1355.

Bourquardez, O. and Chaumette, F. (2007b). Visual servo-

ing of an airplane for auto-landing. In IEEE Interna-

tional Conference on Intelligent Robots and Systems,

pages 1314–1319.

Chatterji, G., Menon, P., K., and Sridhar, B. (1998). Vision-

based position and attitude determination for aircraft

night landing. AIAA Journal of Guidance, Control and

Dynamics, 21(1).

Dickmanns, E. and Schell, F. (1992). Autonomous land-

ing of airplanes by dynamic machine vision. IEEE

Workshop on Application on Computer Vision, pages

172–179.

Faugeras, O. (1993). Three-dimensional computer vision: a

geometric view point. MIT Press.

Hamel, T. and Mahony, R. (2002). Visual servoing of

an under-actuated dynamics rigid-body system: an

image-based approach. In IEEE Transactions on

Robotics and Automation, volume 18, pages 187–198.

Kimmett, J., Valasek, J., and Junkins, J. L. (2002). Vi-

sion based controller for autonomous aerial refueling.

In Conference on Control Applications, pages 1138–

1143.

Mahony, R. and Hamel, T. (2005). Image-based visual

servo control of aerial robotic systems using linear im-

ages features. In IEEE Transaction on Robotics, vol-

ume 21, pages 227–239.

Malis, E. (2004). Improving vision-based control using ef-

ﬁcient second-order minimization technique. In IEEE

International Conference on Robotics and Automa-

tion, pages 1843–1848.

Malis, E. (2007). An eﬁcient uniﬁed approach to direct

visual tracking of rigid and deformable surfaces. In

IEEE International Conference on Robotics and Au-

tomation, pages 2729–2734.

Mati, R., Pollini, L., Lunghi, A., and Innocenti, M. (2006).

Vision-based autonomous probe and drogue aerial re-

fueling. In Conference on Control and Automation,

pages 1–6.

Proctor, A. and Johnson, E. (2004). Vision-only aircraft

ﬂight control methods and test results. In AIAA Guid-

ance, Navigation, and Control Conference and Ex-

hibit.

Rives, P. and Azinheira, J. (2002). Visual auto-landing of

an autonomous aircraft. Reearch Report 4606, INRIA

Sophia-Antilopis.

Rives, P. and Azinheira, J. (2004). Linear structure follow-

ing by an airship using vanishing point and horizon

line in visual servoing schemes. In IEEE International

Conference on Robotics and Automation, volume 1,

pages 255–260.

Sharp, C., Shakernia, O., and Sastry, S. (2002). A vision

system for landing an unmanned aerial vehicle. In

IEEE International Conference on Robotics and Au-

tomation, volume 2, pages 1720– 1727.

Silveira, G., Azinheira, J., Rives, P., and Bueno, S. (2003).

Line following visual servoing for aerial robots com-

bined with complementary sensors. In IEEE Interna-

tional Conference on Robotics and Automation, pages

1160–1165.

Silveira, G. and Malis, E. (2007). Real-time visual tracking

under arbitrary illumination changes. In IEEE Con-

ference on Computer Vision and Pattern Recognition,

pages 1–6.

Vargas, M. and Malis, E. (2007). Deeper understanding of

the homography decomposition for vision-base con-

trol. Research Report 6303, INRIA Sophia-Antipolis.

VISION-BASED AUTONOMOUS APPROACH AND LANDING FOR AN AIRCRAFT USING A DIRECT VISUAL

TRACKING METHOD

101