![](bg2.png)
formance (accuracy and robustness) under various
motions and different levels of illumination, and (4)
we compare our hybrid tracking method with edge
based tracking, implemented by adapting Lie algebra
tracking method (Drummond and Cipolla, 1999).
In the remaining sections of this paper, we present
related current state of the art approach of combined
edge and feature based tracking, and review computer
vision methods for specular objects in Section 2. Sec-
tion 3 presents our tracking method that integrates
keypoints and edges in 2D-3D registration. Section
4 describes experimental setup used to validate the
proposed method, and compares tracking results with
edge based methods. Finally we summarize and con-
clude the paper in Section 5.
2 PREVIOUS WORK
In this section, we review pose estimation and struc-
ture reconstruction methods that exploit specular cues
of the object, and briefly explore model-based track-
ing methods which utilize edge and point features of
an object.
2.1 Pose Estimation from Specular Cues
In the context of pose estimation and surface recon-
struction, there exist several contributions that deal
with specular objects. The Theory of a specular ge-
ometry describes that image features exist as either
real or virtual (Oren and Nayar, 1995). Real fea-
tures are directly used by vision algorithms such as
matching, tracking and structure from motion. Vir-
tual features (specular cues) are specular reflections
of a scene or an object features under change of view-
point. Specular surfaces such as a glass and a smooth
metal create ambiguity (actual scene point or reflec-
tion of another scene point) for visual interpretation.
Various methods have been proposed to utilize
these virtual features. For example, shape and re-
flectance parameters can be simultaneously estimated
from multiple views of an object made of single ma-
terial with known lighting (Yu et al., 2004). Phong
reflectance model is used to compute reflectance of
the object shape modelled with triangular mesh, and
minimize non-linear least square cost function over
the shape and reflectance parameters. When the ob-
ject is in motion, specular reflections produce 2D im-
age motion (specular flow) (Oren and Nayar, 1995).
The specular flow is mathematically related to the
3D structure of textured object or scene (Roth and
Black, 2006) and provided a parametric mixture mod-
els for recovery of a surface. Most methods of shape
from specular reflection assume limited case of sur-
faces, in which its structure is known or qualitatively
sparse, and the environment is calibrated. In contrast
(Vasilyev et al., 2008) presented reconstruction ap-
proach that targets general surfaces under unknown
real-world environments. They recovered 3D shape
from optical flow induced by relative motion between
a specular object, an observer and their environment.
Similarly, (Adato et al., 2010) presented variational
optical flow technique, which accounts for character-
stics of specularity including parabolic singularities
related to surface curvature which are hard to detect.
On the other hand, an object tracking with specu-
lar highlights (Gouiff`es et al., 2006) exploits Phong’s
model by approximatinggeneral photometric changes
with a continuous and differentiable function, approx-
imated with first order Taylor series at a pixel point in
the neighbourhood. This approach extends sparse op-
tical flow tracking method such as (Shi and Tomasi,
1994), by compensating for illumination changes and
specular highlights. In the context of pose estima-
tion, unlike classical methods which discard lighting-
information, (Lagger et al., 2008) refined coarse pose
estimates by incorporating lighting information in
texture and specular cues to improveaccuracy of stan-
dard template matching algorithm. In this approach
environment map is retrieved from the specular pixels
of shiny objects and registration is performed in both
image and lighting environment space.
A practical localization and pose estimation
method is demonstrated by (Chang et al., 2009).
They exploited environment map, specular reflections
and specular flow, handling partial occlusions, back-
ground clutter and inter-reflections. Their method re-
quires to use a calibration object such as a mirror
sphere in the target scene to capture the environment
map. It is, however unsuitable for human inaccessible
environments such as satellite orbits.
Despite existence of several physical and geomet-
ric reflectance models of specular objects, they are
rarely exploited for real world problems, mainly be-
cause of their complexity and underlying assumptions
such as placement of a calibration object in the scene.
For a space object such as a satellite, the appropri-
ate reflectance model consists of specular lobe, spec-
ular spike and diffuse reflections. In the presence
of geometric model of the satellite however, specular
highlights, keypoints and edges of the satellite struc-
ture can be better utilized by integrating into standard
model based tracking.
2.2 Feature-based Tracking
Edges of an image are the most suitable cues for 3D
Monocular3DPoseTrackingofaSpecularObject
459