across multiple views. The door detection relies on
a binocular pan-tilt camera system whereas our ap-
proach uses an omnidirectional camera. An omni-
directional view offers the advantage that an initial
scan of the environment for doors executed by ro-
tating the robot base becomes obsolete. In addition
the omniview guarantees that the door remains visi-
ble throughout the entire door traversal whereas with
a conventionalperspective camera the door eventually
leaves the field of the view such that the final stage of
door traversal is performed open loop. The omniview
also offers an advantage in scenarios with semi open
doors in which the robot still detects the door in the
rear view after it has passed the door leaf.
Omnidirectional vision for door traversing has
also been investigated by (Patel et al., 2002). How-
ever the suggested solution is based on depth infor-
mation obtained from a laser sensor to guide a mobile
platform through a doorway.
To our best knowledge there is still no proposed
solution that handles the three problems of door de-
tection, localisation and door traversal in a coherent
purely vision based framework. The majority of pro-
posed solutions rely on range sensors in one way or
another. Even though the underlying methods for im-
age processing and door frame recognition are stan-
dard, our approach is novel as it provides a robust
and coherent solution to the entire door detection and
navigation problem relying in omnidirectional vision
only.
Images contain a large amount of information
which necessitates the filtering, extraction and inter-
pretation of those image features that are relevant to
the task. Similar to other approaches in the past our
door detection scheme relies on a door frame model
composed of two vertical door posts in conjunction
with a horizontal top segment. The image process-
ing of edges involves edge detection, thinning, gap
bridging, pruning and edge linking. Individual edge
segments are aggregated into lines by means of line
approximation, line segmentation, horizontal and ver-
tical line selection and line merging. Finally the lines
extracted from the omnidirectional image are com-
pared with the door frame model which upon success-
fully matching constitute door hypotheses. These hy-
potheses are tracked over multiple frames and even-
tually confirmed. The position of the door relative to
robot frame is estimated by a Kalman filter that aggre-
gates the robot motion with the door perception. The
vision based door recognition and traversal problem
is structured into the three steps: 1) door detection 2)
door localisation and tracking and 3) door traversal
which are discussed in the three following sections
and are illustrated in figure 1. Section 4 reports ex-
Figure 1: A) Vision based door detection, door localisation
and tracking B) Door traversal by visual servoing.
perimental results of the door traversal in our office
environment. The paper concludes with a summary
and outlook in section 5.
2 DOOR DETECTION
The mobile robot Pioneer 3DX is equipped with an
omnidirectional camera which provides a 360
◦
view
of the scene. Catadioptric cameras employ a com-
bination of lenses and mirrors. Our camera obeys
the single viewpoint property, which is a requirement
for the generation of pure perspective images from
the sensed images. A formal treatment of catadiop-
tric systems is provided by (Baker and Nayar, 1998;
Geyer and Daniilidis, 2001).
The robot navigates through the environment by
means of a topological map. The door detection al-
gorithm is designated to detect doors from arbitrary
robot view points, including lateral and rear views.
The door detection relies on door frame recogni-
tion and thus rests on the reasonable assumption that
the door frame contrasts with the surrounding back-
ground. The map provides no prior information on
door locations, however in conjunction with a mis-
sion plan it enables the robot to either traverse the left
or right of two opposite doors in a corridor.
The door detection is composed into three subse-
quent fundamental steps: image processing, line pro-
cessing and door frame recognition. The image pro-
cessing is executed in the image space; hence, the pro-
cessed entities are pixels. The line processing is per-
formed in Cartesian coordinates and the entities han-
dled are lines. Finally, the detected lines and their spa-
tial relationship are interpreted to recognize the door.
Figure 2 illustrates the visual door detection with piv-
otal processing steps.
IMAGE SIGNAL PROCESSING FOR VISUAL DOOR PASSING WITH AN OMNIDIRECTIONAL CAMERA
473