![](bg2.png)
tance including SIFT features computation and how
to obtain 3-D information. Section 3 shows results
of fundamental experiments. Finally, conclusion and
future works are provided.
2 OBSERVATION-BASED
ASSISTANCE
2.1 Overview of Cooperation
Figure 2 shows an overview of cooperation with
observation-based assistance for the object handling.
The cooperation is made by the following procedure.
(a) At first, the observing robot finds the working
robot with its hand and the target object in the sur-
rounding environment. This study assumes that
the observing robot obtains SIFT features for the
working robot in advance. The observing robot,
therefore, can detect regions for SIFT features
matching to those for the working robot. The
region that has a larger number of the matching
features than a threshold can be extracted as the
working robot. The target object can be detected
in the same way.
(b) After detecting the working robot with its hand
and the target object, the observing robot changes
the viewing location. The observing robot can ob-
tain SIFT features for them in the same way to
(a), and their correspondences. Their 3-D posi-
tions are able to be computed from the correspon-
dences in two or more views based on stereo vi-
sion theory. The observing robot then sends the
relative position information of the target object
to the working robot by a communication.
(c) When the working robot receives the relative po-
sition data of the target object to itself, it starts
planning the path of the hand to the target object.
(d) Then the working robot can control hand motion
based on the planned path. The joint angles at
each time can be calculated using Jacobian which
is computed from movement of the hand in a sam-
pling cycle time.
(e) The observing robot continuously observes hand
movementof the working robot. During the work-
ing robot moves the hand, the observing robot de-
tects 3-D information of the hand motion; that is
possible even though the observing robot stays at
the same position. Then the hand movement in-
formation is sent to the working robot by a com-
munication.
(f) According to the hand movement information re-
ceived form the observing robot, the working
robot corrects the path of hand motion if the cur-
rent path has an error to reach the target object.
Then it updates the control of hand motion as de-
scribed in (d).
In this procedure, the working robot continuously
moves its hand with the correction of the path in the
loop of (d) and (f). The observing robot also keeps
tracking the working robot and its hand motion to ob-
tain more correct motion data in the process of (e).
Updated data are then sent to the working robot.
The following sections describe key functions for
these processes: the object and position detection for
the observing robot by the use of SIFT, and the hand
motion control for the working robot.
2.2 Detection of SIFT Features
SIFT is capable of robust detection of feature points
in an image. It is also able to describe quantities of
detected features to the change of scale, illumination,
and rotation of image robustly. It is, therefore, useful
for object detection and recognition.
The processes of the detection of SIFT features
consist of extraction of feature points, localization,
computation of orientation, and description of quan-
tities of features. In the process of the extraction of
feature points, DoG (Difference of Gaussian) is used
for searching local maxima to detect the positions and
scales of features(D. G. Lowe, 1999). Some features
are then picked up from them by the process of local-
ization. The orientations for those features are then
computed, and their quantities are described.
To describe quantities of features based on the ori-
entation, surrounding region divided by 4×4 blocks at
a feature point is rotated to the direction of the orien-
tation. Making a histogram on 8 directions for each
block produces a 128(4 × 4 × 8)-dimensional feature
vector. The quantity of SIFT feature is represented by
this vector.
2.3 Object Detection
Let us suppose that the SIFT features of the working
robot, hand of its arm, and target object to be manip-
ulated are initially given to the observing robot with
their registered images. In the beginning of obser-
vation, the observing robot looks around and detects
corresponding points on the SIFT features of the ob-
ject in captured images.
In order to find the corresponding points, the 128-
dimensional feature vector, which is described in Sec-
Observation-basedAssistancebyMobileRobotforObjectHandlingofitsPartnerRobot
459