REAL TIME GRASPING OF FREELY PLACED CYLINDRICAL

OBJECTS

Mario Richtsfeld, Wolfgang Ponweiser and Markus Vincze

Institute of Automation and Control, Vienna University of Technology

Gusshausstr. 27-29, Vienna, Austria

Keywords:

Service robotics, laser-range scanning, object detection, task planning.

Abstract:

In the near future, service robots will support people with different handicaps to improve the quality of their

life. One of the required key technologies is to setup the grasping ability of the robot. This includes an

autonomous object detection and grasp motion planning to fulﬁl the task of providing objects from any position

on a table to the user. This paper presents a complete system, which consists of a ﬁxed working station

equipped with a laser-range scanner, a seven degrees of freedom arm manipulator and an arm prothesis as

gripper. The contribution of this work is to use only one sensor system based on a laser-range scanning head

to solve this challenge. The goal is that the user can select any deﬁned object on the table and the robot arm

delivers it to a target position or to the disabled person.

1 INTRODUCTION

At the beginning of the 1970’s the development

of service and rehabilitation robots started to sup-

port disabled people in their daily life. The

goal is to make them more independent. To-

day we differ between ﬁxed systems, in which

an industrial robot is mounted on a working sta-

tion and mobile systems, e.g. wheelchair mounted

manipulators, like MANUS (Mokhtari, 2001) or

FRIEND-I (Martens, 2001) and FRIEND-II (Ivlev,

2005). Popular ﬁxed systems are e.g. De-

Var (Van der Loos, 1995), ProVar (Van der Loos,

1999), RAID (Eftring, 1994), MASTER-RAID (Dall-

away, 1995) or CAPDI (Casals, 1999).

Our vision is a fully autonomous mobile robot,

which is able to detect, grasp and manipulate any kind

of object. One of the key challenges of this work is

the robust perception of objects. This challenge is

analyzed by a ﬁxed setup consisting of a laser-range

scanner and a robot arm. We use an AMTEC

robot

arm with seven degrees of freedom, which is used for

object grasping and manipulation. The joint setup is

assembled similar to a human arm. The robot arm

is equipped with a hand prothesis from the company

Otto Bock

, which we are using as gripper. It is

http://www.amtec-robotics.com

http://www.ottobock.de/

thought that elderly persons will accept this type of

gripper more easily than an industrial gripper, due to

the form and the optical characteristics.

The outline of the paper is as follows: In the next

section the state of the art of grasp robot systems,

grasping technology and object perception based on

structure in 2-d and 3-d is presented. Section 3 in-

troduces our robotic system and its components. Sec-

tion 4 describes the object identiﬁcation to calculate

the object position and Section 5 details the grasping

and manipulation. Section 6 gives some experimental

results during a live demo presentation and Section 7

ﬁnally concludes the paper.

2 STATE OF THE ART

In the early 1970’s one of the ﬁrst wheelchair

mounted manipulator was developed at the V.A. Re-

habilitation Engineering (formerly Prosthetics) cen-

ter (Prior, 1993). From 1983 to 1988 the mobile

manipulator MoVAR (Van der Loos, 1995) was de-

veloped. This PUMA-250 robot was instrumented

with a camera for remote sensing, a six-axis force

sensor and a gripper with ﬁnger pad-mounted prox-

imity sensors. A nice overview of different systems,

such as the Wolfson-Robot and the Wessex-Robot is

given by Hagen and Hillmann (Hagan, 1997). Up

165

Richtsfeld M., Ponweiser W. and Vincze M. (2008).

REAL TIME GRASPING OF FREELY PLACED CYLINDRICAL OBJECTS.

In Proceedings of the Fifth International Conference on Informatics in Control, Automation and Robotics - RA, pages 165-170

DOI: 10.5220/0001481101650170

 SciTePress

to now a number of scientists have been working

on the same idea to develop a wheelchair mounted

robot or a mobile robot system with arms to han-

dle objects and assist elderly and handicapped per-

sons, e.g. (Martens, 2001), (Volosyak, 2005). In the

FRIEND systems (Martens, 2001), (Ivlev, 2005) the

robot arm is controlled by a PC, which is ﬁxed on the

backside of the wheelchair. Both systems use a stereo

camera system for object detection. The user inter-

action is based on a LC-display. The object must be

placed on a predeﬁned position on a tray, mounted at

the front side of the wheelchair. A successful execu-

tion of the grasping task in this system is only pos-

sible for similar types of objects. Additionally they

developed a ”smart tray” that is used in combination

with the vision sensors. This ”smart tray” measures

the weight and the position of objects with a matrix

foil position sensor.

In comparison to the FRIEND systems, Saxena et

al. (Saxena, 2006) developed a learning algorithm that

predicts the grasp position of novel objects as a func-

tion of 2-d images, without building an explicit 3-d

model of the object. This algorithm is trained via su-

pervised learning using synthetic images for the train-

ing set. The work focuses on the task of identifying

grasping positions without taking any complex ma-

nipulation tasks into account. A similar system de-

scribes Miller et al. (Miller, 2003). Their work speci-

ﬁes an automatic grasp planning system for hand con-

ﬁgurations using shape primitives. By modeling an

object as a sphere, cylinder, cone or box. They also

use a set of rules to generate grasp positions.

In our case the vision task is to detect edges of

objects that indicate grasp points. Accurate 3-d data

is achieved by direct depth measurements, like laser-

range scanning. In the range images, grasp points are

indicated by object edges and grasp surface patches.

Wang et al. (Wang, 2005) developed a general frame-

work of automatic grasping of unknown objects by

incorporating a laser scanner and a simulation envi-

ronment. Their algorithms need a lot of time to de-

tect grasp points. To aid industrial bin picking tasks

Boughorbel et al. (Boughorbel, 2007) developed a

system that provides accurate 3-d models of parts and

objects in the bin to realize precise grasping opera-

tions. Due to their superquadrics based object mod-

elling approach only rotation-symmetric objects can

be used. To that effect Biegelbauer et al. describes

a new approach of a hierarchical RANSAC search

to obtain fast detection results of objects, which are

modeled using approximated Superquadrics (Biegel-

bauer, 2007).

One of the most fundamental techniques for edge

detection in range images is the scan line approxima-

tion (Jiang, 1999). It is well known and more ef-

ﬁcient than the standard Canny (Canny, 1986) algo-

rithm. The raw data points are approximated by a set

of bivariate polynomial functions, in which the dis-

continuity of the ﬁtted functions indicate the edge po-

sition. Katsoulas (Katsoulas, 2004) proposed an im-

proved scan line approach by using an additional sta-

tistical merging step for a better handling of outliers.

Based on these techniques we developed a 3-d edge

detection method that enables a faster cylinder ﬁt in

3-d range data.

3 SYSTEM APPROACH

The goal is, that the user can select any object on a

table and the robot arm delivers it to a deﬁned posi-

tion or to the disabled person. The main challenges

to solve are the robust detection of edges and their in-

terpretation as grasping points. Our approach is based

on scanning the objects by a rotating laser-range scan-

ner and execution of subsequent path planning and

grasping motion. Hence the system consists of a

pan/tilt-mounted red-light laser and scanning camera

and a seven degrees of freedom robot arm, which

is equipped with a human like prosthesis hand (see

Fig. 1).

Figure 1: Overview of the system components and their in-

terrelations.

3.1 Laser-Range Scanner

The laser-range scanner records a snap-shot of the ob-

ject scene with the help of a pan/tilt-unit. At present,

it is mounted on a table. We are working to miniatur-

ize the laser-range scanner to mount it on the shoul-

der of the robot later. A high resolution sensor is

needed in order to detect a reasonable number of

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

166

edge-points of the objects with the required accuracy.

The laser-range scanner used for this work consists

of a red-light LASIRIS laser from StockerYale

with

635nm and a MAPP2500 CCD-camera from SICK-

IVP

mounted on a pan/tilt-unit (PowerCube Wrist

from AMTEC robotics). With the help of a cylin-

der lens, the laser-light is expanded and moves hor-

izontally over the scene of interest. The camera grabs

the laser-light proﬁles and extracts the laser-lines with

the integrated microprocessor. The 3-d data is trans-

formed to the world coordinate system. Finally the

result can be displayed as a point cloud (see Fig. 2).

Figure 2: Exposure of the raw point cloud with 75.863

voxel. The two shadows from laser and camera are clearly

visible.

3.2 Robot Arm and Gripper

For this work we use the ”Light Weight Arm 7 DOF”

from AMTEC robotics and a hand prothesis from

Otto Bock as gripper. The robot arm exhibits seven

degrees of freedom with a joint conﬁguration simi-

larly to the human arm (shoulder, elbow and wrist).

The seventh degree of freedom is required to enable

complex object grasping and manipulation and allow

for some ﬂexibility to avoid obstacles. The prosthe-

sis as end effector is selected due to the integrated

force sensors as well as its increased acceptance of

elderly and handicapped persons. It has three active

ﬁngers, the thumb, the index ﬁnger and the middle ﬁn-

ger. The last two ﬁngers are just for cosmetic reasons.

Since, they have no active function in the grasping

process their uncontrollable behavior must be consid-

ered, which reduces the grasping radius (see Fig. 1).

As a huge advantage the integrated tactile sensors are

used to detect a potential sliding of objects, which ini-

tializes a readjustment of the ﬁngers.

3.3 Operation Sequence

The ﬁrst step is to scan the scene on the table by the

laser-range scanner. The camera converts the laser-

http://www.stockeryale.com/index.htm

http://www.sickivp.se/sickivp/de.html

proﬁles to a 3-d point cloud, which can be visualized.

Now the user can select the desired object. The de-

veloped algorithm analyzes the point cloud and calcu-

lates the position of the searched object. A commer-

cial path planning tool from AMROSE

calculates the

trajectory to grasp the object. Before the robot arm

delivers the object, the user can check the calculated

trajectory in a simulation sequence. Then the robot

arm executes the off-line programmed trajectory. The

algorithm is implemented in C++. For displaying the

results the Visualization Tool Kit (VTK)

is used.

4 OBJECT IDENTIFICATION

The main goal of our work is to robustly detect cylin-

drical objects in the recorded point cloud in real

time. Robustness includes the positive detection of

deﬁned objects despite any noise and outliers in a

point cloud, which can be caused by specular surfaces

(see Fig. 2, edges of the objects). To reduce complex-

ity we only consider cylindrical objects for object de-

tection for this work. An additional challenge is the

complex interaction between the different operation

parts. Finally to keep the standby time acceptable for

the user the complete operating cycle should be ﬁn-

ished within 20sec.. This time limit is challenging

since usual object detection starts with an exhaustive

segmentation step. As an example, object segmen-

tation alone by recursive ﬂood-ﬁlling with region-

octree (Burger, 2007) of the desired table scene takes

more than 30sec. (see Fig. 3). Thus a faster solution

must be found. One alternative, which we exploited

in our work is based on well investigated curvatures.

Fig. 4 presents the steps of the fast object detection

Figure 3: Segmentation of the different objects by recursive

ﬂood-ﬁlling. Images are best viewed in color.

method. In the ﬁrst step, the ”raw data preprocessing

and vector estimation” the raw data points are pre-

processed with a low pass ﬁlter to reduce any noise.

http://www.amrose.dk/

Freely available open source software,

http://public.kitware.com/vtk.

REAL TIME GRASPING OF FREELY PLACED CYLINDRICAL OBJECTS

167

One of the most time consuming calculations is the

normal vector estimation based on the orientation of

the local neighborhood of 20mm, for what a region-

octree is used. These vectors are required to compute

the axis of the cylinder objects. A lot of tests have

shown that for a neighborhood of 20mm a reason-

able accuracy can be achieved, while the calculation

time stays acceptable. The ”range image segmenta-

Figure 4: Flow chart of the object detection approach.

tion” starts by detecting the surface of the table with

a RANSAC (Fischler, 1981) based plane ﬁt. Then we

analyze the curvature of the remaining points to ﬁlter

neighbouring voxels with an angle difference between

±78

◦

and ±90

◦

(see Fig. 5) .

The ”fast cylinder ﬁt” starts with a RANSAC based

circle ﬁt. Randomly three high curvature points are

picked. The resulting circle is extended to a poten-

tial cylinder along its circumscribed axis down to the

table. For every vicinity point, within a deﬁned dis-

tance of 2mm of the calculated cylinder barrel, the

normal distance to the cylinder barrel is calculated.

The trial with the lowest mean of these distances is se-

lected as cylinder (see Fig. 6). For comparison Jiang

et al. (Jiang, 2005) published a method for 3-d circle

Figure 5: The acquired range image of the current table

scene. The points with a high curvature are marked with

blue dots. Images are best viewed in color.

Figure 6: Detected objects in the table scene (blue cylinder

- spray-on glue, red cylinder - beverage can, green points -

rigid obstacles). Images are best viewed in color.

ﬁtting. They reduce the number of local minima, but

the error function is no more Euclidian. Here another

simple proposal with an Euclidian function is used.

For an explicit description, the raw data points of a

proﬁle scan are deﬁned as (xi, yi, zi), n is the number

of voxels and (xa, ya, za) is the circle’s center. The

resulting error function e is:

e =

∑

i=1

(xi − xa)

+ (yi − ya)

+ (zi − za)

) − r

(1)

The error must be smaller or equal than a deﬁned

threshold. In our case we use a distance of 2mm:

|e| ≤ 2mm (2)

In the last step of Fig. 4 ”Transmission of the Calcu-

lated Object Position to the Path Planning Tool” the

calculated object position in the actual environment

model for collision avoidance has to be transmitted to

the path planning tool. This 3-d mesh is generated

by using all objects besides the target object, based

on the triangles calculated by a DeLaunay triangula-

tion (O’Rourke, 1998) This step is important to enable

a collision free robot trajectory.

5 OBJECT GRASPING AND

MANIPULATION

The task of this part of our work is to calculate a col-

lision free robot path and to execute the grasping ac-

tivity safely. The ﬁrst step is performed by the path

planning tool from AMROSE. The input to this tool is

the detected object pose, the environment model and

a transformation between the robot coordination sys-

tem and the range scanner coordinate system. The

output is a collision free trajectory to the desired ob-

ject. Before the robot execution is approved, the user

can check a simulation of the calculated trajectory and

decide, if it is safe enough to handle the object (see

Fig. 7 and Fig. 8). After the robot approaches the user

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

168

Figure 7: Visualization of the trajectory by a simulation

tool. The white cylinder is the grasping object. The green

cylinder (= 2.

grasping object) and the blue objects are

the obstacles. Images are best viewed in color.

Figure 8: Real position of the robot arm after the approach

trajectory.

can initiate the closing of the gripper. As soon as the

gripper encloses the object, the robot motion to the

transfer point starts. Finally the desired object can be

placed at a deﬁned position or directly handed over to

the user.

The calculation of the object detection and local-

ization is performed by a PC with 1.8GHz Pentium

IV processor and takes less than 12sec. depending on

the range image size. The reliability depends on the

ambient light, object surface properties, laser beam

reﬂections and vibrations. Therefore, the laser-range

scanner must be conﬁgured to the respective environ-

ment. By using an additional red-light ﬁlter the im-

pact of light or reﬂections can be minimized.

6 EXPERIMENTS

The entire system exhibits its robust behavior and has

been evaluated at a live demo presentation

in front of

more than 1000 college students. During the demon-

stration day about 50 runs were performed. The main

problem that rarely appeared was a malfunction of

the path planning tool, because no suitable trajectory

could be found. Whereby the path planning had to be

restarted. Sometimes the last two ﬁngers, which re-

duce the grasping radius (see Fig. 1), shift the grasp

object, but without a ﬁnal effect on the success of the

grasping process. Tab. 1 shows a short analysis of the

arisen problems within 50 runs. The recoginition of

the cylindrical objects fails at strong environmental

inﬂuences by the ambient light.

The autonomous grasping function should be able

to ﬁnd and grasp a cylindrical object in a deﬁned area.

When objects are positioned closer to each other, the

autonomous grasping function show up difﬁculties to

ﬁnd the correct object. A minimum distance of 20mm

(this distance is equal to the diameter of the thumb

of the hand prothesis) has to be observed between the

objects.

Table 1: Evaluation of the arisen problems in percent [%] at

50 runs.

Arison Problems Number of Events Percent [%]

Path Planning 11 22%

Hand Prosthesis 4 8%

Object Recognition 2 4%

Sum 17 34%

7 CONCLUSIONS AND FUTURE

WORK

This paper presents an approach of a robot system

equipped with a laser-range scanner to get high accu-

racy table scene sensing. It shows that feature detec-

tion, in our case we only consider cylindrical objects,

is a faster way (12sec.), than usual object segmenta-

tion (more than 30sec.) by a ﬂood-ﬁlling recursive

function. The presented method performs with very

high reliability. Thus the approach for object detec-

tion and localization is well suited for use in related

applications under difﬁcult conditions.

A seven degrees of freedom arm manipulator and

an arm prothesis as gripper are used to grasp and de-

liver the desired object. The goal of this system is

http://www.yo-tech.at/

REAL TIME GRASPING OF FREELY PLACED CYLINDRICAL OBJECTS

169

to analyze the feasibility and reliability of object de-

tection, which could be shown at a live demo. The

cylinder detection approach can be extended to detect

any type of object, since it is based on a grouping of

high curvature points. This grasping approach can be

applied for any kind of geometrical ﬁgures. This will

expand the application to other tasks.

In the future, the robot arm will be installed on a

mobile robot and for the object detection we calculate

the grasping points of novel objects. This includes

the revision of the path planning tool and a segmenta-

tion of sharp curvature points to speed up the method.

Summarizing, this work illustrates that the concept of

a 3-d vision guided robot arm can be adopted to many

applications and has high potential to enable a more

complex system. We will also deal with the devel-

opment and the prototypes integration of a new laser

range sensor with additional two cameras for stereo-

vision to increase the robustness and predictability of

the object detection system.

REFERENCES

Biegelbauer, G.; Vincze, M. (2007). Efﬁcient 3d object de-

tection by ﬁtting superquadrics to range image data

for robot’s object manipulation. In International Con-

ference on Robotics and Automation / ICRA, pages

1086–1091. IEEE Press.

Boughorbel, F.; Zhang, Y. (2007). Laser ranging and video

imaging for bin picking. In Assembly Automation, vol-

ume 23, pages 53–59.

Burger, W.; Burge, M. (2007). Digital Image Processing

- An Algorithmic Introduction Using Java. Springer,

UK, London, 1st edition.

Canny, J. (1986). A computational approach to edge detec-

tion. In Transactions on Pattern Analysis and Machine

Intelligence, volume 8, pages 679–698. IEEE Press.

Casals, A.; Merchan, R. (1999). Capdi: A robotized kitchen

for the disabled and elderly people. In Proceedings

of the 5th European Conference for the Advancement

Assistive Technology / AAATE, pages 346–351.

Dallaway, J.L.; Jackson, R. (1995). Rehabilitation robotics

in europe. In Transactions on Rehabilitation Engi-

neering, pages 33–45. IEEE Press.

Eftring, H. (1994). Robot control methods and results from

user trials on the raid workstation. In Proceedings

of 4th Int. Conf. on Rehabilitation Robotics / ICORR,

pages 97–101.

Fischler, M.A.; Boles, R. (1981). Random samples and con-

sensus: A paradigm for model ﬁtting with applications

to image analysis and automated cartography. In Com-

munications of the ACM, volume 24, pages 381–395.

Hagan, K.; Hillman, M. (1997). The design of a wheelchair

mounted robot. In Colloquium on Computers in the

Service of Mankind: Helping the Disabled, pages 1–

Ivlev, O.; Martens, C. (2005). Rehabilitation robots friend-i

and friend-i with the dexterous lightweight manipula-

tor. volume 17, pages 111–123. IOS Press.

Jiang, X.; Cheng, D.-C. (2005). Fitting of 3d circles and el-

lipses using a parameter decomposition approach. In

5th International Conference on 3cartography Imag-

ing and Modeling / 3DIM, pages 103–109. IEEE

Press.

Jiang, X.; Bunke, H. (1999). Edge detection in range im-

ages based on scan line approximation. In Computer

Vision and Image Understanding, volume 73, pages

183–199.

Katsoulas, D.; Werber, A. (2004). Edge detection in range

images of piled box-like objects. In Conference on

Pattern Recognition, volume 2, pages 80–84.

Martens, C.; Ruchel, N. (2001). A friend for assisting hand-

icapped people. In Robotics and Automation Maga-

zine, volume 8, pages 57–65. IEEE Press.

Miller, A.T.; Knoop, S. (2003). Automatic grasp planning

using shape primitives. In International Conference

on Robotics and Automation / ICRA, volume 2, pages

1824–1829. IEEE Press.

Mokhtari, M.; Abdurazak, B. (2001). Assistive technol-

ogy for the disabled people: Should it work? the

french approach. In International Journal of Assistive

Robotics and Mechatronics, volume 2, pages 26–32.

O’Rourke, J. (1998). Computational geometry in C. Univ.

Press, Cambridge, 2nd edition.

Prior, S.D.; Warner, P. (1993). Wheelchair-mounted robots

for the home environment. In Conference on Intel-

ligent Robots and Systems / IROS, volume 2, pages

1194–1200. IEEE Press.

Saxena, A.; Driemeyer, J. (2006). Learning to grasp novel

objects using vision. In RSS Workshop on Manipula-

tion for Human Environments.

Van der Loos, H. (1995). Va/stanford rehabilitation robotics

research and development program: Lessons learned

in the application of robotics technology to ﬁeld of

rehabilitation. In Transactions on Rehabilitation En-

gineering, volume 3, pages 46–55. IEEE Press.

Van der Loos, H.; Wagner, J. (1999). Provar assistive robot

system architecture. In International Conference on

Robotics and Automation / ICRA, volume 1, pages

741–746. IEEE Press.

Volosyak, I.; Ivlev, O. (2005). Rehabilitation robot friend-

ii - the general concept and current implementation.

In 9th International Conference on Rehabilitation

Robotics / ICORR, pages 540–544.

Wang, B.; Jiang, L. (2005). Grasping unknown objects

based on 3d model reconstruction. In Proceedings

of International Conference on Advanced Intelligent

Mechatronics / ASME, pages 461–466. IEEE Press.

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

170