Progression of Human Hand Trajectory Variabilities

during a Pick-and-Place Task

Kolja Kühnlenz

, Sergej Hermann

, Kevin Kalb

, Lucas Marschollek

and Barbara Kühnlenz

Robotics Research Lab, Department of Electrical Engineering and Computer Science,

Coburg University of Applied Sciences and Arts, D-96450 Coburg, Germany

Institute for Cognitive Systems, Department of Electrical Engineering and Information Technology,

TU Munich, D-80290 Munich, Germany

Keywords: Robotics, Human-robot Interaction, Human Factors.

Abstract: This paper investigates the progression of human hand trajectory variabilities during a pick-and-place task. A

user study is conducted and human hand positions are tracked optically. Standard deviations of human hand

positions over all trajectories within a trial are computed point-wise orthogonally to the direct path between

start and goal positions. Statistical tests reveal a decrease of standard deviations from hand start to goal

positions. Moreover, stronger variations of standard deviations are noted in during the center part of the

trajectories. Contrary to expectations, a longitudinal study design does not reveal learning effects in terms of

reduction of trajectory variabilities. The results suggest, that uncertainties of human hand positions increase

with the distance to a goal location and could constitute a larger risk for collisions within a cooperative human-

robot pick-and-place scenario, e.g. assembly.

1 INTRODUCTION

Human-robot interaction in terms of action

coordination and physical interaction has been an

intense research field since more than three

decades. Especially in emerging fields as future

production systems, requirements for ergonomic

work systems are considered important aspects and

the human operator is predicted to still be involved

for a long time in the assembly process of many

products, e.g. for performing certain assembly tasks

(Faber et al., 2015), with synergies of human

cognitive and sensorimotor skills and robotic power

and endurance (Shen and Reinhart, 2013). Moreover,

a trend to small batch sizes may render investments

in complete automation uneconomical (Bley et al.,

2004).

Especially, integrating human heuristics may help

to improve cooperation as well as align operator’s

conformity with expectations (Mayer and Schlick,

2012). Contact and proximity sensing, e.g. (Lee,

2009; Lee and Song, 2014) may help to reduce

collision risks, however, especially in highly dynamic

https://orcid.org/0000-0003-1511-9381

https://orcid.org/0000-0002-9214-1032

interaction tasks, such approaches might be too slow

and adaptive, respectively, collaborative online

planning of robot action, e.g. (Kirsch et al., 2010),

might be more effective.

In previous works, we pointed out the influence of

different robot configurations on human hand

trajectories as well as work load (Kühnlenz and

Kühnlenz, 2019).

In the context of investigating robotic influences

on human hand movements and optimizing robot

action towards reduction of collision probabilities

during cooperative tasks, we were in a next step

interested in the uninfluenced progression of human

hand trajectories and their variabilities in order to

obtain a baseline. So, in this paper, we present results

from a study on human hand trajectories during a

pick-and-place task evaluating average variabilities

in terms of standard deviations along the trajectories.

The remainder of the paper is organized as

follows: Section 2 presents the hypotheses and study

design; Results are presented in Section 3 and

discussed in Section 4; conclusions are given in

Section 5.

Kühnlenz, K., Hermann, S., Kalb, K., Marschollek, L. and Kühnlenz, B.

Progression of Human Hand Trajectory Variabilities during a Pick-and-Place Task.

DOI: 10.5220/0007915403230327

In Proceedings of the 16th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2019), pages 323-327

ISBN: 978-989-758-380-3

323

2 HYPOTHESES AND STUDY

DESIGN

In this section the working hypotheses, study design

and experimental setting are presented.

2.1 Hypotheses

In the context of our investigations, pick-and-place

tasks are considered as basic building blocks, e.g. for

cooperative industrial assembly scenarios.

A fundamental assumption to be investigated here

is, that the variability of human hand trajectories

gradually approaches a minimum, when moving

towards a specified goal location, e.g. for picking up

or placing objects. This assumption seems likely as

the goal location is precisely defined and goal

oriented behaviour of the human is required in order

to accomplish the task.

So, the first hypothesis to be tested in this paper is

H1: Human hand trajectory variabilities

monotonically decrease, when moving from

start to goal location.

Additionally, we hypothesize, that if repeating a

pick-and-place task, trajectory variabilities would

decrease due to learning effects leading to the second

hypothesis to be tested:

H2: Repeated conduction of pick-and-place tasks

reduces human hand trajectory variabilities.

2.2 Study Design and Measures

A within-subjects (repeated measures) design is

chosen for the experimental study. The defined task

to be accomplished for the investigations is to move

a LEGO



block from one pre-defined location to

another in order for the block to be picked up by a

conventional industrial robot, which places it again at

the start location. The task is composed of the

following parts, which are repeated for 1min:

 Move to pick-up-location

 Pick up block repeat for

 Move to goal location 1min

 Place block at goal location

As a measure for trajectory variability we chose

to evaluate the progression of the standard deviations

orthogonally to the direct path (from start to goal

location in x-direction) computed point-wise along

the hand path. So, for each repeated pick-and-place

trajectory, we extracted the hand position at position

measured by an optical tracking system and

computed the standard deviations of the point cloud

over all trajectories at this point.

The repeated measures design for studying

potential learning effects consists in two trials with a

short break in between, where the participants are

required to fill in a questionnaire on work-load, the

results of which being part of another paper.

In order to obtain hand trajectories of

approximately equal duration, a synchronous robot

arm motion is designed in such a way that the

participants are given 5s to pick and place the object

until the robot picks it up from the goal location. The

total duration of each trial of the pick-and-place is set

to 1min.

The complete schedule of the experiment is

shown in the following table.

Table 1: Schedule of the experiment.

Item Content Purpose Duration

briefing

introduction into

purpose of

experiment;

explanation of

questionnaires, etc.

~5min

PRE-

questionnaire

demographical data,

dispositional

parameters, etc.

~1min

participant

placement

sensor placement;

preparation of data

recording

~5min

adaptation no instruction

acclimatization,

familiarization

~1min

Trial 1

Conduct pick-and-

place task

Evaluation of

trajectory

variabilities

~1min

POST-

questionnaire 1

work-load items

~1min

Trial 2

Conduct pick-and-

place task

Evaluation of

trajectory

variabilities

~1min

POST-

questionnaire 2

work-load items

~1min

de-briefing

2.3 Experimental Setting

The experimental setting is composed of LEGO



plates for start and goal locations, a LEGO



block to

be moved, a UR3 (Universal Robots) robot arm, a

Trio optical tracking system (Optitrack) and three

markers (start, goal and hand positions) as shown in

the following figures. Marker positions are tracked

with 120Hz.

ICINCO 2019 - 16th International Conference on Informatics in Control, Automation and Robotics

324

Figure 1: Complete setting.

Figure 2: Goal (left) and start (right) locations on LEGO



plates with optical markers.

Figure 3: Optical marker mounted at participant’s thumb.

3 RESULTS

Experimental results were deduced from 20

participants (age between 21 and 30 years, 6 female

and 14 male) with 40 interactions (two consecutive

trials of each person). One data set had to be dropped

due to corrupt sensor data.

3.1 Trajectory Variability between

Start and Goal Locations

A typical progression of the participants’ hand

trajectories is shown in Figure 4 including an

uncertainty tube quantifying the progression of

standard deviations in terms of ellipses with main

axes being orthogonal to the direct path between start

and goal position.

Figure 4: Example trajectories (green) of participant 6 with

object (red) to be picked up and uncertainty ellipses.

Figure 5: Example progressions of trajectory standard

deviations (blue) of participant 10 and moving average

filtered line (red).

Figure 5 exemplarily shows the progression of the

standard deviations in y-direction orthogonal to direct

path as well as the moving average (10 points) filtered

progression.

A decrease of the standard deviations can be noted

as the participants’ hands move from start to goal

location as expected.

In order to quantify this decrease, a paired

samples t-test is conducted revealing significant

results with a strong effect as shown in Table 2.

Applying a Shapiro-Wilk test, assumption of

normality is shown to be met (W = 0.989, p = 0.969).

Table 2: T-test results.

t df p Cohen’s d



start



oal

7.570 37 <0.001 1.228

These results clearly show, that a significant decrease

of the standard deviation along human hand

trajectories from M = 0.046m (SD = 0.13m) to M =

0.028m (SD = 0.015m) is present.

Progression of Human Hand Trajectory Variabilities during a Pick-and-Place Task

325

3.2 Variability along the Trajectory

Looking at the measurement data of all participants,

it is surprising, that even though a significant decrease

of standard deviations is noted for all participants, the

forms of the progression of standard deviations seem

quite different.

So, in order to quantify this phenomenon, a

repeated measures ANOVA is conducted including

another measurement item of the standard deviation

right in the middle (between start and goal) of the

trajectory in addition to the standard deviations at

start and goal locations. As Mauchly’s test is

significant (W = 0.752, p = 0.006) a Greenhouse-

Geisser correction is conducted. Again, the ANOVA

results show, that a significant difference between the

standard deviations at the different locations with a

strong effect can be noted, see Table 3. Post hoc

results with Bonferroni correction (Table 4) show,

that significant differences exist between all

measurement points (start, middle, goal) supporting

the assumption of a continuous decrease of standard

deviations along the trajectories.

The descriptive statistics, however, also show,

that the largest variability of standard deviations of

human hand trajectories exists in the middle of the

trajectories, see Table 5.

Table 3: ANOVA results.

F df p

௣

ଶ



45.492 1.602 <0.001 0.551

Table 4: Post hoc results.

Mean

Diff. [m]

SE t p



start



middle

0.005 0.002 3.106 0.011



oal

0.018 0.002 7.570 <0.001



middle



oal

0.013 0.002 7.536 <0.001

Table 5: Descriptive statistics.

M [m] SD [m]



star

0.046 0.013



middle

0.041 0.017



oal

0.028 0.015

3.3 Longitudinal Effects

In order to investigate potential learning effects and

resulting improvements of hand positioning in terms

reduced standard deviations, t-tests on the differences

between standard deviations at the start location for

two consecutive trials, respectively, at middle and at

goal locations, are conducted.

The t-test results for each of the measurement

points turn out not to be significant, so significant

differences between the two trials are not observed.

Figure 6: Boxplot of standard deviation measurements at

start location of trajectories.

Figure 7: Boxplot of standard deviation measurements at

center location of trajectories.

Figure 8: Boxplot of standard deviation measurements at

goal location of trajectories.

4 DISCUSSION

The results presented support hypothesis H1 in terms

of variabilities of human hand trajectories decreasing

from a start to a goal location during a pick-and-place

task. However, it is also found, that along the

trajectories at some distance to start and goal

locations, there is a decrease on average, but a larger

uncertainty of the decrease between trajectories,

meaning that the form of the trajectories strongly

varies between pick-and-place runs. Both aspects

suggest, that it might be opportune to include these

ICINCO 2019 - 16th International Conference on Informatics in Control, Automation and Robotics

326

uncertainty variations along the trajectories in a

planning framework with respect to reduction of

collision probabilities between human and robot

arms.

With respect to hypothesis H2, no significant

results are obtained, meaning, that learning effects

cannot be confirmed and there is no longitudinal

variation of trajectories between pick-and-place trials

found. In consequence, a potential action planning

framework for cooperative robots would not have to

account for such learning effects over time, which

simplifies the architecture.

This paper does not claim, that the study is

representative in terms of broad variations of

demographic, dispositional and situational

parameters, but it is nevertheless remarkable, that

statistically significant results are obtained with a

rather small sample size, which provides at least some

potential for generalization.

5 CONCLUSIONS

This paper presents first results on progressions of

uncertainties along human hand trajectories during

pick-and-place tasks, revealing that there is a

significant decrease of uncertainties as the human

hand approaches the goal location. However, between

start and goal locations, a larger variation of these

uncertainties is found due to the variation of

trajectory forms.

The results suggest, that collision probability

increases with distance to the goal location due to

increasing uncertainties and that such spatial

progressions of uncertainties should be integrated in

robot action planning frameworks for cooperative

interaction scenarios, e.g. in industrial assembly or

assistive robotics.

ACKNOWLEDGEMENTS

This work has been supported in part by the Federal

Ministry of Education and Research (BMBF).

REFERENCES

Bley, H., Reinhart, G., Seliger, G., Bernardi, M., and Korne,

T., 2004. Appropriate Human Involvement in

Assembly and Disassembly, In: CIRP Annals -

Manufacturing Technology, vol. 53, pp. 487-509.

Faber, M., Bützler, J., and Schlick, C. M., 2015. Human-

robot Cooperation in Future Production Systems:

Analysis of Requirements for Designing an Ergonomic

Work System. In Procedia Manufacturing, vol. 3,

Elsevier, pp. 510-517.

Kirsch, A., Kruse, T., Sisbot, E. A., Alami, R., Lawitzky,

M., Brscic, D., Hirche, S., Basili, P., and Glasauer, S.,

2010. Plan-based Control of Joint Human-Robot

Activities. In: Künstliche Intelligenz, vol. 24, no. 3, pp.

223-231.

Kühnlenz, K. and Kühnlenz, B., 2019. Insights from a

Study on Motor Interference and Task Load in Close

HRI, Advanced Robotics, Taylor and Francis, under

review.

Lee, H.-K., Chang, S., and Yoon, E., 2009. Dual-Mode

Capacitive Proximity Sensor for Robot Application:

Implementation of Tactile and Proximity Sensing

Capability on a Single Polymer Platform Using Shared

Electrodes. In IEEE Sensors Journal, vol. 9, no. 12, pp.

1748-1755.

Lee, S.-D. and Song, J.-B., 2014. Collision Detection for

Safe Human-Robot Cooperation of a Redundant

Manipulator. In: Proc. of the 14th International

Conference on Control, Automation and Systems

(ICCAS 2014).

Mayer, M., Schlick, C. M., 2012. Improving operator’s

conformity with expectations in a cognitively

automated assembly cell using human heuristics. In: S.

Trzcielinski; W. Karwowski (Eds.) Advances in

Ergonomics in Manufacturing, CRC Press, Boca Raton,

FL, USA, 2012.

Shen, Y. and Reinhart, G., 2013. Safe Assembly Motion -

A Novel Approach for Applying Human-Robot Co-

operation in Hybrid Assembly Systems, In: 2013 IEEE

International Conference on Mechatronics and

Automation (ICMA).

Progression of Human Hand Trajectory Variabilities during a Pick-and-Place Task

327