Progression of Human Hand Trajectory Variabilities
during a Pick-and-Place Task
Kolja Kühnlenz
1a
, Sergej Hermann
1
, Kevin Kalb
1
, Lucas Marschollek
1
and Barbara Kühnlenz
2b
1
Robotics Research Lab, Department of Electrical Engineering and Computer Science,
Coburg University of Applied Sciences and Arts, D-96450 Coburg, Germany
2
Institute for Cognitive Systems, Department of Electrical Engineering and Information Technology,
TU Munich, D-80290 Munich, Germany
Keywords: Robotics, Human-robot Interaction, Human Factors.
Abstract: This paper investigates the progression of human hand trajectory variabilities during a pick-and-place task. A
user study is conducted and human hand positions are tracked optically. Standard deviations of human hand
positions over all trajectories within a trial are computed point-wise orthogonally to the direct path between
start and goal positions. Statistical tests reveal a decrease of standard deviations from hand start to goal
positions. Moreover, stronger variations of standard deviations are noted in during the center part of the
trajectories. Contrary to expectations, a longitudinal study design does not reveal learning effects in terms of
reduction of trajectory variabilities. The results suggest, that uncertainties of human hand positions increase
with the distance to a goal location and could constitute a larger risk for collisions within a cooperative human-
robot pick-and-place scenario, e.g. assembly.
1 INTRODUCTION
Human-robot interaction in terms of action
coordination and physical interaction has been an
intense research field since more than three
decades. Especially in emerging fields as future
production systems, requirements for ergonomic
work systems are considered important aspects and
the human operator is predicted to still be involved
for a long time in the assembly process of many
products, e.g. for performing certain assembly tasks
(Faber et al., 2015), with synergies of human
cognitive and sensorimotor skills and robotic power
and endurance (Shen and Reinhart, 2013). Moreover,
a trend to small batch sizes may render investments
in complete automation uneconomical (Bley et al.,
2004).
Especially, integrating human heuristics may help
to improve cooperation as well as align operator’s
conformity with expectations (Mayer and Schlick,
2012). Contact and proximity sensing, e.g. (Lee,
2009; Lee and Song, 2014) may help to reduce
collision risks, however, especially in highly dynamic
a
https://orcid.org/0000-0003-1511-9381
b
https://orcid.org/0000-0002-9214-1032
interaction tasks, such approaches might be too slow
and adaptive, respectively, collaborative online
planning of robot action, e.g. (Kirsch et al., 2010),
might be more effective.
In previous works, we pointed out the influence of
different robot configurations on human hand
trajectories as well as work load (Kühnlenz and
Kühnlenz, 2019).
In the context of investigating robotic influences
on human hand movements and optimizing robot
action towards reduction of collision probabilities
during cooperative tasks, we were in a next step
interested in the uninfluenced progression of human
hand trajectories and their variabilities in order to
obtain a baseline. So, in this paper, we present results
from a study on human hand trajectories during a
pick-and-place task evaluating average variabilities
in terms of standard deviations along the trajectories.
The remainder of the paper is organized as
follows: Section 2 presents the hypotheses and study
design; Results are presented in Section 3 and
discussed in Section 4; conclusions are given in
Section 5.
Kühnlenz, K., Hermann, S., Kalb, K., Marschollek, L. and Kühnlenz, B.
Progression of Human Hand Trajectory Variabilities during a Pick-and-Place Task.
DOI: 10.5220/0007915403230327
In Proceedings of the 16th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2019), pages 323-327
ISBN: 978-989-758-380-3
Copyright
c
2019 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
323
2 HYPOTHESES AND STUDY
DESIGN
In this section the working hypotheses, study design
and experimental setting are presented.
2.1 Hypotheses
In the context of our investigations, pick-and-place
tasks are considered as basic building blocks, e.g. for
cooperative industrial assembly scenarios.
A fundamental assumption to be investigated here
is, that the variability of human hand trajectories
gradually approaches a minimum, when moving
towards a specified goal location, e.g. for picking up
or placing objects. This assumption seems likely as
the goal location is precisely defined and goal
oriented behaviour of the human is required in order
to accomplish the task.
So, the first hypothesis to be tested in this paper is
H1: Human hand trajectory variabilities
monotonically decrease, when moving from
start to goal location.
Additionally, we hypothesize, that if repeating a
pick-and-place task, trajectory variabilities would
decrease due to learning effects leading to the second
hypothesis to be tested:
H2: Repeated conduction of pick-and-place tasks
reduces human hand trajectory variabilities.
2.2 Study Design and Measures
A within-subjects (repeated measures) design is
chosen for the experimental study. The defined task
to be accomplished for the investigations is to move
a LEGO
block from one pre-defined location to
another in order for the block to be picked up by a
conventional industrial robot, which places it again at
the start location. The task is composed of the
following parts, which are repeated for 1min:
Move to pick-up-location
Pick up block repeat for
Move to goal location 1min
Place block at goal location
As a measure for trajectory variability we chose
to evaluate the progression of the standard deviations
orthogonally to the direct path (from start to goal
location in x-direction) computed point-wise along
the hand path. So, for each repeated pick-and-place
trajectory, we extracted the hand position at position
x
i
measured by an optical tracking system and
computed the standard deviations of the point cloud
over all trajectories at this point.
The repeated measures design for studying
potential learning effects consists in two trials with a
short break in between, where the participants are
required to fill in a questionnaire on work-load, the
results of which being part of another paper.
In order to obtain hand trajectories of
approximately equal duration, a synchronous robot
arm motion is designed in such a way that the
participants are given 5s to pick and place the object
until the robot picks it up from the goal location. The
total duration of each trial of the pick-and-place is set
to 1min.
The complete schedule of the experiment is
shown in the following table.
Table 1: Schedule of the experiment.
Item Content Purpose Duration
briefing
introduction into
purpose of
experiment;
explanation of
questionnaires, etc.
~5min
PRE-
questionnaire
demographical data,
dispositional
parameters, etc.
~1min
participant
placement
sensor placement;
preparation of data
recording
~5min
adaptation no instruction
acclimatization,
familiarization
~1min
Trial 1
Conduct pick-and-
place task
Evaluation of
trajectory
variabilities
~1min
POST-
questionnaire 1
work-load items
~1min
Trial 2
Conduct pick-and-
place task
Evaluation of
trajectory
variabilities
~1min
POST-
questionnaire 2
work-load items
~1min
de-briefing
2.3 Experimental Setting
The experimental setting is composed of LEGO
plates for start and goal locations, a LEGO
block to
be moved, a UR3 (Universal Robots) robot arm, a
Trio optical tracking system (Optitrack) and three
markers (start, goal and hand positions) as shown in
the following figures. Marker positions are tracked
with 120Hz.
ICINCO 2019 - 16th International Conference on Informatics in Control, Automation and Robotics
324
Figure 1: Complete setting.
Figure 2: Goal (left) and start (right) locations on LEGO
plates with optical markers.
Figure 3: Optical marker mounted at participant’s thumb.
3 RESULTS
Experimental results were deduced from 20
participants (age between 21 and 30 years, 6 female
and 14 male) with 40 interactions (two consecutive
trials of each person). One data set had to be dropped
due to corrupt sensor data.
3.1 Trajectory Variability between
Start and Goal Locations
A typical progression of the participants’ hand
trajectories is shown in Figure 4 including an
uncertainty tube quantifying the progression of
standard deviations in terms of ellipses with main
axes being orthogonal to the direct path between start
and goal position.
Figure 4: Example trajectories (green) of participant 6 with
object (red) to be picked up and uncertainty ellipses.
Figure 5: Example progressions of trajectory standard
deviations (blue) of participant 10 and moving average
filtered line (red).
Figure 5 exemplarily shows the progression of the
standard deviations in y-direction orthogonal to direct
path as well as the moving average (10 points) filtered
progression.
A decrease of the standard deviations can be noted
as the participants’ hands move from start to goal
location as expected.
In order to quantify this decrease, a paired
samples t-test is conducted revealing significant
results with a strong effect as shown in Table 2.
Applying a Shapiro-Wilk test, assumption of
normality is shown to be met (W = 0.989, p = 0.969).
Table 2: T-test results.
t df p Cohen’s d
start
-
g
oal
7.570 37 <0.001 1.228
These results clearly show, that a significant decrease
of the standard deviation along human hand
trajectories from M = 0.046m (SD = 0.13m) to M =
0.028m (SD = 0.015m) is present.
Progression of Human Hand Trajectory Variabilities during a Pick-and-Place Task
325
3.2 Variability along the Trajectory
Looking at the measurement data of all participants,
it is surprising, that even though a significant decrease
of standard deviations is noted for all participants, the
forms of the progression of standard deviations seem
quite different.
So, in order to quantify this phenomenon, a
repeated measures ANOVA is conducted including
another measurement item of the standard deviation
right in the middle (between start and goal) of the
trajectory in addition to the standard deviations at
start and goal locations. As Mauchly’s test is
significant (W = 0.752, p = 0.006) a Greenhouse-
Geisser correction is conducted. Again, the ANOVA
results show, that a significant difference between the
standard deviations at the different locations with a
strong effect can be noted, see Table 3. Post hoc
results with Bonferroni correction (Table 4) show,
that significant differences exist between all
measurement points (start, middle, goal) supporting
the assumption of a continuous decrease of standard
deviations along the trajectories.
The descriptive statistics, however, also show,
that the largest variability of standard deviations of
human hand trajectories exists in the middle of the
trajectories, see Table 5.
Table 3: ANOVA results.
F df p
ߟ
45.492 1.602 <0.001 0.551
Table 4: Post hoc results.
Mean
Diff. [m]
SE t p
start
middle
0.005 0.002 3.106 0.011
g
oal
0.018 0.002 7.570 <0.001
middle
g
oal
0.013 0.002 7.536 <0.001
Table 5: Descriptive statistics.
M [m] SD [m]
star
t
0.046 0.013
middle
0.041 0.017
g
oal
0.028 0.015
3.3 Longitudinal Effects
In order to investigate potential learning effects and
resulting improvements of hand positioning in terms
reduced standard deviations, t-tests on the differences
between standard deviations at the start location for
two consecutive trials, respectively, at middle and at
goal locations, are conducted.
The t-test results for each of the measurement
points turn out not to be significant, so significant
differences between the two trials are not observed.
Figure 6: Boxplot of standard deviation measurements at
start location of trajectories.
Figure 7: Boxplot of standard deviation measurements at
center location of trajectories.
Figure 8: Boxplot of standard deviation measurements at
goal location of trajectories.
4 DISCUSSION
The results presented support hypothesis H1 in terms
of variabilities of human hand trajectories decreasing
from a start to a goal location during a pick-and-place
task. However, it is also found, that along the
trajectories at some distance to start and goal
locations, there is a decrease on average, but a larger
uncertainty of the decrease between trajectories,
meaning that the form of the trajectories strongly
varies between pick-and-place runs. Both aspects
suggest, that it might be opportune to include these
ICINCO 2019 - 16th International Conference on Informatics in Control, Automation and Robotics
326
uncertainty variations along the trajectories in a
planning framework with respect to reduction of
collision probabilities between human and robot
arms.
With respect to hypothesis H2, no significant
results are obtained, meaning, that learning effects
cannot be confirmed and there is no longitudinal
variation of trajectories between pick-and-place trials
found. In consequence, a potential action planning
framework for cooperative robots would not have to
account for such learning effects over time, which
simplifies the architecture.
This paper does not claim, that the study is
representative in terms of broad variations of
demographic, dispositional and situational
parameters, but it is nevertheless remarkable, that
statistically significant results are obtained with a
rather small sample size, which provides at least some
potential for generalization.
5 CONCLUSIONS
This paper presents first results on progressions of
uncertainties along human hand trajectories during
pick-and-place tasks, revealing that there is a
significant decrease of uncertainties as the human
hand approaches the goal location. However, between
start and goal locations, a larger variation of these
uncertainties is found due to the variation of
trajectory forms.
The results suggest, that collision probability
increases with distance to the goal location due to
increasing uncertainties and that such spatial
progressions of uncertainties should be integrated in
robot action planning frameworks for cooperative
interaction scenarios, e.g. in industrial assembly or
assistive robotics.
ACKNOWLEDGEMENTS
This work has been supported in part by the Federal
Ministry of Education and Research (BMBF).
REFERENCES
Bley, H., Reinhart, G., Seliger, G., Bernardi, M., and Korne,
T., 2004. Appropriate Human Involvement in
Assembly and Disassembly, In: CIRP Annals -
Manufacturing Technology, vol. 53, pp. 487-509.
Faber, M., Bützler, J., and Schlick, C. M., 2015. Human-
robot Cooperation in Future Production Systems:
Analysis of Requirements for Designing an Ergonomic
Work System. In Procedia Manufacturing, vol. 3,
Elsevier, pp. 510-517.
Kirsch, A., Kruse, T., Sisbot, E. A., Alami, R., Lawitzky,
M., Brscic, D., Hirche, S., Basili, P., and Glasauer, S.,
2010. Plan-based Control of Joint Human-Robot
Activities. In: Künstliche Intelligenz, vol. 24, no. 3, pp.
223-231.
Kühnlenz, K. and Kühnlenz, B., 2019. Insights from a
Study on Motor Interference and Task Load in Close
HRI, Advanced Robotics, Taylor and Francis, under
review.
Lee, H.-K., Chang, S., and Yoon, E., 2009. Dual-Mode
Capacitive Proximity Sensor for Robot Application:
Implementation of Tactile and Proximity Sensing
Capability on a Single Polymer Platform Using Shared
Electrodes. In IEEE Sensors Journal, vol. 9, no. 12, pp.
1748-1755.
Lee, S.-D. and Song, J.-B., 2014. Collision Detection for
Safe Human-Robot Cooperation of a Redundant
Manipulator. In: Proc. of the 14th International
Conference on Control, Automation and Systems
(ICCAS 2014).
Mayer, M., Schlick, C. M., 2012. Improving operator’s
conformity with expectations in a cognitively
automated assembly cell using human heuristics. In: S.
Trzcielinski; W. Karwowski (Eds.) Advances in
Ergonomics in Manufacturing, CRC Press, Boca Raton,
FL, USA, 2012.
Shen, Y. and Reinhart, G., 2013. Safe Assembly Motion -
A Novel Approach for Applying Human-Robot Co-
operation in Hybrid Assembly Systems, In: 2013 IEEE
International Conference on Mechatronics and
Automation (ICMA).
Progression of Human Hand Trajectory Variabilities during a Pick-and-Place Task
327