Operational Assistance System using 3-DOF Joystick with Reaction
Force Display to Load Transfer Machine in a Plane
Tomoya Kuneguchi
1
, Yoshiyuki Noda
1
, Yukinori Sago
2
and Kiyoaki Kakihara
3
1
Department of Mechanical Systems Engineering, University of Yamanashi, Takeda 4-3-11, Kofu, Japan
2
Department of Mechanical Engineering, Toyohashi University of Technology, Toyohashi, Japan
3
KER Co., Ltd, Toyokawa, Japan
Keywords:
Operational assistance, load transfer machine, 3-DOF Joystick, Reaction Force Display, Obstacles Avoidance.
Abstract:
This paper is concerned with an operational assistance system to a load transfer machine in a plane such
as a manually guided vehicle and a crane without vertical transfer. In the transfer machine, collisions with
obstacles are often occurred by the incorrect operations. Therefore, in order to avoid the obstacles to the
transfer machine, the operational assistance system using 3-DOF(3 Degree of Freedom) joystick with reaction
force display is proposed in this study. The 3-DOF joystick enables operator to manipulate the transfer machine
with back-and-forth, left-and-right, and rotation motions. And, it has motors for presenting the reaction force
on each axis. In this system, the joystick displays the reaction force which consists of the virtual elastic
force and viscosity resistance, which are controlled by PD control gains. By associating the gains in the PD
control with the distance between the obstacles and the transfer object, the joystick operation is constrained
for avoiding the obstacles. The effectiveness of the proposed operational assistance system is verified by the
experiments using the simulator of the transfer machine manipulated by the 3-DOF joystick.
1 INTRODUCTION
A load transfer machine such as a crane and an indus-
trial vehicle is widely used in industry. It is required
to transfer the load efficiently(Sawodny, Aschemann
and Lahres, 2002), (Osumi, Kubo, Yano and Saito,
2010). However, since there are some obstacles in
the transfer space, the operator has to manipulate the
transfer machine to avoid the obstacles. The skill is
required to manipulate the transfer machine. How-
ever, even if the skilled worker operates the transfer
machine, the collisions with the obstacles are often
occurred by the incorrect operation. 30 fatal accidents
by cranes occurred in Japan in 2011, of which fifteen
were the accidents by the incorrect operation(Japan
Crane Association, 2011). Including the unreported
accidents such as a near-miss accident, there are a lot
of accidents by the incorrect operation in the transfer
machines.
In order to operate safety the crane, some op-
erational assistance systems have been proposed in
the previous studies. The sway of the load in
the crane is occurred by the inappropriate opera-
tion. For suppressing the sway of the load, the op-
erational assistance system with the haptic device
was proposed(Yoneda, Arai, Fukuda and Miyata,
1999). The crane operation assistance system using
the augmented reality technology was proposed in the
study(Yi-Chen, Hung-Lin, Shin-Chung and Shang-
Hsien, 2011). In this system, the actual information
of the working environment and the virtual informa-
tion such as building information are integrated and
displayed on the monitor. However, the operational
assistance systems for avoiding the obstacles have not
been proposed in most previous studies as seen from
the above. In one of the previous studies, the oper-
ational assistance system for avoiding the obstacles
using the haptic joystick was proposed(Sato, Noda,
et.al., 2009). However, since the overall size of the
transfer object has not been considered in the sys-
tem, the collision of the obstacles and the edge of the
transfer object might be occurred. Moreover, the de-
sign method of the parameters in the joystick control
system are unclear in the paper. It is desired to de-
sign clearly the operational assistance system using
the haptic joystick with consideration of the overall
size of the transfer object for obstacles avoidance.
Recently, an omnidirectional mobile vehicle has
been developed as a next-generation personal vehi-
cle(Noda, Kawaguchi and Terashima, 2010). This
346
Kuneguchi T., Noda Y., Sago Y. and Kakihara K..
Operational Assistance System using 3-DOF Joystick with Reaction Force Display to Load Transfer Machine in a Plane.
DOI: 10.5220/0005026303460352
In Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics (ICINCO-2014), pages 346-352
ISBN: 978-989-758-040-6
Copyright
c
2014 SCITEPRESS (Science and Technology Publications, Lda.)
Figure 1: Crane system manipulated by 3-DOF joystick.
vehicle has 3-DOF motion with back-and-forth, left-
and-right, and rotation. However in the previous stud-
ies, the vehicle has been operated by the 2-DOF joy-
stick and switching equipment which switches be-
tween the lateral and the rotation motions. Therefore,
it cannot be transferred freely in a plane. For improv-
ing the operability of the vehicle, 3-DOF operation
device is required. On the other hand, the incorrect
operation will be increased by applying the 3-DOF
joystick. The operation device with high operability
and safety is demanded for the useful transfer ma-
chine with high degree of freedom motion.
Therefore, the present authors are developing the
operational assistance system using 3-DOF joystick
with reaction force display for the load transfer ma-
chine. The example of the application of the develop-
ing operational assistance system is shown in Figure
1. In Figure 1, the operational assistance system is
integrated into the overhead traveling crane system.
The crane system has 3-DOF transfer with back-and-
forth, left-and-right, and rotation. In order to avoid
the obstacles, the joystick operation is constrained dy-
namically by the reaction force display.
In this paper, 3-DOF joystick with the reaction
force displayed on each axis is proposed. In order to
constrain dynamically the joystick operation for the
obstacle avoidance, about 30[N] of the maximum re-
action force can be generated by the servomotors in-
stalled the joystick. Moreover, the operational assis-
tance system using the 3-DOF joystick is proposed
for operating the machine freely and safety. In this
system, the joystick displays the reaction force which
consists of the virtual elastic force and viscosity re-
sistance controlled by PD control gains. Furthermore,
by associating the gains in the PD control with the
distance between the obstacles and the transfer object,
the joystick operation is constrained for avoiding the
Figure 2: Photo of 3-DOF joystick.
Figure 3: Schematic view of 3-DOF joystick structure.
Table 1: Specifications of DC servomotors.
Rating Torque Rating Current
DC Motor (A) 23.3[mNm] 0.49[A]
DC Motor (B) 33.6[mNm] 1.48[A]
obstacles.
The effectiveness of the proposed operational as-
sistance system is verified by the experiments using
the simulator of the transfer machine manipulated by
the 3-DOF joystick.
2 3-DOF JOYSTICK
The photo and the schematic view of the 3-DOF joy-
stick proposed in this study are shown in Figures 2
and 3, respectively. This joystick has three servomo-
tors, and can provide the reaction force to operation of
back-and-forth, left-and-right, and rotation as shown
in Figure 3. The specifications of the installed mo-
tors in the joystick are shown in Table 1. The reaction
force to the back-and-forthoperation can be displayed
OperationalAssistanceSystemusing3-DOFJoystickwithReactionForceDisplaytoLoadTransferMachineinaPlane
347
by increasing the driving torque of the motor through
the reducer and the bevel gear. The reaction force to
the left-and-right operation also can be displayed by
the same structure with the back-and-forth operation.
The tilting angle of the operating lever can be detected
by the rotary encoders attached to the motors. The
motor for displaying the reaction force to the rotation
operation is installed into the operating lever. The re-
action force can be displayed by self-rotating the mo-
tor with the reducer. The rotating angle of the operat-
ing lever can be detected by the hall sensor installed
into the motor. The maximum reaction forces at the
operation point to the back-and-forthand the left-and-
right operations are 30.18[N]. The maximum reaction
torque to the rotation operation is 0.76[mNm]. They
are sufficient to constrain the operation for preventing
the incorrect operation.
3 OPERATIONAL ASSISTANCE
SYSTEM
The operational assistance system which avoid the
collision with the obstacles by the incorrect operation
using the 3-DOF joystick is proposed in this study.
The block diagram of the proposed assistance system
is shown in Figure 4.
In the proposed assistance system, the servomotor
with the current control mode is used. The reaction
torque T is generated by the motor, and the reaction
force F at the operation point is represented by di-
viding the reaction torque T by the length L of the
operating lever. The operating lever is tilted depend-
ing on the error between the reaction force F and the
force f by the operator. Here, J is the inertia moment
about the tilting center to the operating lever, and θ is
the tilting angle of the operating lever. The velocity of
the transfer machine corresponds to the tilting angle θ
of the operating lever. The reaction force F can be in-
creased by increasing the virtual elastic force and the
viscous resistance in the operational assistance sys-
tem. Therefore, the PD control to the tilting angle
θ is constructed for generating the elastic force and
the viscous resistance. Since the PD gains correspond
to the distance between the obstacles and the transfer
object, the reaction forces are increased by approach-
ing the transfer object to the obstacles. The reference
force F
∗
is generated by the PD control, and trans-
formed to the current command i through the function
ξ(F
∗
). The operational assistance system as shown in
Figure 4 is constructed to each operational axis.
Figure 4: Block diagram of operational assistance system.
Figure 5: Measurement of reaction force using force gauge.
3.1 Transform Function between
Current Command to Motor and
Reaction Force
ξ shown in Figure 4 is the function between the reac-
tion force and the current command to the motor. In
design of the function ξ, the relation between the re-
action force and the current command has to be clar-
ified. Therefore, the reaction force at the operation
point is measured by using the force gauge as shown
in Figure 5. The measurement results are shown in
Figure 6. In Figure 6, the markers are the reaction
force measured by the force gauge. The reaction force
cannot be displayed to the small current command.
On the other hand, the reaction force to the current
command over 0.3[A] is increased linearly to the cur-
rent command. The function ξ is designed by the in-
verse function to the measurement data in Figure 6.
However, the inverse function cannot be uniqueness
on the small current command. Therefore, the thresh-
old is set at the reaction force 5[N], and the inverse
function between the reaction force and the current
command is represented as
ICINCO2014-11thInternationalConferenceonInformaticsinControl,AutomationandRobotics
348
0 0.5 1 1.5
0
5
10
15
20
25
Reaction Force F [N]
Current Command [A]
Approximation
Experimental data
Threshold f
b
Figure 6: Relation between reaction force and current.
i =
(
i
a
(F
∗
), (F
∗
> f
b
)
i
a
( f
b
)
f
b
F
∗
, (F
∗
≤ f
b
),
(1)
where f
b
is the threshold, and F
∗
is the reference
force. i
a
is linear approximation of the current com-
mand to the large reaction force. In this study, the
function i
a
is represented as
i
a
(F
∗
) = α
1
F
∗
+ α
0
, (2)
where α
0
= 51.81 and α
1
= 56.04.
3.2 Derivation Method of Shortest
Distance between Obstacles and
Transfer Object
The transfer object varies widely depending on the in-
tended use. In the crane system, a long object such
as a steel beam or a cylindrical container such as a
liquid tank is transferred. Therefore, it is difficult to
recognize the accurate geometry of the transfer ob-
ject automatically. In this study, the transfer object
is replaced by the ellipse including the transfer ob-
ject as shown in Figure 7. Even the long object can
be included reasonably by the ellipse. The ellipsoidal
Figure 7: Transfer object included into ellipse and opera-
tional direction.
Figure 8: Positional relationship between ellipsoidal trans-
fer object and obstacle.
transfer object needs the 3-DOF motion with back-
and-forth, left-and-right, and rotation, and it can be
manipulated by the proposed 3-DOF joystick.
In order to generate the reaction force for the ob-
stacles avoidance, the shortest distance between the
obstacles and the ellipsoidal transfer object is derived
in this section. The relationship between the obstacle
and the ellipsoidal transfer object is shown in Figure
8. In the operational assistance system as shown in
Figure 1, the outline of the obstacles can be detected
by a laser sensor. The shortest distance is selected
from among the detected positional data, R
i
(x
i
,y
i
). At
first in the derivation method of the shortest distance,
the detected positional data on the transfer direction
are picked up as
S
d
= {R
i
(x
i
,y
i
) ∈ S| ax
i
+ b
1
< y
i
< ax
i
+ b
2
,
sgn(x
i
) = sgn(θ
x
),sgn(y
i
) = sgn(θ
y
)}, (3)
where S is the set of the detected positional data, and
S
d
is that on the transfer direction. θ
x
and θ
y
are
the tilting angles of the operating lever on x- and y-
axes, respectively. The linear functions ax
i
+ b
1
and
ax
i
+ b
2
show the tangent lines to the ellipse which
have same slope to the transfer direction of the trans-
fer object. The slope a is represented as
a =
θ
y
θ
x
. (4)
The intercept b
1
and b
2
can be derived from the tan-
gent points of the tangent line and the ellipse. The
distances |R
di
| of the detected positional data on the
transfer direction are calculated as
|R
di
| =
q
x
2
i
+ y
2
i
, ∀R
di
∈ S
d
. (5)
OperationalAssistanceSystemusing3-DOFJoystickwithReactionForceDisplaytoLoadTransferMachineinaPlane
349
Figure 9: Relation between proportional gain and shortest
distance.
The shortest distance |R|
min
can be derived as
|R|
min
= min|R
di
|. (6)
3.3 Design of Virtual Elastic Force
In the proposed operational assistance system shown
in Figure 4, the reaction force can be represented as
J
i
¨
θ
i
+ K
di
˙
θ
i
+ K
pi
θ
i
= f
i
L, (7)
F
∗
i
=
K
di
L
˙
θ
i
+
K
pi
L
θ
i
, i = x,y, (8)
where F
∗
i
is the reference force on each axis, and K
pi
and K
di
are the proportional gain and the derivative
gain on each axis, respectively. θ
i
is the tilting an-
gle of the operating lever on each axis. In the equa-
tion (8), (K
pi
/L)θ
i
is the virtual elastic force associ-
ated with the proportional gain. The reaction force
is increased by increasing the tilting angle of the op-
erating lever by the effect of the virtual elastic force.
Then, the operating lever is pulled back to the orig-
inal attitude by large proportional gain. Therefore,
the joystick operation is constrained dynamically by
changing the proportional gain. In order to avoid the
obstacles, the relation between the proportional gain
and the shortest distance |R|
min
is represented as
K
pi
=
K
pil
,(|R|
min
> γ
l
)
K
pil
−K
pih
γ
l
−γ
h
(|R|
min
− γ
h
) + K
pih
,
(γ
h
< |R|
min
≤ γ
l
)
K
pih
,(|R|
min
≤ γ
h
), i = x, y,
(9)
where K
pil
is the small proportional gain when the
transfer object is far away from the obstacles. K
pih
is the large proportional gain when the transfer object
is close to the obstacles. γ
l
and γ
h
are the thresholds as
shown in Figure 9. The thresholds have to be satisfied
0 1 2 3 4 5
0
1
2
3
4
5
6
x 10
−3
Propotional Gain
Inertia Moment [kgm
2
]
Experimental data
Linear approximation
Figure 10: Relation between inertia moment and gain.
the following condition.
γ < γ
h
< γ
l
, (10)
where γ is the long axis of the ellipsoidal transfer ob-
ject.
3.4 Design of Virtual Viscous Resistance
As seen from the equation (7), the motion of the op-
erating lever without the viscous resistance (K
di
/L)
˙
θ
i
is the undamped vibration. Therefore, the virtual vis-
cous resistance associated with the derivative gain K
di
is designed for suppressing the vibration. Firstly, the
undamped natural frequency ω
n
can be obtained by
the free vibration of the operating lever which is oc-
curred by K
di
= 0. Then, the inertia moment J is de-
rived as
J =
ω
2
n
K
pi
, i = x,y. (11)
The inertia moment obtained from the above proce-
dure is shown in Figure 10. In this figure, the inertia
moment is increased with increasing the proportional
gain. The backlash of the bevel gear and the reducer
is regarded as the cause of varying inertia moment.
In order to design the derivative gain, the relation be-
tween the inertia moment and the proportional gain is
represented by the linear approximation as
J
(K
pi
)
= β
1
K
pi
+ β
2
, i = x,y. (12)
In this study, β
1
and β
2
are given as 1.012× 10
−3
and
1.570× 10
−3
, respectively.
Using the inertia moment and the proportional
gain, the derivative gain can be derived as
K
di
= 2ζ
q
K
pi
J(K
pi
), i = x,y, (13)
where ζ is the damping ratio. The damping ratio
should be ζ = 1 for realizing the critical damping,
theoretically. However, the noise in the detected an-
gular velocity of the operating lever is amplified by
high derivative gain, and the motion of the operating
ICINCO2014-11thInternationalConferenceonInformaticsinControl,AutomationandRobotics
350
0 0.2 0.4 0.6 0.8 1 1.2 1.4
−40
−30
−20
−10
0
10
20
Time [s]
Joystick Angle[deg]
ζ=0
ζ=0.2
ζ=0.5
Figure 11: Experimental results to motion of operating lever
depended on damping ratio.
Figure 12: Photo of simulator of transfer machine manipu-
lated by 3-DOF joystick.
lever is in unstable. The damping ratio is adjusted in
the range of the damping vibration. In this study, the
damping ratio was determined as ζ = 0.5. The ex-
perimental results to the motion of the operating lever
depended on the damping ratio are shown in Figure
11. It can be seen that the vibration is suppressed by
the damping ratio ζ = 0.5.
4 EXPERIMENTAL VALIDATION
The effectiveness of the proposed operational assis-
tance system is verified by the experiments using the
simulator of the transfer machine manipulated by the
3-DOF joystick. The simulator is shown in Figure
12. The long and short axes of the ellipsoidal transfer
object are 18(pixel) and 4(pixel), respectively. The
thresholds γ
l
and γ
h
for changing the proportional
gain are given as 48(pixel) and 28(pixel). The low
and high proportional gains are K
pxl
= K
pyl
= 0.5 and
K
pxh
= K
pyh
= 5.0, respectively. The threshold γ
h
is
larger than the long axis of the ellipsoidal transfer ob-
ject. Therefore, the joystick operation is constrained
before the obstacle. In order to prevent the motion of
transfer object influenced by the fluctuating hand of
operator, we set up the dead band between the tilting
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
0
5
10
Angle of operating lever [deg]
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
0
20
40
60
80
100
Shortest distance [pixel]
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
0
2
4
6
Proportional Gain
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
0
10
20
30
Reference reaction force [N]
Time [s]
(a)
(b)
(c)
(d)
γ
h
γ
l
Limitation of Dead Band
Figure 13: Experimental results to operation on x-axis.
angle of the operating lever and the command input
to the transfer object. In this study, the dead band is
with in ±5[deg] of the angle of the operating lever.
In the experiments, the transfer object is ap-
proached to the obstacle by manipulating the joystick.
The experimental results to the operations on x- and
y- axes are shown in Figures 13 and 14, respectively.
In these figures, (a) and (b) show the tilting angle of
the operating lever and the shortest distance |R|
min
,
respectively. (c) and (d) are the proportional gain K
p
and the reference reaction force F
∗
, respectively. We
assume the reference reaction force as the actual re-
action force displayed on the top of the joystick. In
figures (a), the broken line shows the limitation of the
dead band which the transfer object does not work on
the tilting angle of the operating lever within ±5[deg].
In figures (b), the broken lines show the thresholds for
changing the proportional gains. As seen from these
figures, the reaction force is increased by approaching
the obstacle, and the operating lever is pulled back to
the vertical posture. The tilting angle of the operating
lever is within the dead band, which is 5[deg]. Then,
the transfer object can be stopped before the object as
shown in Figures 13 (b) and 14 (b). Therefore, the
collision avoidance to the obstacle is realized by the
proposed operational assistance system.
OperationalAssistanceSystemusing3-DOFJoystickwithReactionForceDisplaytoLoadTransferMachineinaPlane
351
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
0
5
10
Angle of operating lever [deg]
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
20
40
60
80
Shortest distance [pixel]
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
0
2
4
6
Proportional Gain
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
0
10
20
30
Reference reaction force [N]
Time [s]
Limitation of Dead Band
γ
l
γ
h
(b)
(c)
(d)
(a)
Figure 14: Experimental results to operation on y-axis.
5 CONCLUSIONS
The 3-DOF joystick with reaction force display has
been proposed in this study. The joystick enables the
operator to manipulate intuitively the 3-DOF trans-
fer object on a plane. Furthermore, the operational
assistance system for avoiding the collision with the
obstacles has also proposed by using the 3-DOF joy-
stick. The experiments using the simulator of the 3-
DOF transfer object manipulated by the 3-DOF joy-
stick shows that the proposed operational assistance
system was effective.
In the future works, we will apply the operational
assistance system to the actual transfer machine such
as the crane system.
REFERENCES
Sawodny, O., Aschemann, H., and Lahres, S., ”An auto-
mated gantry crane as a large workspace robot”, Con-
trol Engineering Practice 10, pp.1323 - 1338, 2002.
Osumi, H., Kubo, M., Yano, S., and Saito, K., ”Devel-
opment of tele-operation system for a crane with-
out overshoot in positioning”, Proceedings of 2010
IEEE/RSJ International Conference on Intelligent
Robots and Systems, pp.5799 - 5805, 2010.
Japan Crane Association, http://www.cranenet.or.jp/, 2011.
Yoneda, M., Arai, F., Fukuda, T., and Miyata, K., ”As-
sistance system for crane operation with haptic dis-
play operational assistance to suppress round payload
awing”, Proceedings of International Conference on
Control, Robotics and Automation, pp.2923 - 2929,
1999.
Yi-Chen, C., Hung-Lin, C., Shin-Chung, K., and Shang-
Hsien, H., ”A smart crane operations assistance sys-
tem using augmented reality technology”, Proceed-
ings of 28th International Symposium on Automation
and Robotics in Construction, pp.643 - 649, 2011.
Sato, R., Noda, Y., Miyoshi, T., Terashima, K., Kakihara,
K., Nie, Y., and Funato, K., ”Operational support
control by haptic joystick considering load sway sup-
pression and obstacle avoidance for intelligent crane”,
Proceedings of Annual Conference of the IEEE Indus-
trial Electronics Society, pp.2321 - 2327, 2009.
Noda, Y., Kawaguchi, A., and Terashima, K., ”A mecha-
tronics vision for smart wheelchairs”, Mobile Robots
Navigation, Intech, pp.609 - 628, 2010.
ICINCO2014-11thInternationalConferenceonInformaticsinControl,AutomationandRobotics
352
