Human Motion Assistance using Walking-aid Robot and Wearable

Sensors

Jian Huang

, Wenxia Xu

, Samer Mohammed

and Zhen Shu

School of Automation, Huazhong University of Science and Technology, Luoyu Road, Wuhan, China

LISSI Lab, University Paris Est Créteil (UPEC), Paris, France

Keywords: Walking-Aid Robot, Wearable Sensors, Fall Detection.

Abstract: An omni-directional walking-aid robot is developed for the elderly in this study. A motion control strategy

of walking-aid robot based on observing human status by wearable sensors is proposed. During normal

walking, the robot is controlled by a conventional admittance control scheme. When the tendency of a fall is

detected, the robot will immediately react to prevent the user from falling down. The distance between the

human Centre of Pressure (COP) and the midpoint of two human feet is assumed to be a significant feature

to detecting the fall events. Dubois possibility theory is applied to describe the membership function of

‘normal walking’ state. A threshold based fall detection approach is obtained from online evaluation of the

walking status. Finally, experiments demonstrate the validity of the proposed strategy.

1 INTRODUCTION

The elderly people population is rapidly increasing

in developing and developed countries. The increase

of human average lifespan escalated the need for

elderly-care technologies (Alwan 2008). This

increase along with a shortage of skilled caregivers

presents an opportunity for robotic applications to

address some of the disparities in elderly patient

care. In addition, as many elderly and handicapped

people suffer from lower extremity deceases,

increasing demand for walking aid devices has

increased. Robotic applications such as walking-aid

robots to assist the elderly in daily activities and to

help the elderly regain independence and an

increased quality of life will play an important role

in the rehabilitative care systems of increasingly

aging societies (Jonsson, 2001) and (Mohammed,

2012).

The current walking-aid robot systems proposed

so far may be classified into two groups according to

mobility factor, i.e., the system moving on the

ground according to the motion of the subjects and

the system giving effects of walking to the subjects

(Lee et al. 2004). The former system is active-type

walker (Dubowsky, 2000) and (Hirata, 2003) which

is driven by servo motor. The latter corresponds to a

system driven by servo brakes and is passive-type

walker (Rentschler, 2003) and (Hirata, 2004).

Most of current walking-aid robots only consider

the motion control when user is in the normal

walking states (Huang, 2008) and (Wakita, 2012),

but did not discuss the motion control when user is

in abnormal gait situation such as falling down.

Falling down is a major cause of fatal injury

especially for elderly and may create a serious

obstruction for independent living (Griffiths, 2008).

The development of walkers should also be based on

improving the ability of interaction based on the data

from the environment and especially the user to

develop monitoring systems especially for fall

detection and fall prevention.

Force control techniques including impedance

and admittance control methods are widely used in

walking-aid robot motion control because they

enable user-friendly Human-Robot-Interfaces (HRI)

that transform interaction forces from the user to the

desired robot motion velocity. Whereas it is difficult

to identify emergency cases (e.g. stumbling or

slipping) by only using force measurement, and

impractical to detect accurate phases of gait training

without human body status, intelligent technologies

such as wearable sensors and model based

estimation approaches allow the realization of

suitable motion control to correctly respond to

emergency cases and gait training phases. Therefore

wearable sensors are used to detected human body

status in this paper.

199

Huang J., Xu W., Shu Z. and Mohammed S..

Human Motion Assistance using Walking-aid Robot and Wearable Sensors.

DOI: 10.5220/0004664101990204

In Proceedings of the International Congress on Neurotechnology, Electronics and Informatics (RoboAssist-2013), pages 199-204

ISBN: 978-989-8565-80-8

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

Wearable device based fall detection method

mainly relies on various sensors to detect the motion

and location of the body of the user (Mubashir,

2013). To analyse the output of a waist

accelerometer, researchers use variance and average,

kurtosis and skewness statistics to realize eight kinds

of activities such as walking, climbing, running,

standing, sitting, and lying prone (Baek, 2004).

Masaki (Masaki, 2002) classified the output of waist

accelerometers during human climbing motion using

fractal analysis method based on a wavelet transform

process. Wang (Wang, 2007) obtained data from a

3-axis waist accelerometer, then used 33 different

time-domain characteristics to classify the five

different types of gaits characteristic of walking on

flat ground, uphill, downhill, upstairs and

downstairs. In this paper, the force sensors are used

to ensure motion control while the wearable sensors

are used to detect user’s gait through fall

classification into three fall gestures.

First, the walking-aid robot and its motion

control algorithm are introduced. The framework of

the fall detection system based on observing human

status by wearable sensors is presented in section 3.

Having such data, the test results of proposed

algorithm are shown in chapter 4. The conclusions

would be the last part of this article.

2 WALKING-AID ROBOT

The prototype of an omni-directional walking-aid

robot proposed in our previous work (Ye 2012) is

illustrated in Fig. 1. The robot consists of an omni-

directional base, a support frame, a motion controller

and a battery system.

Figure 1: Omni-directional walking-aid robot.

The omni-directional mobile base comprises

three commercial Sweden wheels and actuators.

Several passive casters are also mounted on the base

to widen the support area so as to enhance the

stability. Coordinate systems are depicted in Fig. 2.

System {0} is the reference coordinate system and

{1} is a local system fixed to the robot. Three one-

dimensional force sensors are used to measure the

interaction forces between the robot and its operator.

Both the forward and lateral forces can be obtained,

as well as the exerted rotation torque.

Figure 2: Robot coordinate system (top view) and

measurement of interaction forces.

By using the force sensor array, the measured

interaction force vector represented in {2} is denoted

which can be calculated from the measured

forward and lateral forces. During normal walking, a

conventional admittance control is assumed to

generate a comfortable human-robot interaction

(HRI). The control algorithm is then given by

222

RRI



 Mv Bv f



(1)

where

is the robot velocity vector in system {2}.



,,M

xyz

Mdiag M M

is the virtual mass

parameter matrix and



,,B

xyz

Bdiag B B

is the

virtual damping parameter matrix.

Figure 3: The motion control block diagram of the system.

There are many possible walking modes during

the operation of the walking-aid robot. In this paper,

NEUROTECHNIX2013-InternationalCongressonNeurotechnology,ElectronicsandInformatics

200

we divided all possible human walking states into

two modes, ‘normal walking’ and ‘emergency’.

During ‘normal walking’ mode, the admittance

control strategy (1) is assumed. Wearable sensors

are used to detect the occurrence of ‘emergency’

including stumbling or slippage. If an emergent state

is detected, the walking-aid robot is quickly braked

to prevent the operator from falling down. The

whole control architecture is depicted by Fig. 3. The

fall detection approach is illustrated in the following

sections.

3 FALL DETECTION METHOD

3.1 Wearable Sensors

A wearable sensor unit consists of a tri-axial

magnetometer, a tri-axial accelerometer and a tri-

axial gyroscope for measuring the acceleration and

the angular velocity along three orthogonal axes

simultaneously. All sensor units were checked on a

mechanical turntable to establish offset values for

sensor data in addition to obtaining inclination

relationships of the measured values.

The wearable sensor units are placed on five

body segments, including both shanks, both thighs

and waist (see Fig. 4).

Figure 4: Wearable gait recognition sensor units.

3.2 Gait Recognition

Firstly, several coordinate systems are defined to

fulfil the gait recognition (see Fig. 5). System {1} is

the human reference coordinate system which is

fixed at the waist level with the z-axis pointing to

the ground and the x-axis to the magnetic north

direction. The direction of y-axis of system {1} is

obtained following the right hand rule. For each joint

point, there are two coordinate systems. For example,

system



is a coordinate system that is fixed on

point R

and has the same orientation as system {1}.

System



is a sensor coordinate system that is

fixed on point R

with same orientation as a local

sensor coordinate system {s}. The z-axis is

perpendicular to the sensor surface and the x-axis

points upward and is parallel to the sensor surface.

For any other joint point P, similar coordinate

systems {P} and {sP} are similarly defined.

Normally, the sampled wearable gait recognition

sensory data are represented in the corresponding

sensor coordinate system.

Figure 5: Coordinate definition of gain recognition task.

Due to motion acceleration, the accelerometer

output is a synthetic acceleration which consists of

gravity acceleration and motion acceleration.

Kalman filtering algorithm is used to suppress the

adverse impact of motion acceleration, and then to

estimate the gravity acceleration and geomagnetic

intensity in a sensor coordinate system (Zhu 2004).

For any sensor unit, the model of Kalman filter is

given by:

() ()( 1) ()

() ( 1) ()

nnn n

nn n













sAs w

zs v

(2)

where matrix

is composed of gyroscope

measurements



and



The state vector satisfies

()

ggg

xyz x y z

ngggHHH











(3)

where g

, g

and g

are the acceleration components

of gravity represented in system {sP}.

ggg

xyz

HHH

is the geomagnetic intensity vector represented in

system {sP}.

The observation vector satisfies

()

xyzxyz

naaahhh







(4)

HumanMotionAssistanceusingWalking-aidRobotandWearableSensors

201

where

zyx

aaa ,,

are the outputs of accelerometer and

zyx

hhh ,,

are the outputs of magnetometer.

()nw

a covariance matrix of process noise.

()nv

is a

covariance matrix of sensor output noise.

The pitch, roll and yaw angle (







) of any

sensor unit can be calculated as follows:

arctan( / )

gandg

g and g







































(5)

arctan( / )

HandH









































(6)

arctan( / )

yx z

gg g



 

(7)

After estimating these angles of all sensor unit, the

coordinate values of all joint points (L

and R

with

i=1, 2, 3) and segment COG points (G

with i=1,

2, …, 5) can be easily obtained because the length of

each segment is known. Then, the real-time gait is

well described by all the point coordinate values.

3.3 Fall Detection Method

We investigated forward, left lateral and right lateral

fall in this study. The Centre of Pressure (COP)

point, which is equivalent to the well-known Zero

Moment Point (ZMP), is a good feature to decide

whether a fall occurs (Lee, 2006). It is observed that

while human is walking, the COP is nearby the

supporting foot. If the distance between the COP and

supporting foot suddenly increases, a fall may

happen.

In the case of an elderly operating the walking-

aid robot, the acceleration of human body is usually

small. The COP can then be approximated by





cop

x ,

(8)

where

(, )

cop cop

y is the coordinate value of

approximated COP on the ground.

is the mass of

the i-th segment of human body and

(, ,)

iii

yz is

the coordinate value of point G

in system {1}.

To identify whether a fall occurs, a significant

feature is assumed based on the relative position

between the COP and the midpoint of two feet. This

feature is denoted by vector d (see Fig. 5), which

can be calculated as

33 33

LR LR

cop co

xx yy







 









(9)

where

333

(, ,)

LLL

yz and

333

(, ,)

RRR

yz are the

coordinate values of points L

and R

Empirically, when the user is falling down the

norm of d will increase suddenly in a certain

direction. We investigate the distribution of

during the normal walking state by Dubois

possibility theory (Dubois 2003). Similar

distribution was regarded as normal distribution and

estimated to infer the user's walking state.

Firstly, the probability distribution

},2,1:)({ hjnp



is calculated by dividing the

height of each bin by the total number of sample

points of normal walking.

is the number of bins

for a histogram. Each bin is represented by the

centre of interval denoted by n

. The possibility

distribution

},2,1:)({ hjn







is deduced from

the probability distribution by the bijective

transformation of Dubois and Prade defined by





jkk

npnpn

)](),(min[)(



(10)

Note that the smaller

is, the more stable the

human walks. We propose the membership function

)(





for ‘normal walking’ as follows:







()max







(11)

Based on the above definitions, a very simple fall

detection algorithm can described as: (assuming the

human walking behaviour is monitored at discrete

times, t denotes the current time)

Algorithm 1: (Fall detection).

(())tc





AND

(( 1))tc



d

AND … AND

(( ))tk c





d

THEN a fall is detected.

NEUROTECHNIX2013-InternationalCongressonNeurotechnology,ElectronicsandInformatics

202

where constant c is a small positive number which

indicates a very low possibility of ‘normal walking’

state. k is assumed to remove disturbance.

4 EXPERIMENT

In this section, several experiments were performed

using wearable sensor group to detect human

walking gait and falls, as described previously to

verify the validity of fall detection method.

Firstly, the possibility distribution of ‘normal

walking’ was obtained as well as the membership

function used in the fall detection algorithm. The

subject was requested to walk normally for about

two minutes. The trajectory of

is shown by Fig.

6. Assuming h=10, the probability distribution

()

n , possibility distribution ()



and

membership function

()



for ‘normal walking’

are calculated and depicted in Fig. 7.

Figure 6: Trajectory of

during ‘normal walking’.

Figure 7: Probability, possibility distributions of

and

the membership function of ‘normal walking’.

After obtaining the membership function, several

fall detection experiments were performed. The

subject was requested to intentionally fall forward,

left and right after walking along with the robot for a

while. Constant c is chosen as 0.02. The experiment

results of the three cases of falling down are

illustrated in Fig. 8, 9 and 10. Applying fall

detection algorithm 1, the falls were detected around

10 [sec], 11 [sec] and 15 [sec]. This is coincident to

the real experiment and the robot was safely braked

to prevent further falling down.

Figure 8: Experiment results of fall forward.

Figure 9: Experiment results of left lateral fall.

Figure 10: Experiment results of right lateral fall.

5 CONCLUSIONS

In this paper, we proposed a motion control of

walking-aid robot based on observing human status

by wearable sensors. During normal walking, the

robot is controlled by a conventional admittance

HumanMotionAssistanceusingWalking-aidRobotandWearableSensors

203

control strategy. If any fall is detected by wearable

sensors the robot will stop immediately to prevent

the user from falling down. The proposed fall

detection scheme is based on a threshold approach

considering the distance between the COP and

midpoint of two feet of user. Possibility theory was

applied to describe the membership function of

‘normal walking’. The effectiveness of proposed

methods is confirmed through experiments.

ACKNOWLEDGEMENTS

This work was supported by the International

Science & Technology Cooperation Program of

China "Precision Manufacturing Technology and

Equipment for Metal Parts" under Grant

No.2012DFG70640 and by International Science &

Technology Cooperation Program of Hubei Province

"Joint Research on Green Smart Working Assistance

Rehabilitant Robot" under Grant No.

2012IHA00601.

REFERENCES

Alwan, M., Felder, R. A., 2008. Eldercare Technology for

Clinical Practitioners Totowa, NJ: Humana Press.

Baek J., Lee G., Park W., et al. 2004. Accelerometer

Signal Processing for User Activity Detection.

Knowledge-Based Intelligent Information and

Engineering Systems Lecture Notes in Computer

Science, 32(15): 610-617.

Dubowsky, S., Genot, F., Godding, S., K. Skwersky, H. A.

Yu, H., and Yu, L., 2000. Pamm—A robotic aid to the

elderly for mobility assistance and monitoring: A

“helping-hand” for the elderly, Proceeding of IEEE

International Conference on Robotics and

Automation, San Francisco, CA, 570-576.

Dubois D., Prade H., 2003. Possibility Theory and Its

Applications: A Retrospective and Prospective View.

Proceeding of IEEE International Conference on

Fuzzy Systems, 25-28.

Griffiths, C., Rooney, C., Brock, A., 2008. Leading causes

of death in England and Wales—how should we group

causes? Health Statistics Quarterly 28. Office for

National Statistics.

Hirata, Y., Baba, T., and Kosuge, K., 2003. Motion

control of omni-directional type walking support

system ‘walking helper’, Proceeding of ROMAN,

Millbrace, CA, 85-90.

Hirata, Y., Hara, A., Kosuge, K., 2004. Passive-type

Intelligent Walking Support System “RT Walker”,

Proceedings of the 2004 IEEE/RSJ International

Conference on Intelligent Robots and Systems, 3871-

3876.

Huang J., Di P., Wakita, K., Fukuda, T., Sekiyama, K.,

2008. Study of Fall Detection Using Intelligent Cane

Based on Sensor Fusion. 2008 International

Symposium on Micro-Nano Mechatronics and Human

Science, 495-500.

Huang, J., Di, P.; Fukuda, T., Matsuno, T., 2008. Motion

control of omni-directional type cane robot based on

human intention. IEEE/RSJ International Conference

on Intelligent Robots and Systems, IROS, 273-278.

Jonsson, L., 2001. The Importance of the 4-

WheeledWalker for Elderly Women Living in their

Home Environment, The Swedish Handicap Institute.

Lee, C. Y., Jeong, I. K., Lee, I. H., 2004. Development of

Rehabilitation Robot Systems for Walking-Aid,

Proceeding of IEEE International Conference on

Robotics & Automation, New Orleans, LA. 3: 2468-

2473.

Lee H. J., Chou L. S., 2006. Detection of Gait Instability

Using the Center of Mass and Center of Pressure

Inclination Angles. Archives of Physical Medicine and

Rehabilitation, 87(4): 569-575.

Masaki S., Toshiyo T., Metin A., etc al. 2002.

Discrimination of Walking Patterns Using Wavelet-

Based Fractal Analysis, IEEE transactions on neural

systems and rehabilitation engineering, 10(3): 188-

196.

Mohammed, S., Yacine A. and Hala R., 2012. Lower-limb

movement assistance through wearable robots: state of

the Art and challenges. Advanced Robotics, 26(1-2): 1-

22.

Mubashir, M., Shao, L., Seed, L., 2013. A survey on fall

detection: Principles and approaches.

Neurocomputing, 100(16): 144-152.

Rentschler, A. J., Cooper, R. A., Blaschm, B., Boninger,

M. L., 2003. Intelligent walkers for the elderly

Performance and safety testing of VA-PAMAID

robotic walker, Journal of Rehabilitation Research

and Development, 40(5): 423-432.

Wang N., Eliathamby A., Nigel H. L., Branko G. C., 2007.

Accelerometry Based Classification of Walking

Patterns Using Time-frequency Analysis, Proceedings

of the 29th Annual International, Conference of the

IEEE EMBS, Lyon, France, 4899-4902.

Wakita K., Huang J., Di P., Sekiyama K., Fukuda T.,

2012. Human-Walking-Intention-Based Motion

Control of an Omnidirectional-Type Cane Robot.

IEEE/ASME Trans on Mechatronics. (Online

Available)

Ye J. Y., Huang J., He J. P., Tao C. J., Wang X. T., 2012.

Development of a width-changeable intelligent

walking-aid robot. IEEE International Conference on

Micro-Nano Mechatronics and Human Science, 358-

363.

Yu H., Spenko M., Dubowsky S., 2003. An adaptive

shared control system for an intelligent mobility aid

for the elderly, Autom. Robots, 15(1): 53-66.

Zhu R., and Zhou Z. Y., 2004, A Real-Time Articulated

Human Motion Tracking Using Tri-Axis

Inertial/Magnetic Sensors Package. IEEE Transactions

on Neural Systems and Rehabilitation Engineering,

12(2): 295-302.

NEUROTECHNIX2013-InternationalCongressonNeurotechnology,ElectronicsandInformatics

204