Towards gait modeling for fall prevention

Pélissier Loïc, Boyer Anne, Charpillet François

LORIA – INRIA Lorraine

615 rue du jardin botanique

54600 Villers-les-Nancy

France

Abstract. This paper describes a monitoring system prototype dedicated to fall

prevention for elderly. Our system designed in collaboration with physicians,

aims at assessing the risk that a fall occurs, relying on the signature of the walk.

The signature is defined as a set of relevant characteristics, computed from the

observation through a video camera of the behavior at home of the elderly and

can be viewed as a self reference for the telecared person. We model the devia-

tion of this signature to determine an increase of the fall risk.

1 Introduction

Elderly people falls is a major problem of health services because of its frequency

and their medical and social consequences. They have a great impact on the everyday

life of the elderly and their relatives. Let us remind that falls are, for elderly people,

the first cause of accidental death in the USA

[1].

In France, about two millions

[2] of people over sixty five year’s old fall every

year. This number is probably an under estimation since only five to ten percents of

the falls occurring at home are known by the doctors

[3] and most of the falls occur-

ring in institutions are not declared. Falls are responsible for an important morbidity

(50 000 fractures of the thighbone upper extremity each year in France)

[4], a mortal-

ity evaluated to 13 000 death per year (it is more than the death due to road acci-

dents). The risk of mortality is multiplied by four during the year following the fall in

respect with the same population’s age, whatever the gravity of the fall. Finally, falls

are responsible for an important risk factor of autonomy loss because almost 40% of

the elderly hospitalized for a fall will be institutionalized afterwards.

As fall is a major risk, it is a key challenge to design systems able to detect falls as

fast as possible in order to alert the relevant assistance service. Another crucial aspect

is of course to prevent degradations in the health state of the elderly that could in-

crease the risk of a fall, in order to act before the accident occurs either by a modifi-

cation of the treatment or any else appropriate actions.

Our work deals mainly with this second objective, with some strong constraints as

the guaranty of the integrity and the privacy of the elderly. Furthermore, the system

must be easy to install without any degradation or modification of the elderly envi-

Loïc P., Anne B. and François C. (2004).

Towards gait modeling for fall prevention.

In Proceedings of the 1st International Workshop on Tele-Care and Collaborative Virtual Communities in Elderly Care, pages 97-108

DOI: 10.5220/0002680500970108

 SciTePress

ronment, it must be cheap in order to be accessible for everyone without social dis-

crimination.

Firstly, this paper will describe the context of the fall prevention in a usual envi-

ronment and the main initiatives in this domain. Then, we will detail our approach of

an intelligent monitoring for fall prevention based on an ecological analysis of the

walk. We will present a first experimentation before concluding.

2 The elderly’s fall study in a natural environment.

The old persons are often worried of keeping their independence, whatever are the

necessary efforts needed to manage themselves alone at home. The concept of intelli-

gent home supplies an answer to this expectation by proposing information technolo-

gies based services. Detection and control systems used together with telemonitoring

or telemedicine systems can contribute to produce safer and more efficient houses.

Telemonitoring is the action of monitoring remotely, by connecting occasionally or

permanently to the place of residence of the monitored subject by telecommunication

means and modern computing.

The telemedicine is defined like the transmission of medical data for diagnosis or

therapeutic purpose, relative to a patient for his/her individual benefit. The telemedi-

cine is rapidly expanding thanks to the progress of information technologies, commu-

nication and headways in the sensor field.

Telemonitoring for medical purposes is expanding quickly because of the more

and more urgent need to maintain people in loss of autonomy or prone to falls (lonely

old persons, handicapped, or suffering from chronic diseases like heart, respiratory,

neurological pathology) at home.

Our work is situated in the medical attentiveness field and more specifically in the

field of continuous evaluation of elderly security and comfort at home. It consists of

detecting degradation’s trend from the observation of the behavior of a person at

home and evaluating the rise or improvement of the fall’s risk. In order not to disrupt

the life of the person, we choose not to instrument him/her. The sensors are then only

situated in the environment (pervasive computing).

We intend to lead a study in order to determine the different evidences linked to

the elderly fall. This study shall result in a fall’s risk diagnosis help tool able to alert

the circle of acquaintances of the considered person before an accident occurs. This

study relies on a walk and balance (dynamic or static) analysis of a person in his/her

natural, daily environment. It requires a strong doctor’s implication to determine the

appropriate parameters for a relevant and lawful diagnosis.

This study comes along with the implementation of a supervised experiment. It

consists in equipping the room of an elderly with environmental sensors. These sen-

sors are non intrusive, compliant with different medical sensors (pacemaker, prosthe-

sis…) and nevertheless, allowing a precise follow-up of the indicators determined in

collaboration with doctors. The patient can live alone or not, which is an important

advantage of our work. We thus work on the realization of a walk analysis system

(with the aim of a use in free or supervised environment, in order to evaluate the

correlations between measured parameters and fall’s risk) from picture resulting of a

video camera. The camera is used like a position and presence detector. In fact, the

only output of the system will be the behavioral model parameter values, and no im-

ages or video stream are available from a remote location in order to ensure privacy.

In the field of telemonitoring, studies are led to create behavior’s models. Today, the

only not binding actimetry sensors available are infrareds. These sensors rapidly

show their limit because they can only supply binary information (presence detected

or not) in their filed of action. This raises three problems:

− No information on the target

− Localization problem: we know that somebody (or something) is in a given sector,

but we have no information on his/her/its real position in this space.

− Differentiation problem: A detected presence doesn’t always mean a detected

person, it could be a pet for example.

Our work consists in using a video camera to create an evolved motion sensor,

able to solve these problems. The video camera is transformed in a presence sensor

capable of detecting one or more persons, to differentiate these persons and to supply

relevant and reliable information such as trajectory, height, speed and orientation of

the tracked person. This sensor will then be integrated, with other sensors, in a risk

prone environment, in order to create behavior’s models. These models are con-

structed in an incremental way, with the collaboration of the doctors specialized in

walk study, from the collected data. Eventually, this evolved sensor will become an

intelligent sensor able to detect falls and behavior’s drift (like night and day inver-

sion). A working group is constituted, including geriatricians and specialists of the

walk and its rehabilitation to determine pertinent characteristics that have to be meas-

ured.

3 State of the art

Telemedicine projects around the elderly welfare are numerous, proposing various

approaches. Some are futurist like healthcare and rehabilitation robots

[5], [6], [7],

some others seems more reasonable, like pervasive intelligence (intelligent environ-

ment or home), but we can also quote many others, like healthcare through Internet

(ubiquitous healthcare) and so on. It is impossible to list all the works, but let us men-

tion only some specific projects that seems representative of what is being done.

3.1 Monitoring people for healthcare purposes

Remote patient monitoring is becoming an important step for our future. During these

last years, many projects of telemonitoring for healthcare purpose were initiated. For

instance, DIATELIC is a service to secure dialyzed patient at home with the estima-

tion of the hydration rate via a remote markovian expert system accessible through

the web, improving life of the monitored patient, thus making the government health

services to spare ressources

[8]. This project aimed at simplifying the life of the pa-

tient, but also to allow the doctor to follow more patients with a computer decision

support. The information that each patient submits and the expert system diagnostic

are available for the doctor on the web and he could make his/her own diagnostic by

the same way, reducing the number of time the patient had to come to the hospital for

a check up.

This kind of project is very interesting in a societal point of view, but it addresses

one kind of telemonitoring, since the period of monitoring is very short and delayed

in time.

The problem we are interested in is the monitoring in real time of a person’s be-

havior. This problem addresses another field, because it consists of a continuous

measurement of various parameters. Few real experiments are being led at the time,

we can quote the HIS project

[9] from the TIMC-IMAG which aim to help people in

loss of autonomy to remain at their home instead of being institutionalized. It consists

of an apartment that is equipped with various sensors (mainly, infrareds for presence

detection, but also a scale, microphones, etc…). At first, the system learns a model of

the monitored person’s behavior. Once this model is created, the sensors are continu-

ously monitored, the measures are merged to detect abnormality in the person’s

measured behavior. We can also quote the PROSAFE project

[10] of the LAAS labo-

ratory, which aim to identify the circadian cycle of a person through the instrumenta-

tion of a hospital room with sensors (infrared, temperature,…). They have established

a model of the activities of the monitored user. The final objective of this project is to

detect abnormalities in the behavior of the monitored user. The department of bio-

medical instrumentation, the biomaterial institute and the medical and Dental Univer-

sity of Tokyo

[11] led another similar project. Various sensors were installed in an

apartment (once again, mainly infrareds, but also temperature, humidity, and so on) in

which they made an experiment on three persons for a month each. From the ac-

timetry measurement, they have established profile of the monitored person.

All these projects have a common view of the problem: In order to design a non-

invasive monitoring, the patient mustn’t be equipped in any way. His/her usual life

and customs had to be protected.

3.2 Video camera based monitoring and gaits studies

Each individual possesses a specific stride style. Thus, the stride establishes an emer-

gent behavioral biological metric. The interest is that such an evaluation is possible

remotely, without any specific interaction with the subject. The signature (set of gait’s

characteristics) has mainly been used for people identification or classification. These

experiments are assuming strong and restrictive hypothesis: fixed video camera posi-

tion, walking speed almost constant, fronto-parallel movement relative to the camera,

subject completely recognizable, constant luminosity, … These experiments are con-

ducted in laboratories and no real application is demonstrated. We can distinguish

two kinds of different approaches: The calculation of periodic spatio-temporal im-

plicit models

[12], [13], which need an history, and so, are often calculated offline, and

the calculation of characteristics (speed, stride’s length, …) generally in real time

[14],

[15].

These previous works have put in evidence the notion of signature upon which re-

lies our methodology. Our model relies on the notion of step’s signature that could be

defined as a set of specific parameters involved in the determination of fall risk diag-

100

nosis. Our study deals with the determination of the good features and first of all,

with the validation of the notion of signature to measure a risk of fall. Moreover, the

signature is a individual characteristic, each patient constituting his/her own refer-

ence. We are studying the evolution of the signature over time.

3.3 Our approach

Our objective is to create a non-invasive monitoring of the elderly subject’s locomo-

tive behavior in his/her environment in order to isolate parameters which values will

serve as an alert signal. It will thus be a question of in-place gait’s telemonitoring.

The challenge is to isolate, in a resulting behavior, factors that may indicate a risk

of fall. It doesn’t aim to precise the etiopathogeny of the fall (multifactor etiology

linked with individual factors, environmental factors and also a part of chance), nei-

ther to search the observed behavior modification’s origin.

The generated alarm signal would allow the regular doctor to take etiologic or

therapeutic actions (drug-related side effects, another pathology, medical decision

help…).

3.4 Characteristics and interests

The interests of this kind of projects are numerous. We can discuss the economic

interest of such intelligent environments in our societies. Specific health expenses

bound to aging of the population are constantly increasing, and due to the global

population aging, this problem is only at its beginning. This will result with a need for

more specialized institutions and increased nursing staff. Allowing the patient to

remain at his/her home will lower these needs and therefore, the cost of those specific

healthcares. The fact that a patient is able to stay in his/her home is a « societal »

bonus because he won’t have to change his/her way of life for safety.

Finally, a reason motivated by the doctors is that such a system could help them

determining the causes of the falls. In most cases, these causes are unknown and, as

we have said it before, probably multiple. A system that could help knowing this

causes, or explaining falls would be very useful in order to take preventive actions.

3.4.1 Our study

In the following, we will expose the principles of our study. Our approach decom-

poses itself in various steps, as presented below:

Fig. 1. Decomposition of our approach

Principle:

− Feasibility study

− Realisation of an experimentation

− Results analysis

− System improvements

101

The different points of our approach are detailed in the following.

3.4.1.1 Feasibility study

A first feasibility study was led in LORIA with the collaboration of geriatricians

and specialists of the walk and its rehabilitation. With their help, we determined sim-

ple to measure or calculate, yet relevant, parameters to initiate the process. Then, we

developed the algorithm allowing us to measure these parameters and finally, we

realized a first experimentation in order to prove the reproducibility of the results and

the validity and accuracy of the measurements.

3.4.1.2 First experiment description

The first experimentation aimed at determining whether it is possible to detect ab-

normalities and took place in our laboratory. We placed a standard CCD color video

camera at a fixed position in the angle of a corridor, and we recorded about twenty

volunteers walking back and forth in that corridor. The acquisition was performed in

a resolution of 320x240 pixels in a 24 bits RGB color space. The corridor is 1,56

meters width and we monitored people on a 5 meters length section of the corridor.

The monitored plane is shown on the figure 2.

Fig. 2. Sample snapshot of the corridor.

The motion detection was performed by a difference between consecutives images

and a reference image of the scene, thresholding the results to obtain blobs represent-

ing the moving objects of the scene

[16][17], after a pass of noise elimination, a de-

convolution is performed to address the lens distortion. The reference image is up-

dated after every step of calculation to offer more robustness to natural illumination‘s

variations. We decided to use a simple difference of images as motion detection algo-

rithm because we wanted the behavior to be evaluated in real time, so other methods,

like optical flow

[18], were too expensive in calculation time, thus resulting in a poor

frame rate (~0.3 frame/sec). It was not suitable for the accuracy of some calculated

parameters, like the trajectory since it can be used to determine an approximation of

the stride length.

102

()

⎩

⎨

⎧

<−

≥−

TyxByxIif

yxM

),(,0

),(,1

(1)

Mt is the motion picture obtained at time t, It is the image extracted from the video

stream at time t, B is the background image and T the threshold.

Fig. 3. This image present successive image of the video stream, and the associated detected

moving blobs surrounded by their bounding boxes.

The indicators are then calculated with the detected form in motion. The first pa-

rameters that are calculated are the position on the ground floor relative to the cam-

era, the height of the detected shape, and also its orientation, which is the angle

formed by the shape with the ground floor. From the consecutive positions, we obtain

a time-stamped trajectory, which enable us to obtain the immediate speed and accel-

eration. From this point, we are able to determine a mean speed. A vector is com-

posed of these informations. It is the first basis for our observations and these meas-

urement vectors constitutes the gait’s signatures.

This set of parameters, used as a first estimation of a signature, gave us encourag-

ing results as for the precision and the reproducibility of the calculated parameters so

we decided to take this set of parameters as the starting point for our model of the

walk.

3.4.1.3 Results

The computation was done afterwards on the films and presented under the form

of graphs for readability reason. As we expected, some indicators have very little

variation (like the calculated height, for example) but, even thought the frame rate

wasn’t very high (~11 fps), we saw emerging pattern from the curves. One of them in

particular holds our attention. Its pattern was a bit different from the others. After a

little chat with the considered person, it was established that she had an imperceptible

limp due to an old accident. We also chat with all the other participants of the ex-

periment, and established that nobody else had such kind of stride problem.

103

Speed

MeanSpeed

Acceleration

Speed

MeanSpeed

Acceleration

Fig. 4. The first curve is obtained from a person with no walking problem, the second one is

obtained from a person with a limp problem.

In abscissa is the time. As we can see, on the first graph, the curves are somewhat

regular with an almost constant period of acceleration and speed oscillation. But, on

the second graph, we can see a pattern emerging on the speed and acceleration, with

an almost constant period. This has encouraged us, because it seems that this simple

model can be used as is to detect some walking abnormalities like limp, so we de-

cided, with the doctors to go further in this way and created a project.

3.4.1.4 System improvement

We have improved the acquisition and computation algorithms in order to increase

the frame rate at least to twenty-five images per second, which also mean we increase

the sampling rate of our observations. The critical factor is still the deconvoluting

filter used to fix the lens distortion which increase dramatically the video processing

time, but this correction is essential to ensure the accuracy of the measurements.

104

Another problem we have to address concerns the future users of the system. The

patient’s acceptation of the changes it means in his/her environment should be kept to

a minimum and the respect of his/her private life should be guaranteed. To ensure the

respect of his/her private life, the video stream is processed locally and will never be

accessible from another network. Only the diagnosis, the alarms/alerts and some

calculated parameters will be available from a remote location by the doctor. But this

will definitively not be sufficient for a full acceptation. We need to explain to the

patient the benefits he/she will get from the installation and to inform him/her about

what is really going on in his/her home in a simple, yet accurate manner.

A limitation is, of course, the time needed to realize real tests. In the case of fall

detection, we could simulate it, but it won’t be revealing of a real fall case. On the

other hand, a study in a real environment will take a lot of time as the behavior’s

degradation is very slow, and falls occurs occasionally (fortunately).

The last, but not least point is the scalability. An apartment is not a corridor, there

will be no fixed walk position. We have to take into account many artifacts like pets,

or new furniture or a change in the disposition of the scene, or moving objects and

address many problems like overlapping of multiple peoples or objects (like chairs).

In order to address some problems highlighted by the scalability, we could make a

hypothesis: we don’t need to monitor a whole room, but we should rather focus on

some risks prone zones. These zones have yet to be identified with a clear methodol-

ogy. We hope that our system will help us to create this methodology, or find heuris-

tics to identify this kind of zones, thus reducing the monitored spaces to the mini-

mum.

Finally, we created a French RNTS project with the doctors. RNTS stands for Na-

tional network of technologies for health, it has been collectively created by the

French new technology and research ministry and the French’s industry ministry. The

purposes of RNTS projects are to reinforce the innovation potential in the healthcare

domain, to identify priority holder themes and to help the creation of companies.

Our project aims at creating a real experimentation center within a retirement's

home, with the collaboration of the staff and the pensioner. We will therefore be able

to acquire corpuses and lead a long-term experimentation. We might discover new

relevant indicators that can complete our parameters. Finally, we are planning to use

modeling technique to process these parameters automatically.

4 Conclusion and future work

Since now, we have implemented the basis of the system, which include the video

acquisition and processing, the parameter calculations and presentation. The doctors

agreed that the causes of a fall could be numerous. As a consequence, it won’t be

easy to find relevant evidences. For that reason, we want to extend our parameter set

with other parameters. For instance, we envisage to use biological factors an in-place

evaluation of the static and dynamic balance of the subject, and other non biological

related factors like spatio-temporal models (frieze patterns

[19]…).

As we have seen in the limitation section, this will be a long-term project that will

involve many consecutive steps. At first, we plan to study a behavior degradation

105

model on the basis of regular observations in a test environment in order to identify

the relevant factors of dynamic balance. We already have identified easily measurable

parameters, which gave us reliable and reproducible results. Now we are interested in

the detection of variations in those parameters over the time, which are linked with

the fall’s risk.

There is a rising use of Hidden Markov Models in vision based monitoring sys-

tems. We have seen them used to make classification of office room activities

[20],

parking lot activities

[21] and it has also been successfully tried in gait classification

[22].

Because our calculated parameters define rough observations of a walking behav-

ior, it seems that the Hidden Markov Model is adequate to process these observations.

The advantage of the HMM is the ability to learn a normal walking model from

our parameters in a first step, then to process the same parameters, calculated in a real

situation, and detecting any abnormality in those parameters sequences, like a degra-

dation of the behavior. An interesting point is that the learned model is the model of

the tracked person, meaning that each person is his/her own reference. The model we

propose is the following:

Fig. 5. The proposed hidden markov model.

This HMM models the tracked person’s behavior. Each of its state maps the state

of the walking subject. We considered that if someone fall, he/she couldn’t go back to

a normal or risky state, which explain that the state “fall” is an absorbing state.

If this model proves itself to be relevant enough, we foresee to test a partially ob-

servable Markov decision process to process these information, giving us the possi-

bility to include actions in our models, which will be very useful when we will need

to generate alarms or alerts. Eventually, we will integrate the resulting model, in a

new sensor, based on a video camera, so the complete system will gain in discretion

and affordability.

After few meetings, doctors (specialists on walk study) have shown their interest

in our system, and even found new motivations for it that we hadn’t thought of, like

the ability, by maintaining an history, to identify unexplained causes of falls. Work-

ing in collaboration with practitioners is essential for us because they are our main

advisors and experts about what information we should focus on. They also are the

only people that can evaluate our results in an objective way. The next step is to cre-

ate an experimentation room with the doctors, in order to acquire corpuses that will

be used to train, validate and refine the model (and maybe even the information ac-

quisition).

rmal

ll ri

Fall

106

References

1. Hindmarsh J. J., Estes E. H., Jr., “Falls in older persons. Causes and interventions ”,

Arch Intern Med, 1989, 149(10):2217-2222.

2. Tavernier-Vidal B., Camus A., Mourey F., Kagan Y., “Chutes et troubles de la fonc-

tion d'équilibration ”, Encycl Méd Chir (Elsevier, Paris) Encyclopédie pratique de

Médecine, 1998, 3-1135:8 p.

3. Meyer P, Boucher D, Martinet N, André JM., “Troubles attentionnels et chutes du

sujet âgé”, Ann Méd Nancy, 1998, 37(2):145-156.

4. Baudoin C., Fardellone P., Potard V., Sebert J. L., “Fractures of the proximal femur

in Picardy, France, in 1987”, Osteoporos Int, 1993, 3(1):43-49.

5. M. Montemerlo, J. Pineau, N. Roy, S. Thrun, and V. Verma, “Experiences with a

Mobile Robotic Guide for the Elderly”, Proceedings of the AAAI National Confer-

ence on Artificial Intelligence 2002

6. N. Roy et al., “Towards Personal Service Robots for the Elderly”, in Proceedings of

the Workshop on Interactive Robotics and Entertainment (WIRE), Pittsburgh, PA,

2000.

7. S. Dubowsky, F. Genot, and S. Godding, “PAMM: A robotic aid to the elderly for

mobility assistance and monitoring: A ”helping-hand” for the elderly,” in IEEE In-

ternational Conference on Robotics and Automation (ICRA), San Francisco, CA,

2002, ICRA.

8. Jeanpierre L, Charpillet F , “Remote monitoring of chronical pathologies using per-

sonalized Markov models ”, CIMED 2003 ( International Conference on Computa-

tional Intelligence in Medicine and Healthcare, IEEE conference, 6 pages ).

9. Rialle V., N. Lauvernay, A. Franco, J.-F. Piquard, P. Couturier, “ A smart room for

hospitalised elderly people : essay of modelling and first steps of an experiment”,

Technology and Health Care 1999, 7(5), 343-57.

10. M. Chan, H. Bocquet, E. Campo, T. Val, J. Pous, “Alarm communication network to

assist carers of the elderly for safety purposes : a survey of a project”, International

Journal of Rehabilitation Research, vol. 22, p. 131-136, 1998.

11. M. Ogawa, T. Togawa, “ Monitoring Daily Activities and Behaviors at Home by Us-

ing Brief Sensors ». 1st Annual International IEEE-EMBS Special Topic Conference

on Microtechnologies in Medicine & Biology, October 12-14, 2000, Lyon, France,

pp.611

12. S. Niyogi et E. Adelson, “ Analysing and recognising walking figures in xyt ”, Com-

puter Vision and Pattern Recognition, pp. 469-74, 1994.

13. A. Kale, A. N. Rajagopalan, N. Cuntoor, et V. Krueger, “ Gait based recognition of

human using continuous HMMs ”, Proceedings of Automatic Face and Gesture Rec-

ognition, Washington, DC, USA, 2002.

14. P. Huang, C. Harris, et M. Nixon, “ Comparing Different Template Features for Rec-

ognizing People by their Gait ”, British Machine Vision Conference, University of

Southampton, UK, pp. 639-48, 1998.

15. C. BenAbdelkader, R. Cutler, et L. Davis, “ View-invariant Estimation of Height and

Stride for Gait Recognition ”, Workshop on Biometric Authentication of the Euro-

pean Conference on Computer Vision, Copenhagen, Denmark, pp. 155-67, 2002.

16. Wenstop O., “ Motion detection for image information ”, In 3th Scandinavian con-

ference on image analysis. 1983. pages 381 - 386, Tromso, Norway.

17. Shapiro L, Stockman G. Textbook: Computer Vision 2001, p. 275-303

18. Beauchemin and Barron, “ The Computation of Optical Flow ”, ACM Computing

Surveys, 27:3, pp. 433-467, 1995

107

19. Y. Liu, R. Collins, et Y. Tsin, “ Gait Sequence Analysis using Frieze Patterns ”, Ro-

botics Institute, Carnegie Mellon University, Technical Report CMU-RI-TR-01-38,

December 2001.

20. M. Brand, V. Kettnaker, “ Discovery and segmentation of activities in video”, IEEE

Transaction on pattern Analysis and machine intelligence, vol. 22, no. 8, pp. 844-

851, 2000

21. Y. A. Ivanov, A. F. Bobick, “ Recognition of visual activities and interactions by sto-

chastic parsing ”, IEEE Transaction on pattern analysis and machine intelligence,

vol. 22., no. 8, pp. 852-872, 2000

22. D. Meyer, « Human gait classification based on hidden Markov model », 3D Image

Analysis and Synthesis, Sankt Augustin, pp. 139-46, 1997.

108