Experimenting an Embedded-sensor Network for Early Warning of

Natural Risks Due to Fast Failures along Railways

Andrea Fantini

, Massimo Magrini

, Salvatore Martino

, Davide Moroni

, Gabriele Pieri

Alberto Prestininzi

and Ovidio Salvetti

Institute of Information Science and Technologies, National Research Council, Via Moruzzi, 1, Pisa, Italy

Department of Earth Sciences and Research Centre for Geological Risks (CERI), University of Rome “Sapienza”,

Piazzale Aldo Moro, 5, Rome, Italy

Keywords: Real-Time Imaging, Embedded Systems, Natural Risk Monitoring, Fast Failures, Railway Monitoring.

Abstract: This paper deals with a project for real-time monitoring of railway tracks to detect events, such as fast

failures from natural risks, which may threaten the transit of trains. The paper describes a network of smart

sensors for early warning of these endangering events. Three main types of fast-failure events involving

railways were identified: sinkhole, rock and debris falls. A case study on a known test site and

experimentation with various scenarios were carried out with a view to developing algorithms capable of

spotting and localising them. Results demonstrate the good performance of the network in monitoring the

investigated events.

1 INTRODUCTION

In the last decade, particular attention was focused

on the monitoring of fast to very fast failures, which

include landslides from rocky slopes (e.g. falls,

topples and wedge sliding), but also sinkholes on

plains and debris flows channelled along high-

inclination slopes. The relevance of such events is

mainly related to the short time available for taking

action in case of exposed infrastructures (highways,

railways and so on), since no significant

displacements are generally detected before failure.

In this regard, two are the possible strategies to

manage the natural risk: i) monitoring precursors by

using micro- or nano-seismometric devices as well

as of acoustical emission records (Amitrano et al.,

2005; Lenti et al., 2012); ii) monitoring the site as

well as the exposed infrastructures, by using optical

devices (e.g. cameras, interferometers, videos)

capable of detecting fast morphological changes or

abnormal and unexpected objects hazardous for the

infrastructure (Antonello et al., 2004; Lai et al.,

2006; Gaffet et al., 2010; Bigarre et al., 2011;

Martino and Mazzanti, 2014).

Integrated monitoring systems should be

designed for this purpose and adapted to meet the

following requirements: i) investigating or detecting

the site at different evolutionary stages (forward

prevention), corresponding to different distributions

of the landslide hazard; ii) understanding and

controlling the parameters for forecasting the short-

term evolution of gravitational instabilities (e.g.

high-velocity landslides) and for planning alert

systems (real-time prevention).

Experiments were conducted on a railway where

a Wireless Sensor Network (WSN), with hardware-

connected cameras, had been installed. The WSN

was intended to detect morphological anomalies,

such as rocky slopes prone to falls, and the presence

of unexpected objects along the monitored railway

track. A WSN, which can transmit real-time data,

can provide an early warning system.

The experiments were carried out by artificially

causing block falls and localising them along the

railway. The experiments were part of a research

project, implemented by the Research Centre for

Geological Risks (CERI) of the University of Rome

“Sapienza”. The project will include new sites to be

monitored, where natural ongoing processes can be

detected. The end goal of the research is to integrate

the WSN into a multi-sensor network for detecting

natural fast-landslide processes from precursors to

failure.

Fantini A., Magrini M., Martino S., Moroni D., Pieri G., Prestininzi A. and Salvetti O..

Experimenting an Embedded-sensor Network for Early Warning of Natural Risks Due to Fast Failures along Railways.

DOI: 10.5220/0005462200850091

In Proceedings of the 5th International Workshop on Image Mining. Theory and Applications (IMTA-5-2015), pages 85-91

ISBN: 978-989-758-094-9

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

2 ISSUE AND ANALYSIS

The infrastructure chosen for the project was a

railway; in particular, some railway tracks with

limited train transit (secondary railways) in Central

Italy were selected. The experimental activity

consisted in monitoring the railway and the

surrounding area, in order to derive a real-time

report of obstacles hazardous for train transit. In the

selected site, three different types of fast failure may

affect the railway:

- Sinkholes: fast vertical collapses generating

holes of variable diameter below the railway causing

its interruption or flexion. The expected precursor

signals are micro- or nano-seismic emissions due to

micro-cracks that anticipate the collapse phase and

preliminary deformations of the ground surface that

may affect the railway track.

- Rock falls: these fast landslides generally occur

on cliff slopes (natural or man-made) due to rock

mass jointing. They cause the accumulation of rocky

blocks of variable size that may occupy and/or

damage the railway infrastructure. In this case, the

expected precursor signals are micro- or nano-

seismic emissions due to micro-cracks that

anticipate the collapse phase but rarely are

preliminary and detectable deformations observed.

- Debris flows: this phenomenon consists in the

fast triggering of a flow-like event due to a mixture

of water, air and debris originally accumulated along

narrow and deep creeks elongated on high-dip

slopes. The velocity and volumes of the flowing

debris can produce severe damage to infrastructure

due to impact energy as well as debris accumulation.

The predisposing conditions for debris-flow

triggering are the presence of debris fill along the

creek when intense rainfall occurs.

The most suitable site for experimenting a

monitoring system meeting the above requirements,

i.e. detecting the occurrence of fast failures, should

make it possible to:

• detect deformations or abnormal objects

along the railway in real time;

• provide multiple points of observation;

• measure the observed objects;

• transmit the detection dataset in real time;

• ensure the interaction between the

wireless-connected sensors.

As a preliminary action, the WSN should be

calibrated to better identify any “regular” objects

statically or dynamically located on the railway, e.g.

trains in transit, animals running or existing

vegetation. The sensors should also be capable of

filtering spurious noise generated by system

instabilities, e.g. vibrations induced by regular

railway traffic.

2.1 Methodology

The pilot sites were selected after field surveys

aimed at assessing their suitability for the planned

experiments in terms of both expected failure

occurrences and adequacy for the installation and

calibration of the WSN. For each of the main events

(sinkholes; debris flows; falls) in all the selected

sites, the monitoring and early warning network was

designed with specific sensors.

On the selected sites, the activity was carried out

in the following steps:

- preliminary geological and geomorphological field

survey on longer railway tracks and surrounding

areas, in order to check the existence of slopes

prone to fast-landslide events;

- selection of shorter railway tracks to be monitored;

- planning of the WSN installation;

- setting of sensor resolution;

- evaluation of alarm thresholds.

The monitoring and early warning system

designed for the sinkhole consisted of two sensors,

detecting any railway deformation with high

accuracy and in real time. The two wireless sensors

covered the same portion of land from different

points of view. This type of installation allows

individual cameras to better define the objects and to

communicate useful information to better identify

the phenomenon on a wider scale. For the

monitoring of rock falls and debris flows, the system

was configured with one or multiple wireless sensors

with multiple points of observation based on both

the “background target” (i.e. the static regular scene)

and the object to be detected. The number of sensors

was also dependent on the monitoring accuracy of

the expected railway deformations, as well as of the

expected abnormal object along the railway track.

For all the investigated phenomena, the definition of

the WSN parameters was fundamental to perceive

and then process only the differences between a step

of the background and the next disturbance, so as to

minimise false alarms and optimise data collection

and transmission. This optimisation procedure

contributes to providing more efficiency to the

management process.

2.2 Materials

This section of the paper presents the design and

development of a sensor prototype based on the

IMTA-52015-5thInternationalWorkshoponImageMining.TheoryandApplications

previously defined WSN concept. This prototype

was particularly suited for the investigated scenarios.

In particular, the prototype was a sensor node having

enough computational power to accomplish the

computer vision task envisaged for the railway

monitoring scenarios, as described in the previous

section. In the design of the prototype, an important

requirement was the use of low-cost technologies.

The node uses sensors and electronic components of

low cost so that, once engineered, the device can be

manufactured at low cost in large quantities. In the

design and planning of the architecture, an important

consideration was the ease of installation of the

device: the protective shield used for the sensor

nodes was compact but capable of accommodating

all components of the device. A WSN having its

nodes so engineered was called Smart Camera

Network (SCN).

Going into detail, the single sensor node had a

main board that managed both the vision tasks and

the networking tasks thanks to an integrated wireless

communication module (RF Transceiver).

Other components of the sensor node were the

power supply system that controlled charging and

permitted to choose optimal energy-saving policies.

The power supply system included the battery pack

and an optional module for harvesting energy, e.g.

photovoltaic panels (Figure 1).

Figure 1: Architecture of the sensor node.

For building the vision board, an embedded

Linux architecture was selected in the design stage,

so as to provide enough computational power and

ease of programming. A set of ready-made Linux-

based prototyping boards was evaluated in terms of

computing power, flexibility/expandability,

price/performance ratio and technical support. They

were all found to have common disadvantages: high

power consumption and electronic components not

suitable for the tasks of a smart camera node.

It was thus decided to design and build a custom-

made vision component, by designing, printing and

producing a new PCB. The new PCB (see Figure 2)

was conceived to have maximum flexibility of use

while maximising the performance/consumption

ratio. A good compromise was achieved by using a

Freescale CPU based on the ARM architecture, with

support for MMU-like operating systems GNU/

Linux.

This architecture had the advantage of

integrating a Power Management Unit (PMU), in

addition to numerous peripheral interfaces, thus

minimising the complexity of the board. In addition,

the CPU package of type TQFP128 helped us to

minimise the layout complexity, since it was not

necessary to use multilayer PCB technologies for

routing. Thus, the board could be printed also in a

small number of copies. This choice had the further

benefit of reducing development costs: the CPU only

needed an external SDRAM, a 24MHz quartz

oscillator and an inductance for the PMU.

It had an average consumption, measured at the

highest speed (454MHz), of less than 500mW.

The system included an on-board step-down

voltage regulator, type LM2576, featuring high

efficiency to ensure a range of voltages from 6 to

25V, making it ideal for battery-powered systems, in

particular for power supply by lithium batteries (7.2

V packs) and lead acid batteries (6V, 12V, 24V

packs).

The board had several communication interfaces

including an RS232 serial port for communication

with the networking board, SPI, I2C and USB.

For radio communication, a transceiver

compliant with IEEE 802.15.4 was integrated, in

line with modern approaches to the Internet of

Things applications. Appropriate glue was used to

integrate the transceiver into the IPv6 stack, also

containing the 6LoWPAN header compression and

adaptation layer for IEEE 802.15.4 links. Therefore,

the operating system was well capable of supporting

ETSI M2M communications over the SCN.

Figure 2: Design of the PCB and main features.

For integration of a camera sensor into the vision

board, some specific requirements were defined in

the design stage: ease of connection to the board and

of management through it, and minimum

performance under difficult visibility conditions, i.e.

night vision. Thus, the minimal constraints were:

compliance with USB Video Class devices (UVCs)

ExperimentinganEmbedded-sensorNetworkforEarlyWarningofNaturalRisksDuetoFastFailuresalongRailways

and possibility to remove the IR filter or capability

of Near-IR data acquisition. Moreover, the selection

of a low-cost device was an implicit requirement

considered for the whole sensor node prototype.

The previously described boards and camera

were housed into an IP66 shield. Another important

component of the node was the power supply and

the energy harvesting system that controlled

charging and permitted to choose optimal energy-

savings policies. The power supply system included

the lead (Pb) acid battery pack and the optional

module for harvesting energy through a photovoltaic

panel.

Figure 3: General setup of the monitoring node.

Figure 3 shows the general setup of a single node

with the electrical connections of the related

components.

2.3 Image Analysis

Sample applications based on computer vision for

monitoring railways via the SCN are described

below. They concern the detection and real-time

alert of dangers strictly related to the flow of trains

on the railway, and are based on a lightweight

computer vision pipeline different from the one used

in standard architectures.

More precisely, the analysis of the railway scene

status and the estimation of the level of service were

usually carried out by collecting data about the

railroad track in terms of events occurring, their type

and extension in space and time. Conventional

pipelines start with i) background subtraction and

move forward to ii) object detection, iii) object

classification, iv) object tracking and v) final data

extraction. On the SCN, instead, it is convenient to

adopt a lightweight approach; in particular,

processed data reside only in the Region of Interest

(RoI), where the presence of an obstructing object

needs to be detected. On the basis of these

detections, then, flow information is derived without

making explicit use of classical tracking algorithms.

More in detail, background subtraction is

performed only on convex quadrangular RoIs. Such

shape is sufficient for modelling physical rectangles

under perspective skew. In this way, when low

vision angles are available, it is possible to manage a

skewed scene even without performing direct image

rectification, which can be computationally intensive

on an embedded sensor. The quadrangular RoI can

be used to model lines on the image (i.e. a 1 pixel

thick line) as well.

On such RoI, lightweight detection methods are

used to classify a pixel as changed (in which case it

is assigned to the foreground) or unchanged (in

which case it is deemed to belong to the

background). Such decision is made by modelling

the background. Several approaches are feasible.

The simplest one is represented by straightforward

frame differencing. Under this approach, the frame

before the one being processed is taken as

background. A pixel is considered to be changed if

the frame difference value is higher than a given

threshold. Frame differencing is one of the fastest

methods but has some limitations; for instance, a

pixel is considered to be changed twice: first, when

an object enters into and, second, when it exits from

the pixel area. In addition, the RoI is placed in an

area of train transit; thus, when an event of transit

occurs, and if the object (i.e. the train occluding the

RoI) is homogeneous and imaged in more than one

frame, it might be not detected in the frames after

the first. Another approach is based on the static

background. Under this approach, the background is

taken as a fixed image without objects, possibly

normalised to factor illumination changes. Due to

weather, shadows and light changes, the background

should be updated to yield meaningful results in

outdoor environments. However, strategies for

background update might be complex; indeed, it

should be guaranteed that the scene is without

objects passing through when updating. To

overcome these issues, algorithms featuring adaptive

backgrounds are used. Indeed this class of

algorithms is the most robust for use in uncontrolled

outdoor scenes. The background is constantly

updated by merging the old background model with

the new observed image. There are several ways of

obtaining adaptation, with different levels of

IMTA-52015-5thInternationalWorkshoponImageMining.TheoryandApplications

computational complexity. The simplest is to use an

average image. In this method, the background is

modelled as the average of the frames in a time

window. Online computation of the average is

performed. Then a pixel is considered to be changed

if it exceeds a given threshold of the corresponding

pixel in the average image. The threshold is uniform

on all the pixels. Instead of modelling just the

average, it is possible to include the standard

deviation of pixel intensities, thus using a statistical

model of the background as a single Gaussian

distribution. In this case, both the average and

standard deviation images are computed with an

online method on the basis of the frames already

observed. In this way, instead of using a uniform

threshold on the different image, a constant

threshold is used on the probability that the observed

pixel is a sample drawn from the background

distribution, which is modelled pixel by pixel as a

Gaussian. Gaussian Mixture Models (GMMs) are a

generalisation of the previous method. Instead of

modelling each pixel in the background image as a

Gaussian, a mixture of Gaussians is used. The

number k of Gaussians in the mixture is a fixed

parameter of the algorithm. When one of the

Gaussians has a marginal contribution to the overall

probability density function, it is disregarded and a

new Gaussian is instantiated. GMMs are known to

be capable of modelling changing backgrounds even

in cases where there are phenomena such as

trembling shadows and tree foliage (Stauffer and

Grimson, 1999). Indeed, in those cases, pixels

clearly exhibit a multimodal distribution. However,

GMMs are computationally more intensive than a

single Gaussian. Codebooks (Kim et al., 2004) are

another adaptive background modelling technique

presenting computational advantages for real-time

background modelling with respect to GMMs. In

this method, sample background values at each pixel

are quantified in codebooks, which represent a

compressed form of background model for a long

image sequence. This makes it possible to capture

even complex structural background variations (e.g.

due to shadows and trembling foliage) over a long

period of time under limited memory.

Several ad-hoc procedures can be envisaged

starting with the methods just described. In

particular, one important issue concerns the policy

by which the background is updated or not. In

particular, if a pixel is labelled as foreground in

some frame, we might want this pixel not to

contribute to updating the background or to

contribute thereto to a lesser extent. Similarly, if we

are dealing with a RoI, we might want to fully

update the background only if no change has been

detected in the RoI; if a change has been detected

instead, we may decide not to update any pixel in the

background.

3 RESULTS

This section reports the preliminary results for the

identified case study site, where the experimental

activity was performed in order to monitor the

railway and derive a real-time report of obstacles

endangering train transit. The main objective was to

define the scenarios and set-up for the above-

mentioned three different types of fast-failure events

that might locally involve the railway.

3.1.1 Case Study

The selected pilot site for the first test was located

close to Terni, Central Italy, along a secondary line

of the Italian railway network. The site of Terni is

subject to rock falls and is characterized by a narrow

man-made trench cut in intensely jointed limestones.

From the trench walls, which are partially bounded

by wire mesh, stones of a size from few centimetres

to about one meter may fall onto the railway. In this

site, several tests were carried out for analysing and

verifying the installation's positions and the data

acquisition methods to monitor the railway tracks.

The tests were also aimed at verifying the SCN

suitability for field acquisitions in case of real

running trains as well as in case of artificially caused

rock falls.

Some video sequences were recorded including

the following scenes:

A. sideway scanning of tracks to catch events;

B. semi-perpendicular scanning to catch the rails to

detect any changes;

C. railway scanning without trains, representing the

"background scene";

D. scanning steps with trains in both directions;

E. scanning during vibration generated by train

transit;

F. scanning of simulated anomalous transits as well

as of artificially caused falls of "objects" on the

railway.

It was possible to record and quantify image

artefacts induced by vibrations and air movement.

The collected data enabled to estimate the possible

consequences on the image analysis algorithms and,

therefore, to improve a software solution for

reducing disturbances.

ExperimentinganEmbedded-sensorNetworkforEarlyWarningofNaturalRisksDuetoFastFailuresalongRailways

If compared with the static background image, the

recorded noise is very low and can be directly

managed by the acquisition software.

However, the reduction of noise via specifically-

implemented tools should be encouraged in order to

further optimise the performance of the device.

Figure 4: Example of event detected in a RoI over the

tracks (e.g. falling debris).

A final test was performed by recording videos

during train stops along the railway, a type of

disturbance due to the specific event expected to

occur along the railway track. This final test

demonstrated the excellent performance of the

implemented algorithms: once the directional

parameters referred to the monitored scene have

been defined, the software only processes the

camera records that express differences between the

background scenario and the changed one,

minimizing false alarms and optimising data

collection and transmission.

More in detail, scenarios were defined for

training and testing the implemented algorithm to be

robust and satisfy the specific requirements defined

for the case study application. In particular, the

robustness of the image analysis algorithm was

tested under scenarios such as the management of

false alarms due to train transit, coherence in case of

vibrations caused by train transit, adequacy of

detection of debris passing by the rails but not

impeding subsequent train transit, capability of

detecting early signs of sinkholes.

In Figure 4, an example of a detected event

(falling debris) is shown (right), in comparison with

the normal situation (left). The amount of the

detected change in the scene is above the threshold

fixed as definition of an endangering situation.

Moreover, Figure 5 shows the amount of

maximum displacement (in pixels) during the transit

of a train causing vibration to the installation; such

amount was quantified to be less than 1% with

respect to the image size, thus a very limited and

manageable difference.

Figure 5: Details (zoom 400%) of the maximum

displacement between two frames during vibration due to

train transit.

With regard to the capability of detecting early

signs of sinkholes, a different scenario was set up

with the SCN device placed perpendicularly over the

train tracks. With this set-up, it was possible to

define the limit of distance from the railway and the

limit of movement in the parallelism of the rails. The

set-up is shown in Figure 6, with a highlighted area

indicating the distance representing the limit of the

detection capability of the sensor (i.e. around 150

pixel size) for determining discrepancies in the

parallelism of the rails.

4 CONCLUSIONS

In order to monitor railway tracks in real time and

detect fast failures occurring and threatening train

transit, a Smart Camera Network was put in place.

Three different types of fast-failure events, involving

the railway, were identified with the goal of

developing algorithms capable of spotting and

localising them. Furthermore, scenarios were

defined to experiment, in the case study site, the

capability of the SCN to successfully monitor the

various events. While the simulation of some events

was feasible (e.g. debris falling onto the railway),

the simulation of other events was not feasible due

to risks for safety (e.g. early signs of the sinkhole).

Therefore, the latter events were only simulated

IMTA-52015-5thInternationalWorkshoponImageMining.TheoryandApplications

through software changes, but the results were good

and promising in both cases.

Figure 6: Sinkhole detection set-up highlighting the

detection area.

ACKNOWLEDGEMENTS

The authors are grateful to Eng. Giovanni Costa and

Eng. Paola Firmi for allowing the dissemination of

some monitoring data applied in some experimental

tracks of the Italian Rail Network in the frame of the

Agreement between CERI-Sapienza and Italian Rail

Network.

REFERENCES

Amitrano, D., Grasso, J.R. and Senfaute, G.: Seismic

precursory patterns before a cliff collapse and critical

point phenomena. Geophysical Research Letters, 32,

L08314, doi:10.1029/2004GL022270, 2005.

Antonello G, Casagli N, Farina P, Leva D, Nico G, Siebar

AJ, Tarchi D (2004) Ground-based SAR

interferometry for monitoring mass movements.

Landslides 1:21-28.

Bigarré, P., Klein, E, Gueniffey, Y., and Verdel, T.: Cloud

monitoring: an innovative approach for the prevention

of landslide hazards. The Second World Landslide

Forum Rome, 2011. Abstracts Book WLF2, L16, 475

(free download from the web-site:

www.wlf2.org/home/proceedings/WLF2_Abstract_Bo

ok_2011.pdf), 2011.

Gaffet, S., Guglielmi, Y., Cappa, F., Pambrun, C.,

Monfret, T. and Amitrano, D.: Use of the

simultaneous seismic, GPS and meteorological

monitoring for the characterization of a large unstable

mountain slope in the southern French Alps. Geophys.

J. Int., 182, 1395–1410, 2010.

Kim K., Chalidabhongse T., Harwood D. and Davis L.,

2004. Background modeling and subtraction by

codebook construction, IEEE International Conference

on Image Processing (ICIP), 2004, pp. 2-5.

Lai, X.P., Cai, M.F. and Xie, M.W.: In situ monitoring and

analysis of rock mass behavior prior to collapse of the

main transport roadway in Linglong Gold Mine,

China. International Journal of Rock Mechanics &

Mining Sciences 43, 640–646, 2006.

Lenti, L., Martino, S., Paciello, A., Prestininzi, A. and

Rivellino S.: Microseismicity within a karstified rock

mass due to cracks and collapses triggered by

earthquakes and gravitational deformations. Natural

Hazards, 64, 359–379, 2012.

Martino S., Mazzanti P (2014) Integrating geomechanical

surveys and remote sensing for sea cliff slope stability

analysis: the Mt. Pucci case study (Italy) Nat. Hazards

Earth Syst. Sci., 14, 831–848.

Stauffer C. and Grimson W.E., 1999. Adaptive

background mixture models for real-time tracking,

Proceedings of the Conference on Computer Vision

and Pattern Recognition, Fort Collins, CO, USA:

1999, pp. 2: 246-252.

ExperimentinganEmbedded-sensorNetworkforEarlyWarningofNaturalRisksDuetoFastFailuresalongRailways