In-Vehicle IoT Platform Enabling the Virtual Sensor Concept:

A Pothole Detection Use-case for Cooperative Safety

Ilaria Bosi, Enrico Ferrera, Daniele Brevi and Claudio Pastrone

LINKS Foundation – Leading Innovation & Knowledge for Society, Turin, Italy

Keywords: Pothole Detection, IoT Platform, Virtual Sensor, Automotive, Intelligent Transportation System (ITS),

Autonomous Vehicle, Smartphone, 6LoWPAN Sensor, Inertial Measurement Unit, Accelerometer.

Abstract: Nowadays the number of on-board sensors increases continuously due to their benefits in many different

areas, such us driving efficiency, maintenance, autonomous driving, etc. Usually the vehicle itself and its

users are those which take direct advantage from these benefits. By leveraging Internet-of-Things (IoT)

technologies, it is possible to abstract data and functionalities provided by on-board sensors and actuators

exposing relevant services outside the vehicle to external cloud-based applications and other vehicles. With

these technologies the vehicle is thus transformed in an IoT object which can be part of external IoT

platforms. This work focuses on the design and implementation of an in-vehicle IoT platform which

exposes internal functionalities as IoT services enabling also the concept of “Virtual Sensor”, which

leverages sensor fusion techniques to provide enhanced services combining raw data coming from on-board

devices. This IoT platform solution is validated through a use case in which virtual real-time pothole

detection sensor is implemented to evaluate the road surface conditions. In such use-case, multi-source

sensing information - coming from 6LoWPAN sensors as well as Smartphones and Inertial Measurement

Units - is fused, enabling IoT applications such as cooperative safety and early road maintenance.

1 INTRODUCTION

Today’s autonomous driving applications are relying

on autonomous vehicle systems in which the needed

information is gathered, processed and analysed on

the vehicles themselves. Automated vehicles largely

rely on on-board sensors (LiDAR, radar, cameras,

sensors…) or interconnects surrounding sensors

(cameras, traffic light radars, road sensors…) to

detect the environment and make autonomous cars a

full entity in the IoT eco-system. This paper is based

on the use cases carried out during the European

project AUTOPILOT (Ertico, 2017), that focuses on

utilizing the IoT potential for automated driving and

making data from autonomous cars available to the

Internet-of-Things. In this way the Automated

vehicle systems consist of inputs from a large variety

of sensors, data signal condition and decision making

by central or edge processing units and outputs to a

large variety of actuators. The overall objective is to

bring together relevant knowledge and technology

from the automotive and the IoT value chains in

order to develop IoT-architectures and platforms

which will bring Automated Driving towards a new

dimension. In order to demonstrate how additional

IoT sensors placed in the AUTOPILOT prototype

can enhance the functions of the car itself, the

vehicle can be used for example as an IoT sensor for

detecting the surface condition for both highway and

urban scenarios.

Smart sensors and actuators in the vehicles, roads

and traffic control infrastructures collect a variety of

information to serve enhanced automated driving,

while considering the timing, safety and security

constraints (Maag et al., 2012).

Road surface anomalies, such as potholes, speed

bumps, railroad crossing and joints, can determine

some problems for vehicles and can affect road users

safety. Road quality assessment plays a key role in

infrastructure management and it is useful to an

adequate allocation of road maintenance operations

(Mukherjee and Majhi, 2016).

A pothole refers to a shallow pit on a road’s

surface, caused by activities like erosion, weather,

traffic and some other factors. Detecting and hence

avoiding potholes may reduce the fuel consumption,

wear-tear and maintenance cost of a vehicle. With

the availability of information regarding the road

232

Bosi, I., Ferrera, E., Brevi, D. and Pastrone, C.

In-Vehicle IoT Platform Enabling the Virtual Sensor Concept: A Pothole Detection Use-case for Cooperative Safety.

DOI: 10.5220/0007690602320240

In Proceedings of the 4th International Conference on Internet of Things, Big Data and Security (IoTBDS 2019), pages 232-240

ISBN: 978-989-758-369-8

conditions, road users can be cautious about or avoid

the bad spots (Langle and Dantu, 2009), in addition,

Autonomous vehicles can make the right manoeuvres

to avoid potholes or other dangerous situations. It is

desirable to have a mechanism for detecting the

condition of roads and get them repaired as soon as

possible. As a result, working on monitoring road

conditions has gained significant attention in recent

time.

Related with the concept of Fog Computing,

since there is no sensor on the vehicle that

"physically" detects the roadway potholes, it is

thought to use a combination of sensors that can

already be integrated into the Original Equipment

Manufacturer (OEM) dispositive (e.g.

accelerometers, gyroscopes, etc.) or use sensors from

external devices to be placed on-board (e.g.

smartphones, cameras, external accelerometers, etc.)

to recover the same type of data (acceleration,

orientation, etc.) (De Silva et al., 2013).

This study proposes a pothole detection method

based on the in-vehicle platform that will act as a

“virtual sensor” for vibration. The information can be

taken by 1) a 6LoWPAN sensor, 2) a

smartphone/tablet or 3) an Inertial Measurement Unit

(IMU). The “virtual sensor” can work with only one

source of information or combines different sources.

The accelerometer data is normalized and is adopted

in the pothole detection algorithm to obtain the

pothole information (interpolation with global

positioning system (GPS) data). When a pothole is

detected, a message is sent to the OneM2M cloud

platform (Scarrone, 2016) where it is available for all

the other vehicles and services. The proposed real-

time pothole detection method based on mobile

sensing includes three main steps: (1) accelerometer

data normalization, (2) pothole detection algorithms,

and (3) pothole location determination.

Till now, smartphones used for road condition

monitoring, is limited to recording of accelerations,

processing them to discern potholes and monitoring

the overall condition of road surfaces. Therefore, the

data must be pre-processed before it can be used.

This can be done for example by using a passband

filter. It removes low and high frequencies from the

measured data. This makes the data much cleaner

and easier to process. Data can also be divided into

small segments and normalized to some specific

scale to make the feature extraction and classification

easier (Aksamit and Szmechta, 2011).

Our main goals are to classify road surfaces and

further evaluate road conditions. Two basic

categories for describing the road surface has been

defined: smooth roads and rough roads (roads

containing surface anomalies). A smooth road offers

a high-quality driving surface to vehicles traversing

over it, while a rough road is its complement.

The study begins with a general overview of the

different already implemented methods for the

detection of the road bumps, followed by the

presentation of IoT platform on board vehicle

implemented in AUTOPILOT project. Then the core

of this work related to pothole detection is described

and the developed algorithms with the procedures

adopted in the tests are presented. Finally, the

discussion of the first results obtained and the

conclusions of the entire work are proposed.

2 STATE OF THE ART

With the increasing popularity of new road

technologies and smartphones among people,

researchers are showing interest in building smart

IoT solutions using smartphones (because of the

embedded sensors, like a GPS, accelerometer,

gyroscope or magnetometer) or other image

recognition methods in order to monitoring overall

condition of road surfaces and improve the driving

safety.

An image processing approach was proposed by

Danti et al., (2012)

where the potholes are first

photographed or recorded using camera on a car. The

algorithm that process these images of the road,

represents the pothole as a distinct black colour.

Also Mertz (Mertz, 2011) uses light sensor and a

camera mounted on vehicles (equipped with GPS) to

detect road damages. The data is collected from

many vehicles, aggregated and analysed at a central

location and the assessment results are displayed

interactively to facilitate road maintenance

operations.

 A similar approach was provided by

(Balakuntala and Venkatesh, 2013): the system

comprises a laser sensor and pressure sensors in

shock absorbers to detect and quantify the intensity

of a pothole, a centralized server which maintains a

database of locations of all the potholes which can be

accessed by another unit inside the vehicle.



In a recent study (Jothi et al., 2010), the potholes

and the hump locations are detected using GPS,

GSM and Ultrasonic sensor, to alert the driver,

reduce the vehicle speed decreasing the accident

impact and consequently the vehicle damage. One

database server collects the information about the

potholes and humps and the information are sent to

government authorities through the TCP protocol.



Regarding the use of smartphones, Wang et al.,

(2015) proposes a pothole detection method based on

In-Vehicle IoT Platform Enabling the Virtual Sensor Concept: A Pothole Detection Use-case for Cooperative Safety

233

the mobile sensing (G-sensors and GPS) and shares

the pothole information with road users and

government. The accelerometer data is normalized

by Euler angle computation and is adopted in the

pothole detection algorithm to obtain the pothole

information.

In a different study (Astarita et al., 2012), the

accelerometer of five different devices (all placed in

a test vehicle in three different placement conditions)

is used for detecting surface conditions, using three

different filters to analyse acceleration signals.

Moreover, verification of the rate of false detections

and undetected road anomalies is planned, using

georeferenced photos that allow the correct

localization on the map and the assessment of the

correspondence between the elements, detected with

the accelerometer, and real road conditions. Also

Kalra et al., (2014) used data from smartphone

accelerometer sensor: various thresholds are set and

used for distinguishing and classifying various

driving events and road anomalies.

The accelerometer sensor and navigation system

interfaced microcontroller based embedded device

mounted inside the vehicle works as a pothole

inspector which updates the database on instant basis

for every single experienced pothole (Aniket and

Vivek, 2016).

A recent study (Akinwande et al., 2015) proposed

a real-time pothole detection and traffic monitoring

system and has been able to harness smartphone

sensors to solve a global challenge, applying

Machine Learning to a real world problem and

developing a scalable and reliable system driven by

the power of crowdsourcing.

Jakob Eriksson (Eriksson et al., 2008) have

proposed The Pothole Patrol: this system uses the

inherent mobility of the participating vehicles,

opportunistically gathering data from vibration and

GPS sensors (3-axis accelerometers and GPS sensors

mounted on the dashboard of cars) and processing

the data to assess road surface conditions. It not only

identifies potholes but also differentiate potholes

from other road anomalies.

Another method (Bhoraskar et al., 2012) uses

Smartphone sensors for traffic state monitoring and

detection of bumps. This system reorients the phone

in two steps using accelerometer and magnetometer.

The accelerometer sensor data is classified using k-

means clustering algorithm into two classes which is

labelled manually as either smooth or bumpy (for

bump detection) and brake or not (for braking

detection). This labelled data is used to train Support

Vector Machine (SVM) for classification of data

points during test phase for vehicle state prediction.

Also Bhatt et al., (2017) assess roads using

gyroscope and accelerometer sensors in the phone,

training SVM models to classify road conditions with

93% accuracy and potholes with 92%. Then, the

classification results are used to create data-rich

maps that illustrate road conditions across the city.



Nericell (Mohan et al., 2008) focuses specifically

on an array of sensors; accelerometer, microphone,

GSM radio, and/or GPS sensors are used to detect

potholes, bumps, braking, and honking. They also

proposed a method to virtually reorient a disoriented

accelerometer and to use multiple sensors in tandem,

with one triggering the other, to save energy.



Mednis et al., (2011) describes accelerometer

data based on different Android smartphones, for

pothole detection algorithms deployed on devices

with limited hardware/software resources. They have

proposed four algorithms for detection of potholes in

real-time and for off-line post-processing of data.

3 IN-VEHICLE IOT PLATFORM

Thanks to AUTOPILOT’s aim, the IoT eco-system

will involve vehicles, road infrastructure and

surrounding objects in the IoT, with a particular

attention to safety critical aspects of automated

driving.

Extending the work described in (Ferrera et al.,

2017), the IoT in-vehicle platform of the

AUTOPILOT’s Italian Pilot Site is a modular

software including Application Container and

Communication System, which are deployed on the

On Board Unit (OBU) inside the vehicle. The

“Runtime Environment” part of the OBU is

composed by several software modules, as showed in

Figure 1.

Figure 1: Italian IoT in vehicle platform AUTOPILOT

project.

The functionality of Remote Management is

implemented by a software (OSGi remote

management tool) (Ferrera et al., 2017), that allows

IoTBDS 2019 - 4th International Conference on Internet of Things, Big Data and Security

234

to configure the platform by adding/removing

bundles, introducing the idea of remote monitoring

and control of external application based on OSGi

platform. Through the Event Admin Internal Bus the

connectors have the same communication interface

to the bundles which they interfaced in the

Application Container.

The Application Container also encase the

functionality of Data Management, with the modules

implementing a Local Dynamic Maps (LDM) and the

Pothole Detector. LDM is a database that achieves

integrated management of static digital map

information and dynamic object information such as

the one coming from vehicles (functional

requirement of Context Awareness). The bundle of

Pothole detector represents the implementation of

the pothole detection algorithm. It is based on data

fusion techniques in order to implement the concept

of "virtual sensors". This module collects data from

multiple sensors on the vehicle (IoT in-vehicle

components or OEM in-vehicle components),

processes the various data and sends the results of

this elaboration to the cloud OneM2M platform or

Road Side Units or other vehicles (via

communication system).

Regarding the IoT device adaptation, it is planned

to support different IoT communication protocols

with the devices: the IoT connector are used to

integrate with 6LoWPAN data coming from

additional IoT devices (i.e. Inertial sensors), that are

used by edge applications on the OBU

(CoAP/6LoWPAN connector) and also to integrate

with MQTT protocol data coming from additional

IoT devices (i.e. smartphone), that are used by edge

applications on the OBU (MQTT connector).

The “Communication System” part of the OBU,

manages different high-level capabilities. The

module CANBus Interface reads data coming from

the CAN Bus and decodes important data coming

from the in-vehicle sensors that are sent directly to

the OneM2M platform or used by edge applications

on the OBU.

4 “VIRTUAL SENSORS”

The innovative and relevant part of this study is to

demonstrate how additional IoT sensors placed in the

car prototype, in conjunction with a centralized

(OneM2M) platform for data collection, can enhance

the functions of the car itself: the vehicle can be used

for example as an IoT sensor for detecting the

surface condition for both highway and urban

scenarios.

Object virtualization aids to address the issues of

heterogeneity, interoperability, multitenancy,

scalability, counter-productivity, mobility and

protocol inconsistency that are commonly existing in

IoT. Related with the concept of Fog Computing,

since there is no sensor on the vehicle that

"physically" detects the roadway potholes, it is

thought to use a combination of sensors that can

already be integrated into the OEM dispositive (e.g.

accelerometers, gyroscopes, etc.) or use sensors from

external devices to be placed on-board (e.g.

smartphones, cameras, external accelerometers, etc.)

to recover the same type of data (acceleration,

orientation, etc.).

This is the concept of “Virtual Sensor”, that

differentiates the study conducted by previous ones.

In this way, data from different devices are fused

together and processed: from this sensor fusion

outputs, the data can be used to detect the road holes.

The result of this fusion is therefore a “pothole

detector” and these elaborations are sent to the cloud

OneM2M platform or RSU or other vehicles (via

communication system).

The data of the raw signal accelerations on the 3

axes will be collected and analysed using an inertial

6LoWPAN, a Nokia 6 smartphone and the

accelerometer sensor of an inertial measurement unit

(IMU). Figure 2 shows these devices.

Figure 2: a) Nokia smartphone, b) 6LoWPAN Sensor, c)

Inertial Measurement Unit.

The virtual sensor can use one or more

acceleration sensors combining the upcoming data in

a smart way (Figure 3). The accelerometer measures

changes in velocity of the sensor in three dimensions:

the linear sensing provides the sensor information

about its motion and thus taps, or shakes can be

detected.

Figure 3: "Virtual Sensor" concept.

The flow of raw data is (Figure 4):

In-Vehicle IoT Platform Enabling the Virtual Sensor Concept: A Pothole Detection Use-case for Cooperative Safety

235

- A wireless vibration sensor is deployed on the

connected vehicle, which notifies to the OBU via

MQTT/6LowPAN protocols the occurrences of

vibrational shock above a certain level due to a

pothole presence on the road;

- The OBU combines this information with other

data coming from CAN bus (speed, odometer, etc.)

and GPS and sends this data to the OneM2M IoT

Cloud platform, by using CoAP/MQTT and/or HTTP

as application protocols;

- The AD car applications or upcoming AD vehicle,

consumes the information and can arrange its speed

accordingly (Crowdsourced data can be retrieved via

a OneM2M subscription and used by other vehicles.)

Figure 4: Pothole detector algorithm.

5 EXPERIMENTAL SETUP

Road anomalies, such as potholes, sunk-in manhole

covers, or missing pavement, cause the abnormal

vibration of vehicles.

The accelerometer records all vehicle vibrations

including vibrations from the engine and the gear

box and all swings made by passengers: the linear

sensing in the direction of motion of the vehicle to

identify the braking, and in the direction

perpendicular to the direction of motion of the

vehicle to identify bumps and potholes. Similarly,

orientation can be determined by the sensor’s

sensitivity to the local gravitational field.

A continuous stream of data related to the linear

acceleration of the vehicle on three principal axes,

will provide additional measurements related to

distance travelled by the autonomous vehicle,

providing data related to the velocity and the extent

of acceleration towards obstructions (Figure 5).

Figure 5: a) Gravity vector and heading, pitch and roll

about axes b) Downward jerk sensed by accelerometer

which occurred due to potholes over the road.

The data from the accelerometer is

conventionally reported in units of g (1g= 9.81 m/s

In the initial condition and calibration, the

accelerometer reports a value of 1g along the z-axis

and 0 along the x and y axes when lying at rest face

upon a flat table. The gravity vector thus reported is

used as a reference for all other linear motion

sensing. As already mentioned in this case of study,

the data of the raw signal accelerations on the 3 axes

will be collected and analysed using a Nokia 6

smartphone, an inertial 6LoWPAN sensor and the

accelerometer sensor of an inertial measurement unit

(IMU). The virtual sensor can use one or more

acceleration sensors combining the upcoming data in

a smart way. To better understand the different types

of sensors used during the tests, will be made a

summary regarding the characteristics of the devices

used to collect the raw acceleration values.

5.1 Smartphone Accelerometer

To use the accelerometer reading for detecting

various events, it is possible to virtually reorient the

axes of the smartphone to align along the axes of the

vehicle. Readings from the reoriented axes can be

used to detect events. Leveraging an accelerometer

as a vibration sensor, the characterization of potholes

and roads can be done using the readings of the

accelerometer.

It was implemented an App in Android Studio to

collect the raw accelerometer data in the three

directions and send the data to a MQTT broker

(Figure 6).

Figure 6: Android App raw accelerometer.

The using of the mobile device based on mobile

sensing techniques to detect potholes, is suitable and

convenient: all the motion sensors return multi-

dimensional arrays of sensor values for each

SensorEvent. The linear acceleration sensor provides

with a three-dimensional vector representing

acceleration along each device axis, excluding

gravity. [linear acceleration= acceleration -

IoTBDS 2019 - 4th International Conference on Internet of Things, Big Data and Security

236

acceleration due to gravity]. The simplest way to

remove the offset of a linear acceleration sensor, is to

build a calibration step into the application, in order

to iterate the alignment of the smartphone

accelerometer’s coordinate system and the vehicle’s

coordinate system (Android, 2018); (Android, 2018).

Smartphone accelerometers use the standard

sensor coordinate system. In practice, this means that

the following conditions apply when a device is lying

flat on a table in its natural orientation (see Figure 7):

 If the device is pushed on the left side (so it

moves to the right), AccX value is positive.

 If the device is pushed on the bottom (it moves

away from you), AccY value is positive.

 If the device is pushed toward the sky with an

acceleration of A m/s

, AccZ is equal to A+9.81,

which corresponds to the acceleration of the

device (+A m/s

) minus the force of gravity (-

9.81 m/s

 The stationary device will have an acceleration

value of +9.81, which corresponds to the

acceleration of the device (0 m/s

minus the force

of gravity, which is -9.81 m/s

Figure 7: Coordinate system (relative to a device) that's

used by the Sensor API.

5.2 6LoWPAN Sensor

The second sensor used to test the pothole detection

setup, is an inertial unit that supply raw

accelerometer data. SensOne leverages industry

standard IEEE802.15.4 RF protocol for robust and

power aware communication interfaces and USB2.0

connectivity (NGS, 2018). The SensOne has been

designed for battery powered Internet of Things

applications and natively supports state-of-the-art

Internet addressing protocols (e.g. 6LoWPAN), to

interoperate seamlessly with other devices.

The triple-axis MEMS accelerometer (12 bit-

resolution) in MPU-6050 assembled in the SensOne,

includes a wide range of features such as a

programmable full-scale range (±2g, ±4g, ±8g and

±16g), orientation detection, signalling and tap

detection. The values of accelerations sent are in

thousandths of g (9.81/1000 m/s

5.3 Inertial Measurement Unit

Accelerometer Sensor

The third type of sensor that can be used to acquire

raw accelerometric data is the Inertial Measurement

Unit (IMU) that is an electronic device that measures

and reports a body's specific force, angular rate, and

sometimes the magnetic field surrounding the body,

using a combination of accelerometers and

gyroscopes, sometimes also magnetometers.

A programmable sensor module (“PCAN-GPS”)

(Peak-system, 2018) for position and orientation

determination, has a satellite receiver, a magnetic

field sensor, an accelerometer, and a gyroscope. The

sampled data can be transmitted on a CAN bus and

logged on the internal memory card.

The BMC050 (Bosch, 2018) is a fully

compensated electronic compass including a triaxial

geomagnetic sensor and a triaxial acceleration sensor

(6 degrees of freedom) that delivers excellent

performance in very small size.

6 PROCESSING AND

ALGORITHMS

The most common approach followed in previous

studies for detecting road condition is using sensors

to recognize the vibration patterns of the vehicle

caused due to any deformity or obstacle on the road.

Our main goals are not only detecting single

potholes, but also to classify road surfaces and

further evaluate road condition, through the

processing of the same raw data. There are two basic

categories for describing the road surface: smooth

roads (offers a high-quality driving surface to

vehicles traversing over it) and rough roads (roads

containing surface anomalies). The proposed real-

time pothole detection method based on “virtual

sensor” includes three main steps: accelerometer data

acquisition and normalization, pothole detection

approaches with algorithm of signal processing, and

pothole location determination (GPS data). In Figure

8, the block diagram of the whole implemented

algorithm is reported.

Different tests are performed in a test-drive track

with an Autonomous Driving car and a smartphone

blocked with a support horizontally on the

windshield, in order to understand if the

implemented algorithm offered good evaluation

thresholds and analyse the first results obtained with

the detection of potholes.

In-Vehicle IoT Platform Enabling the Virtual Sensor Concept: A Pothole Detection Use-case for Cooperative Safety

237

Figure 8: Block diagram real-time processing raw

accelerometer “Virtual Sensor”.

Data from the gravity and geomagnetic field

sensors is used to transform the raw linear

acceleration values from the device coordinate

system into world’s coordinate system: this

transformation allows the device to be in any

orientation while collecting data. In order to

implement a real-time algorithm, it is important to

integrate the raw accelerometer signals with the GPS

data periodically acquired by a database: in this case

it is possible to provide information about

localization of the holes.

To collect raw accelerometer data from the

different sensors, it is decided to use a frequency rate

of 50Hz (repeatable frequency on all selected

devices): supposing that the car have a travel speed

of 50km/h (speed limit in an urban scenario), using

this frequency rate, the road surface is monitored

every 0.27 meters (compliant with potholes’

dimensions). Unlike the technical tests carried out in

other works on this subject, the 50Hz value chosen

for the frequency is much higher and suitable for

tests on urban circuits (and not just tests in lab).

The raw accelerometer data (whether coming

from the smartphone, the 6LWPAN sensor or the

IMU sensor) are input to a processing algorithm that

uses a double comparison method to signal the single

pothole (Figure 9).

The first and the simplest event is tested on the

acquired data set (performed in a test-drive track

with a car and a smartphone), divided into sampling

windows, is thresholding the acceleration amplitude

at Z-axis. The features that classify the

measurements are the values exceeding specific

thresholds that identify the type of the potholes, e.g.

a large pothole or a cluster of potholes.

Figure 9: Threshold comparison method for raw

accelerometer signals axis Z (Z-THRESH and Z-DIFF).

Next, a slightly more advanced algorithm is

tested on the same acquired data set: it performs a

search for two consecutive measurements with

difference between the values above specific

threshold level; thus the algorithm detected fast

changes in vertical acceleration data.

The pothole will be indicated with the change in

the value of z-axis: in this case an alert is sent to

OneM2M Cloud every time a pothole or possible

bump is detected. If there is no (or slight) deflection

in the value of z-axis, then it is smooth road (no

bump or pothole) and if there is any deflection, a

bump or pothole is detected. If the magnitude of

value of z-axis deflects firstly in positive direction

and then towards negative direction with slight

deflection, then it will indicate a pothole.

In order to classify the different thresholds for the

bumps, it was evaluated all the different data training

implemented and it is decided to have two different

results:

 For 2≤|threshold|≤4 “possible pothole”

 For |threshold|≥ 4 “pothole detected”

The bump detection can be altered slightly to derive

another concept for surface classification.

The main idea is to count the number of bumpy

segments in a certain road section. Depending on that

number, one of the classes “smooth”, “rough”, and

“bumpy” is assigned as follows:

IoTBDS 2019 - 4th International Conference on Internet of Things, Big Data and Security

238

 For 0 ≤ |bumps| ≤ N/3 “smooth class”

 For N/3< |bumps|≤ 2N/3 “rough class”

 For 2N/3< |bumps| ≤ N “bumpy class”

The possibility of classify the types of road surface

(from the optimal to the uneven), also occurs thanks

to the additional values processed for the raw

accelerations: it was decided to communicate every

30 seconds the values of minimum, maximum,

average and variance of the 3 components

acceleration (paying particular attention to the Z

component) so that it is possible to process the data

and compare it with the dangerousness of the road

surface. All this processed data (related to single

pothole and also concerning the classification of the

types of road surface) are formatted using Data

Model provided by Sensoris, that is actually

coordinated by ERTICO (Here, 2018) (Sensor-is,

2018) and are send to the OneM2M Cloud to provide

applications and services, or used by edge

applications on the OBU (Figure 10).

Figure 10: Example of MQTT messages send to OneM2M

Cloud.

As results of this quantitative tests performed in a

test-drive track with an Autonomous Driving car and

a smartphone, we create our model for the pothole

detection algorithm both for the possibility of

acquiring and processing the accelerometric data in

real-time, both for using these prototype within a

more complex and articulated IoT platform for the

management of safety critical aspects of driving and

maintenance of the road surface. For the 6LoWPAN

sensor and the accelerometer sensor on the IMU,

laboratory tests were carried out to set the detection

threshold and to consider these solutions in an

autonomous driving in an urban scenario (even if

they are more expensive and less accessible than a

common smartphone).

7 DISCUSSION & CONCLUSION

This paper explores the possibility of a real-time

monitoring and processing for the automatic

detection of pothole and humps, has a relevant role in

order to reduce the vehicle speed, alerting vehicle

drivers and then avoid potential accidents. The

possibility to use different accelerometer sensors, not

only related with AD cars (such as popular

smartphone) is an additional advantage as it provides

timely alerts about potholes and humps.

With the help of geographic information

technologies (e.g. GPS), it is possible to establish

new-strategy based solutions that use information

about the condition of roadways. These solutions aim

to help people and entities who are responsible for

performing preventive as well as corrective

maintenance to paved roads through continuous

surveillance using their smartphones which are

connected to a cartographic server that eases location

and quality control of the work done.

A given pothole, or any other road anomaly, may

not necessarily give the same pattern during each

drive over it. The sensors readings depend upon the

speed of the vehicle, how it approached the road

anomaly and the position of the sensor (orientation).

It also depends upon the suspension system of the

vehicle (minimum/maximum vibration experienced

by the vehicle).

Since this approach based on "bundles" for the

architectural IoT system has been chosen, it is easy

to select the accelerometric data collection system

between smartphone, 6LoWPAN sensor and IMU:

all the raw accelerometer data can be collected at a

sampling rate of 50Hz and in the first tests that were

carried out there were no gaps of accelerometric

values obtained preferring one type of sensor

compared to others.The proposed solution proven to

be able to quantitatively evaluate the quality of the

road: not only single potholes are detected but also

longer stretches of low-quality pavement. This could

allow taking the pavement quality into consideration

while calculating routes for navigation system.

In order to improve the efficiency and accuracy

of the detection of potholes and to enhance all this

case studies, as future work, it has been planned to

continue to perform more and different tests using

AD cars (also in the AUTOPILOT scenario): with a

larger test database, it will therefore be possible to

exhaustively process the results obtained with the

different sensors and to establish the reliability and

the trust of the implemented algorithms.

This system could be also deployed with a neural

network to implement Machine Learning on

In-Vehicle IoT Platform Enabling the Virtual Sensor Concept: A Pothole Detection Use-case for Cooperative Safety

239

Android: Machine Learning techniques can be

applied instead of threshold-based classification as

different vehicles may yield different sensor data for

same pothole. It will make the system more efficient

and introduce self-calibration functionality.

ACKNOWLEDGEMENTS

This work was supported by the European Union

through the AUTOPILOT project (H2020-IOT-2016,

grant agreement no. 731993).

REFERENCES

Akinwande, V., Adewole, K., Bello, O. & Akintola, A.,

2015. Automatic and real-time pothole detection and

traffic monitoring system using smartphone

technology. International Conference on Computer

Science Research and Innovations (COSRI).

Aksamit, P. & Szmechta, M., 2011. Distributed, mobile,

social system for road surface defects detection. 5th

International Symposium on Computational

Intelligence and Intelligent Informatics (ISCIII). IEEE,

september, pp. 37-40.

Android, 2018. [online] available at:

https://developer.android.com/guide/topics/sensors/sen

sors_overview

Android, 2018. [online] available at:

https://developer.android.com/guide/topics/sensors/sen

sors_motion

Aniket, D. & Vivek, D., 2016. Advance pothole

notification technique for vehicle road safety using

android applications. International Journal of

Enhanced Research in Science, Technology &

Engineering, 5(9).

Astarita, V. et al., 2012. A mobile application for road

surface quality control: uniqualroad. Procedia-Social

and Behavioral Sciences, issue 54, pp. 1135-1144.

Balakuntala, S. & Venkatesh, S., 2013. An intelligent

system to detect, avoid and maintain potholes: a graph

theoretic approach. Arxiv preprint arxiv:1305.5522.

Bhatt, U., Mani, S., Xi, E. & Kolter, J., 2017. Intelligent

pothole detection and road condition assessment. Arxiv

preprint arxiv:1710.02595.

Bhoraskar, R., Vankadhara, N., Raman, B. & Kulkarni, P.,

2012. Wolverine: traffic and road condition estimation

using smartphone sensors. Fourth International

Conference on Communication Systems and Networks

(COMSNETS) IEEE, January, pp. 1-6.

Bosch, 2018. [online] available at: https://www.bosch-

sensortec.com/bst/products/all_products/bmc050

Danti, A., Kulkarni, J. Y. & Hiremath, P. S., 2012. An

image processing approach to detect lanes, pot holes

and recognize road signs in indian roads. International

Journal of Modeling and Optimization, 2(6), p. 658.

De Silva, G. D. et al., 2013. Proceedings of the

International Conference on Advances in ICT for

Emerging Regions. Colombo, Sri Lanka, s.n., pp. 6-9.

Eriksson, J. Et al., 2008. The pothole patrol: using a mobile

sensor network for road surface monitoring.

Proceedings of the 6th International Conference on

Mobile Systems, Applications, and Services. ACM,

june, pp. 29-39.

ERTICO, 2017. [online] available at: http://autopilot-

project.eu/

Ferrera, e. Et al., 2017. Xmpp-based infrastructure for iot

network management and rapid services and

applications development. Annals of

telecommunications, 72(7-8), pp. 443-457.

Here, 2018. [online] available at: https://www.here.com/

en/vision/innovation/sensoris

Jothi, S., Priyanka, S., Yuvaraj, P. & Kalaivani, S., 2010.

Automatic detection of potholes and humps on roads to

aid drivers. International Journal of Advanced

Research in Management, Architecture, Technology

and Engineering (IJARMATE), 2(3).

Kalra, N., Chugh, G. & Bansal, D., 2014. Analyzing

driving and road events via smartphone. International

Journal of Computer Applications, 98(12), pp. 5-9.

Langle, L. & Dantu, R., 2009. Are you a safe driver?.

Computational Science and Engineering, 2009.

CSE'09. IEEE, volume 2, pp. 502-507.

Maag, C., Muhlbacher, D., Mark, C. & Kruger, H. P.,

2012. Studying effects of advanced driver assistance

systems (adas) on individual and group level using

multi-driver simulation. IEEE Intelligent

Transportation Systems Magazine, 4(3), 4(3), pp. 45-

54.

Mednis, A. et al., 2011. Real time pothole detection using

android smartphones with accelerometer. International

Conference on Distributed Computing in Sensor

Systems and Workshops (DCOSS). IEEE, june, pp. 1-6.

Mertz, C., 2011. Continuous road damage detection using

regular service vehicles. Proceedings of the Its World

Congress, October.

Mohan, P., Padmanabhan, V. N. & Ramjee, R., 2008.

Nericell: rich monitoring of road and traffic conditions

using mobile smartphones. Proceedings of the 6th

ACM Conference on Embedded Network Sensor

Systems. ACM, November, pp. 323-336.

Mukherjee, A. & Majhi, S., 2016. Characterisation of road

bumps using smartphones. European Transport

Research Review, 8(2), p. 13.

Ngs, 2018. [online] available at: http://ngs-sensors.it/it/

Peak-system, 2018. [online] available at:

https://www.peak-system.com/pcan-gps.378.0.html

Scarrone, e., 2016. [online] available at:

www.homegatewayinitiative.org

/userfiles/file/downloads/proceedings/s10-

%20scarrone%20-%20onem2m.pdf.

Sensor-is, 2018. [online] available at: http://sensor-is.org/

Wang, H. W. et al., 2015. A real-time pothole detection

approach for intelligent transportation system.

Mathematical Problems in Engineering.

IoTBDS 2019 - 4th International Conference on Internet of Things, Big Data and Security

240