An Assistance System for Collision Avoidance Using

Context-Sensitive Prediction

David Sanders

, Malik Haddad

, Giles Tewkesbury

, Shikon Zhou

and Alexander Gegov

Faculty of Technology, University of Portsmouth, Anglesea Road, Portsmouth, U.K.

Northeastern University – London, St. Katharine’s Way, London, UK

Keywords: Assistance System, Collision Avoidance, Context-Sensitive, Prediction.

Abstract: An alert and collision avoidance system is introduced. A new method has been used to calculate a closest

point of approach, incorporating a context-sensitive prediction. Movement and routing information were used

and an approach for taking evasive action is described. When a potential collision was detected, then an

estimation was made of the direction of movement and an evasive manoeuvre was selected. A closest point

of approach was calculated between the wheelchair and any object detected in its vicinity. A linear motion

vector was calculated based on current speed, position and direction and that vector was compared with the

object position.

1 INTRODUCTION

This paper presents a system to reduce human

workload and misperceptions of driving situations.

The system provides pro-active collision avoidance

including methods for collision avoidance and

predicting wheelchair behaviour. It is a new Traffic

Alert and Collision Avoidance System, similar to

those used with commercial aircraft (Holdsworth,

2003). Figure 1 illustrates system infrastructure and

main functions. The system was created to reduce

collisions and support drivers with a collision alarm.

A closest point of approach was calculated by

applying context sensitive behaviour prediction

(Steidel, 2019). Additionally, an alarm helped with

assessing potential hazards. Cooperative manoeuvre

negotiation and critical situation resolution systems

contributed to reducing potential collisions.

Figure 1: System principles and functionalities.

This paper gives an overview of the concepts of

the system and discusses details of the system

presented in Figure 1, as well as describing the

concept of Escalation States shown in figure 2, which

depicts the likelihood of collisions.

Escalation States are used to assess how critical a

potential collision is and to provide a weighted

warning. The approach for predicting powered

wheelchair behaviours and the introduction of alarms

are developed. Following this, the concept of

reducing misunderstandings during potential

collisions is defined, focusing on evasive

manoeuvres. The integration of the system into a

powered wheelchair is briefed, followed by the

description of the validation process.

Figure 2: Escalation States.

2 DETECTING NEAR MISSES

AND COLLISIONS

The system tracked the movement of a powered

wheelchair and attempted to warn its user while

Sanders, D., Haddad, M., Tewkesbury, G., Zhou, S. and Gegov, A.

An Assistance System for Collision Avoidance Using Context-Sensitive Prediction.

DOI: 10.5220/0011903800003612

In Proceedings of the 3rd International Symposium on Automation, Information and Computing (ISAIC 2022), pages 98-103

ISBN: 978-989-758-622-4; ISSN: 2975-9463

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

minimising the number of alarms. Using context-

sensitive behaviour prediction, not only the number

of collision warnings, but also the number of

exaggerated and false alarms could be reduced.

Auto tracking with situation evaluation was

introduced to improve collision avoidance and reduce

the workload for wheelchair users. A concept of

Escalation States for assessing collisions was

developed in this work. This simplified situation

evaluation used a series of Escalation States.

Kamijo detected traffic patterns in video images

and was able to detect accidents (Kamijo et al, 2000).

Using a hidden Markov Model, the system learnt

various behaviour patterns. Xue presented a method

for collision-free trajectory planning (Xue et al,

2009). The method had three-steps. For the powered

wheelchair this would be: Identify target and powered

wheelchair position; Detect potential collisions; and

Control wheelchair. A three-degree of freedom model

was used to generate a possible route.

Xue used the potential field method (Sanders,

1995; Sanders et al, 1991; 1992a; 1992b; 2010a;

2010b, 2011; 2015) for route finding, whilst Tam and

Bucknall concentrated on close ranges and developed

a method for assessing collision risk by determining

encounter type (Tam, 2013). A method to evaluate the

probability of a collision was presented by Montewka

et al (2015). The shortest distance at which a collision

could be avoided was calculated, namely, Minimum

Distance to Collision. Youssef et al (2014) took work

from (Montewka et al, 2015) to develop a

probabilistic approach to select collision scenarios.

A Conflict Ranking Operator was presented by

Zhang (2015). Distance, speed and relative angle

were considered; therefore, decisions could be made

to avoid near-misses. Van Iperen (2015) introduced

two ways to detect near-misses. One deploys the

closest point of approach calculation while the other

used domains. The author evaluated the main

indicators for the level of safety in a specific area.

A review of work concerning collision avoidance

revealed some factors that can lead to a collision. The

biggest problems in collision avoidance are human

factors. That means to prevent collisions, the

uncertainty of the behaviour of other powered

wheelchairs and other humans needs to be

considered. Risk (and probability) of collision

increases with higher uncertainty. Distance, speed

and angle need to be considered in order to assess the

risk of a collision (Haddad et al, 2020a; 2020b;

2020c; Sanders et al, 2020a; 2020b.

The closest point of approach calculation

primarily considers distance. Manoeuvrability is also

important in solving the uncertainty problem by

generating the possible set of potential states of the

powered wheelchair. These findings were considered

when creating system concepts.

3 COLLISION WARNINGS AND

AVOIDANCE

Assistance systems have tended to generate collision

warnings based on a comparison of linear movement

vectors. However, a closest point of approach

calculated in this way can be unrealistic and

exaggerated, since external conditions and typical

movement patterns are not included. In addition,

alarms can be ignored by wheelchair users. The

newly developed system in this work uses two

concepts for generating improved collision warnings.

The first is called Critical Pose, which is an extension

of the traditional closest point of approach. In

addition to this, the system predicts the most probable

behaviour of a wheelchair. These concepts can avoid

some unnecessary alarms. The system also evaluates

the likelihood of collision using Escalation States. In

the following sections the Escalation States, the

closest point and behaviour prediction are described.

The system uses a concept of Escalation States to

assess the likelihood of collisions. Figure 2 shows the

concept. The further right an escalation state is plotted

in figure 2, the more critical the state.

In order to determine the different states, the

closest point must first be calculated. Next, the time

and distance that a powered wheelchair will need to

reach the calculated closest point is calculated. In this

work, the thresholds for each Escalation State were

obtained experimentally, which could be varied

depending on the abilities of different drivers

(Sanders et al, 2010). If the closest point was more

than a pre-set range (for example one meter) the

situation was labelled “Clear”, i.e.,no danger existed.

If the wheelchair continued to move towards an

object and required less than a pre-set time (for

example one second) to reach the closest point, the

wheelchair was in a Recommendation State. In this

state, the system triggered a warning. Behaviour

prediction would be used. This prediction gave an

estimation about how the powered wheelchair would

travel based on the analysis of historic data.

The wheelchair moved into a Danger State when

the time was less than a pre-set time (for example half

a second) and the distance to the closest point was less

than a set point (for example 0.5 meters). During this

State, the system could help wheelchair users avoid a

collision by applying a cooperative negotiation

An Assistance System for Collision Avoidance Using Context-Sensitive Prediction

algorithm that shared control of the wheelchair with

the driver (Sanders 2017; 2018).

This was the last possible state to prevent

collision. An Alarm was generated if the powered

wheelchairs were less than a critical pre-set time or

distance, such as 300 milliseconds or 0.2 meters from

the closest point. The system performed an

emergency manoeuvre to avoid collision by taking

control of the wheelchair. If a manoeuvre was not

carried out then a collision would become inevitable.

4 CRITICAL POSE

A way of assessing and identifying hazardous

wheelchair encounters was to calculate the closest

point of approach. A linear vector for a powered

wheelchair was generated based on position, speed

and course. These vectors were then compared to

produce a closest point of approach (a geographical

point). The closest point of approach was extended by

the calculation of two additional values: Distance to

closest point of approach (D

) and Time to closest

point of approach (T

). Drivers received a warning

if T

and D

fell below threshold values.

This procedure did have drawbacks. It was based

on the abstraction of wheelchair motion to linear

motion and the system lacked context information.

All sensors used for measuring course, speed and

position as well as those for detecting obstacles had

inaccuracies. A snapshot could combine these errors

and yield an imprecise closest point of approach. To

avoid this, the idea of a context-sensitive closest point

of approach was introduced. The calculation was

combined with a consideration of sensor inaccuracies.

A term for describing the result was introduced

“Critical Wheelchair Pose (closest point)”. In contrast

to the closest point of approach, the closest point was

defined by two values: position at which the

wheelchair had the shortest distance to an object and

the pose of the wheelchair at this position. Figure 3

illustrates the closest point concept and figure 4

represents the Evasive Manoeuvre Negotiation

Figure 3: Calculation of Closest Point of Approach.

Figure 4: Above: Single object (assumed to be another

wheelchair). Below: Multiple objects (assumed to be other

wheelchairs).

A wheelchair is depicted on the left side of figure

3. The straight line running from the wheelchair in the

direction of travel is the current course. The straight

line running from the object on the right side of figure

3 shows the dangerous path that might cause

collision. Sensor inaccuracies and possible changes

are represented by the two funnels. The funnel

describes the probability of the wheelchair's and

object’s position in the future, taking sensor errors

and possible change into account.

The funnel for the wheelchair was smaller than for

the object, because the wheelchair and wheelchair

sensor inaccuracy can be determined whilst the object

is usually unknown. Thus position, speed and course

of the wheelchair can be determined with a higher

accuracy so that future positions of the wheelchair

can be predicted more precisely. To calculate the

closest point, the two funnels are compared. Due to

the inaccuracies, the wheelchairs may be located at

any positions within their respective funnels. For the

closest point, the minimum distance between the

funnels was considered. This pessimistic assumption

helped better clarify the criticality of the situation.

The problem was addressed using path planning

algorithms (Sanders, 1995; Sanders et al, 1991)

during the calculation and introducing No-Go-Areas

and routing information. The outcome for the

calculation was that a wheelchair would not pass

through a No-Go-Area. Thus, a calculated closest

point should not be within a No-Go-Area. The system

used this extra information to calculate a closest point

using the current route for the wheelchair. The

advantages of using this method over conventional

methods was that unnecessary and exaggerated

alarms based on a linear closest point of approach

were avoided. This reduced stress and workload for

the wheelchair user.

When powered wheelchairs were in the

Recommendation State, the system predicted the

most probable behaviour of the powered wheelchair

using a rule-based approach (Sanders et al, 2011).

The approach had two levels. On the first level, a

prediction about the most probable behaviour of the

ISAIC 2022 - International Symposium on Automation, Information and Computing

100

powered wheelchair was made. A possible influence

of obstacles in the proximity were ignored. This

resulted in the most probable situation in the

proximity of a wheelchair, which was then used to

estimate potential collision risks. In order to support

the wheelchair users to assess and avoid potential

collisions, situations were classified as Head-On,

Overtaking or Crossing

(Steidel, 2019). This

classification was made in a second level. Following

this, the most probable resolution was predicted.

Mean values for course and speed of matching

historic movements were considered as a future value

for prediction. The most probable behaviour of the

wheelchair was predicted using an Artificial Neural

Network. Predicted behaviour was modelled using a

process similar to the one used by Ornstein-

Uhlenbeck to model uncertainty for.

Typical tracks were generated based on historic

data. Rule-based prediction was based on an

association between historic tracks and the current

track. The whole track was not predicted, but instead

the next likely powered wheelchair position was

produced a set time ahead (one second). Hence

typical behaviour was used as a prediction. Behaviour

of a powered wheelchair was modelled using Kernel

Density Estimation and predicted for a set time span.

In order to extract and learn patterns, the working area

was divided into different regions, i.e. a grid. Typical

patterns in a region could then be learned. This

enabled an extended possibility to predict behaviour

ahead of a certain time frame. As a result, future

positions in a grid were predicted. By applying Neural

Associative Learning, future behaviour was predicted

based on the patterns.

5 PREDICTION

The prediction algorithm in the first level required the

most common behaviour as a basis to predict the most

probable powered wheelchair behaviour.

5.1 Most Probable Behaviour

A rule-based system was used to make-a-decision

based on previous knowledge. Required knowledge

was obtained by analysing historic data to extract

patterns and rules for predicting behaviour in a

similar way to Oltmann and Pallotta (is there a

reference here? ). Extracted behavioural patterns were

modelled as a graph. The graph contained two types

of nodes: one representing geographical points where

objects were in the vicinity; the other represented

usual (historical) behaviour, that is points (targets) the

wheelchair was expected to move towards. For each

of those target nodes, a frequency distribution was

calculated which described recorded wheelchair

behaviours. The first-level prediction algorithm used

this information to decide where a powered

wheelchair would head. The rule-based approach for

predicting powered wheelchair behaviour on the first-

level was divided into two parts. The first aimed to

predict the most probable target. The node with the

highest accordance was selected as a potential

destination so that a possible path through the graph

from the current position to the predicted target could

be generated.

5.2 Prediction of Evasive Manoeuvres

This required predicted behaviour as an input.

Potential hazardous collisions were estimated. The

closest points on the predicted object track and

wheelchair route were predicted. If distance between

them at the closest point fell below a threshold, this

encounter was examined. The encounter was labelled

Head-On, Overtaking or Crossing. In the data

analysis phase, encounters for each of these situations

were extracted. A rule for evasive manoeuvres was

statistically extracted and applied to the identified

encounter based on the first-level prediction. As a

result, an evasive manoeuvre could be performed /

executed.

5.3 Evaluation

Prediction algorithms were applied using historical

data collected over three months of testing by two

interdisciplinary MEng Project Groups at the

University of Portsmouth. The testing region

consisted of two downstairs rooms and a section of

connecting corridor at the University. For evaluation

purpose, the related data was grouped as tracks. racks

and positions were selected randomly. Based on

positions, a potential destination and corresponding

tracks were predicted. Afterwards, the distance

between the historic and predicted track was

calculated. To evaluate the prediction algorithm for

the second-level, a route for a wheelchair was

generated. Several encounters with predefined

evasive manoeuvres were created. Starting from a

point on the track, the second algorithm yielded an

evasive manoeuvre. The predicted manoeuvre was

compared to the real evasive manoeuvre.

The algorithm for predicting Give-Way

performed better than the prediction of the destination

as shown in Table 1. For determining the distance

between the predicted track and the historic track, the

An Assistance System for Collision Avoidance Using Context-Sensitive Prediction

101

median of the distance between these two tracks was

used. This yielded an average distance of 20

centimetres.

Table 1. Prediction results.

Predicted Destination Give-Way

Correct 65% 76%

Incorrect 35% 24%

6 REDUCING

MISUNDERSTANDINGS

Before taking over the control from the wheelchair

user and performing an evasive manoeuvre it was

important that a consistent operational picture was to

be maintained. Misunderstandings could lead to

erroneous actions, which would result in collisions

even though the wheelchair user might be trying to

avoid it. To address this problem, the system included

an algorithm to negotiate evasive manoeuvres.

If a driver entered a Danger State, an evasive

manoeuvre would be applied. The calculation of the

evasive manoeuvre and the negotiation principles

aimed to assist a wheelchair driver but not to take over

control. A near miss or collision was classified as

Head-On, Crossing or Overtaking. This was a

parameter for the negotiation algorithm since

different kinds of evasive manoeuvres were

recommended depending on the classification. In

addition, it is necessary to have a list of assumptions

as:

• Objects were another wheelchair or person;

• Other wheelchairs and people would tend to want

to avoid collision; and

• Any other powered wheelchairs were equipped

with sensor systems.

Two cases were examined and analysed (Figure 4).

For the single object shown at the top part in Figure

4, the negotiation algorithm generated a manoeuvre

based on a wheelchair-to-wheelchair encounter.

Negotiation started with the evasive manoeuvre. The

proposed moves were then stored until the situation

turned into a potential near miss or collision. The

system monitored the developmnet of the situation

and would raise an alarm if a dangerous situation

(potential collision) was detected.

In the second case, n powered wheelchairs are

present (where n>1). The processing procedure was

similar to that for a single object. The difference was

that manoeuvres depended on the projected

movements of multiple wheelchairs rather than a

single one. If one of the powered wheelchairs

continued to get closer, then negotiation was

cancelled, and the system took control to avoid

collision. The use of a pseudo-collaborative approach

to negotiating evasive manoeuvres brought some

benefits. The negotiation algorithm classified

potential collisions as wheelchair-to-wheelchair

encounters. Within the system an exchange of the

result from calculations for both wheelchairs was

carried out. This prevented possible

misunderstandings in the assessment of wheelchair-

to-wheelchair encounters. That also enabled a distinct

classification into “Continue” or “Avoid”. As the

wheelchair kept moving forward, it became clear

whether each powered wheelchair should take a

proposed manoeuvre. This prevented

misunderstandings, human errors and reduced the

workload of the wheelchair user.

The system was tested in a number of simulations

and representative scenarios were generated to cover

different wheelchair-to-wheelchair, wheelchair-to-

people, and wheelchair-to-static-object encounters. In

addition, scenarios were extracted from real

wheelchair collision incidents during real world

testing. Iterative testing and development enabled

faster integration and validation. Besides testing in

simulation, a Bobcat II wheelchair was used to

perform practical tests. In a total of three tests, the

system was tested at different stages of development.

The general functionality of the features described

above was successfully tested and validated.

7 DISCUSSION AND

CONCLUSION

Concepts were created to support wheelchair users in

situations where a collision might occur. An

intelligent hazard assessment considered external

information such as No-Go areas, route information

and sensor inaccuracies. The intelligent prediction of

wheelchair behaviours enabled drivers to be warned

of potential collisions at an early stage. A function for

the evaluation of wheelchair encounters and the

cooperative negotiation of evasive manoeuvres

allowed intelligent collision avoidance. Concepts

were successfully tested and evaluated through

simulations, as well as practically with two

wheelchairs. Future work will consider more

affecting factors, especially reaction times and the

ISAIC 2022 - International Symposium on Automation, Information and Computing

102

delays, as well as ways of improving assisted driving

and modelling.

REFERENCES

Haddad, M., Sanders, D., 2020a Deep Learning architecture

to assist with steering a powered wheelchair, IEEE

Trans. Neur. Sys. Reh. 28 12 pp 2987-2994.

Haddad, M., Sanders, D., Ikwan, F., Thabet, M., Langner,

M., Gegov, A., 2020b Intelligent HMI and control for

steering a powered wheelchair using a Raspberry Pi

microcomputer, Proc. 2020 IEEE 10th International

Conference on Intelligent Systems-IS (Varna) pp 223-

228.

Haddad, M., Sanders, D., Langner, M., Ikwan, F.,

Tewkesbury, G., Gegov, A., 2020c Steering direction

for a powered-wheelchair using the Analytical

Hierarchy Process, Proc. IEEE 10th International

Conference on Intelligent Systems (Varna) pp 229-234.

Holdsworth, R., 2003 Autonomous In-Flight Planning to

replace pure Collision Avoidance for Free Flight

Aircraft using Automatic Dependent Surveillance

Broadcast, Swimburne University IMO (1996),

Resolution MSC.64 67 Adoption of New and Amended

Performance Standards, Int. Maritime Org.

Kamijo, S., Matsushita, Y,, Ikeuchi, K., Sakauchi, M., 2000

Traffic monitoring and accident detection at

intersections. IEEE Trans. Intell Transp Syst 1 2

pp108–118.

Montewka, J., Goerlandt, F., Kujala, P., 2012

Determination of collision criteria and causation factors

appropriate to a model for estimating the probability of

maritime accidents, Ocean Eng. 40 pp 50–61.

Sanders, D., Moore, A., Luk, B., 1991 A joint space

technique for real time robot path planning Proc. Fifth

Int’ Conf on Advanced Robotics, 1991. 'Robots in

Unstructured Environments', IEEE, pp 1683 – 1689.

Sanders, D., Jaques, M., Clothier, H., 1992a Geometric

modelling for real time flight simulator applications

Proc. IEE Colloquium on Advanced Flight Simulation.

IET Conference Publications, pp 4/1 - 4/6.

Sanders, D., Harris, P., Mazharsolook, E., 1992b Image

modelling in real time using spheres and simple

polyhedral Proc. H Schroder (ed.), International

Conference on Image Processing and its Applications

IET Conference Publications, pp 433 – 436.

Sanders, D., 1995 The modification of pre-planned

manipulator paths to improve the gross motions

associated with the pick and place task. Robotica 13 1

pp 77-85.

Sanders, D., Graham-Jones, J., Gegov, A., 2010a

Improving ability of tele-operators to complete

progressively more difficult mobile robot paths using

simple expert systems and ultrasonic sensors. Industrial

Robot: An International Journal 37 5 pp 431-440.

Sanders, D., Lambert, G., Graham-Jones, J., Tewkesbury,

G., Onuh, S., Ndzi, D., Ross, C., 2010b A robotic

welding system using image processing techniques and

a CAD model to provide information to a multi-

intelligent decision module, Assembly Autom 30 4 pp

323-332.

Sanders, D., Tewkesbury, G., Graham-Jones, J., 2011

Simple rules to modify pre-planned paths and improve

gross robot motions associated with pick & place

assembly tasks. Assembly Autom, 31 1 pp 69-78.

Sanders, D., Tewkesbury, G., Gegov, A 2015 Fast

transformations to provide simple geometric models of

moving objects. Intelligent robotics and applications,”

,Lecture notes in computer science 9244

Springer pp

604-617.

Sanders, D., 2017 Using self-reliance factors to decide how

to share control between human powered wheelchair

drivers and ultrasonic sensors, IEEE Trans Neur Sys

Rehab 25 8 pp 1221-1229.

Sanders, D., 2018 Non-model-based control of a wheeled

vehicle pulling two trailers to provide early powered

mobility and driving experiences, IEEE Trans Neur Sys

Rehab 26 1 pp 96-104.

Sanders, D., Haddadm M., Tewkesbury, G., Thabet, G.,

Omoarebun, P., Barker, T., 2020a Simple expert system

for intelligent control and HCI for a wheelchair fitted

with ultrasonic sensors, Proc. of the 2020 IEEE 10th

International Conference on Intelligent Systems

(Varna) pp 211-216.

Sanders, D., Haddad, M., Langner, M., Omoarebun, P.,

Chiverton, J., Hassan, M., Zhou, S., Vatchova, B.,

2020b Introducing time-delays to analyze driver

reaction times when using a powered wheelchair, Proc.

of SAI Intelligent Systems Conf (Amst) pp 559-570.

Steidel, M., 2019 Axel Hahn system – An Assistance

System for Collision Avoidance at Sea,” Proc. 18 th

International Conference on Computer and IT

Applications in the Maritime Industries Technische

Universität (Hamburg) pp 261-273.

Tam, C., Bucknall, R., 2010 Collision risk assessment for

ships. J. Marine Science and Technology 15 3 pp257–

273.

Van Iperen, E., 2015 Classifying Ship Encounters to

Monitor Traffic Safety on the North Sea from AIS Data,

TransNav. Int. J. Marine Navigation and Safety of Sea

Transportation 9 1 pp 51–58.

Xue, Y., Lee, B., Han, D., 2009 Automatic collision

avoidance of ships J. Eng. for the Maritime

Environment 223 1 pp 33–46.

Youssef, S., Kim, Y., Paik, J., Cheng, F., Kim, M., 2014

Hazard identification and probabilistic scenario

selection for ship-ship collision accidents, Int. J.

Maritime Eng 156 A1 pp 61–80.

Zhang, W., Goerlandt, F., Montewka, J., Kujala, P., 2015 A

method for detecting possible near miss ship collisions

from AIS data, Ocean Eng 107 pp 60–69.

An Assistance System for Collision Avoidance Using Context-Sensitive Prediction

103