Multicriteria Analysis of the Robotic Systems Autonomy Using Fuzzy

Calculations

Sergey Sokolov

and Vladimir Sudakov

Keldysh Institute of Applied Mathematics of Russian Academy of Sciences, Miusskaya sq. 4, Moscow, Russia

Keywords: Robotic Systems, Degree of Autonomy, Multicriteria Analysis, Fuzzy Areas, Preferences, Unification.

Abstract: Against the background of the ever-increasing needs for robotic systems (RS) with an increased degree of

autonomy and the emerging transition to their widespread use, the need for technologies for quality

assessment and multi-criteria analysis of the autonomy degree of such devices is becoming more urgent. The

article describes the current state of issues assessing and comparing the degree of autonomy of unmanned

systems using the vector criterion. Well-known estimates of the degree of autonomy are given. The existing

classification system distinguishes between informational and intellectual autonomy, which are considered in

close connection. Solutions are proposed that make it possible to formulate estimates of the autonomy degree

of robots in various areas of economics based on the theory of fuzzy sets. Based on the method of fuzzy areas

of preference, it becomes possible to obtain estimates of the degree of autonomy, taking into account the

judgments of the decision-maker. One of the positive consequences of this approach is the unification of

formulations and solutions in the tasks of information support in the RS, which, in turn, facilitates interaction

between users, customers and developers.

1 INTRODUCTION

The International Federation of Robotics (IFR)

defines a robot as a working mechanism that is

programmable along several axes with some degree

of autonomy and is capable of moving within a

defined environment to perform assigned tasks (IFR,

2015). From this definition, it follows that the

essential features of the concept of “robot” (i.e.,

criteria for the analysis of mechanisms created in

different historical periods) are: autonomy, which

means that “a robot is able to interpret the

environment in which it is located and adapt to the

assigned tasks "(Kaysner et al., 2016); and the ability

to program it in several directions. Another common

definition among scientists and practitioners is the

following: a robot is “any machine capable of

perceiving the environment and reacting to it based

on independently made decisions” (Nesmelov, 2022).

Thus, the key difference between robots and other

machines is considered to be “autonomy”: the robot

is able to interpret the environment in which it is

located and adapt to the assigned tasks. Robots are

https://orcid.org/0000-0001-6923-2510

https://orcid.org/0000-0002-1658-1941

evolving from programmed automation to semi-

autonomous and more autonomous complex systems.

Fully autonomous systems will be able to

independently make “decisions” in their intended

environment and perform tasks without human

assistance. In general, we can say that the trends in

modern robotics are increasing their autonomy and

the ability to solve various problems through the use

of artificial intelligence. Among the known types of

autonomy of technical devices such as logistical,

informational, and intellectual (Ermolov, 2012), it is

intellectual autonomy that stands out, closely related

to informational autonomy, and necessary for solving

problems in a previously unknown, changeable

environment.

What should RS have in order to be classified as

autonomous, and what should it be able to do execute

what algorithms? Experts predict that in development

work to create special-purpose robots, in accordance

with existing trends, the following should be

implemented (Murphy, 2020):

• increased resource autonomy;

• modularity of construction and reconfigurability;

Sokolov, S. and Sudakov, V.

Multicriteria Analysis of the Robotic Systems Autonomy Using Fuzzy Calculations.

DOI: 10.5220/0012418200003636

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2024) - Volume 3, pages 915-919

ISBN: 978-989-758-680-4; ISSN: 2184-433X

915

• constructive and technological unification of

samples and their key functional components;

• noise-resistant multi-channel means and systems

of information and control interaction and

identification;

• intelligent software and algorithmic tools that

allow for recognition of objects and the working

environment, reflexive forecasting of the

development of events, planning of rational (optimal)

behavior, and, as a consequence, adaptively

controlled functioning of special-purpose robots in

uncertain, dynamically changing heterogeneous

application conditions;

• intelligent software and algorithmic tools that

allow for the integration of different types of special-

purpose robots into a single group with subsequent

control of their joint actions in similar,

heterogeneous, and mixed combat formations;

• intelligent systems for human-machine interface

and decision support for operators controlling

special-purpose robots when solving combat (strike,

fire), support and special tasks.

Various criteria for autonomy are found in

publications, for example, the Society of Automotive

Engineers (SAE). To help automotive engineers,

governments, and insurance companies better

understand this new technology, SAE has defined six

(including no autonomy) levels of automotive

autonomy (SAE, 2023):

• Level 0: Not at all autonomous; the driver has sole

control of the vehicle.

• Level 1: One function is automated, but does not

necessarily use information about driving conditions.

A vehicle operating with simple cruise control will

qualify as Level 1.

• Level 2: Acceleration, deceleration, and steering

are automated and use sensory data from the

environment to make decisions. Modern cars with

cruise control automatic lane keeping, or collision

mitigation braking fall into this category. The driver

remains solely responsible for the safe operation of

the vehicle.

• Level 3: At this level, all safety functions are

automated, but the driver must still take control in an

emergency that the car cannot handle. An example

would be Tesla cars with the Autopilot feature

enabled. This is the most controversial level because

it requires the human driver to remain alert and

focused on the driving task even though the car is

doing most of the work. People would naturally find

this situation more tedious than simply driving a car,

and many in the autonomous vehicle community

worry that the driver's attention could be diverted

from the task at hand, leading to disastrous results.

Some automakers choose to skip Level 3 and go

straight to Level 4.

• Levels 4 and 5: These are fully autonomous levels

where the car makes all driving decisions without

human intervention. The difference is that Level 4

cars are limited to a specific set of driving scenarios,

such as city, suburban, and highway driving, while

Level 5 cars can handle any driving scenario,

including off-road driving.

A similar autonomy scale has been adopted

among drone developers (PROXIMA, 2023). There

are five levels of UAV autonomy, based on the

principles of self-driving vehicle autonomy.

• Level 0: No autonomy.

• Level 1: Some systems are automated, such as

altitude control, but a human controls the UAV.

• Level 2: Multiple simultaneous systems are

automated, but a human still controls the UAV.

• Level 3: The UAV operates autonomously under

certain conditions, but a person monitors its

movement.

• Level 4: The drone is autonomous in most

situations; a person can take over control, but this is

not necessary.

•

Level 5: The drone is completely autonomous.

Currently, the development of UAV technology is

between levels 3 and 4, where the drone can make

some decisions autonomously, but a person still needs

to observe the operation process of the device. The

main challenge in reaching level 5 is solving technical

problems and overcoming laws, regulations, and even

social acceptance in different regions.

Through the efforts of this ALFUS group, a clear

diagram has been proposed of what constitutes an

idea of the autonomy of a system and by what

indicators the autonomy of a particular system can be

assessed (ALFUS, 2004).

Autonomy indicators (sets of metrics) for a

detailed model that determines the level of autonomy

are summarized in the “space” of autonomy.

Mission complexity can be measured using

indicators such as levels of subtask completion,

decision-making and collaboration, knowledge and

perception requirements, planning and execution

efficiency, etc.

The level of human dependency can be measured

using indicators such as interaction time and

frequency, operator workload, skill levels, robot

initiation, etc.

Environmental complexity can be measured by

the size of obstacles, density and traffic, terrain types,

urban traffic characteristics, ability to recognize

friends, enemies, bystanders, etc. The detailed model

of the ALFUS framework contains the following

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

916

defining concepts:

• Unmanned systems (UMS) autonomy touches

many technical areas. Task complexity and

adaptability to the environment are some of the key

aspects.

• The nature of UMSs' collaboration with human

operators, such as levels of involvement and types of

interactions, is important to the possibility of

autonomy.

• Performance factors such as mission success

probability, response time, accuracy, resolution, and

latency tolerance influence UMS autonomy levels

(Huang et al., 2003).

Work is currently underway to determine

measurement scales for the proposed metrics.

Decision-makers may be guided by some complex

algorithms, as opposed to simple weighted averages,

to determine the resulting levels of vehicle autonomy.

It is recognized that the level of autonomy is an

extremely complex issue. There are a number of

problems that require resolution. These include:

1. Clarification of quantitative indicators and

prioritization. Identification of coincidences and

conflicts between them along the three axes of the

proposed space.

2. Development of standard measuring scales for

metrics.

3. Create high-level definitions of levels of

autonomy for the summary or executive model.

4. Development of methods and plans for testing

and confirming the levels of autonomy of unmanned

vehicles.

5. Defining and installing a domain-specific

autonomy level model for selected programs.

In addition, it is necessary to note the following

disadvantages of the proposed scheme:

• Assessments based on indicators (criteria) are

often assigned by experts, but the method does not

take into account the degree of confidence of the

expert in the assigned assessment.

• Criteria weights may also have a fuzzy (blurry)

character.

• The use of weighted summation leads to implicit

mutual compensation of criteria, which means that

unsatisfactory scores for one criterion will be offset

by good scores for others.

Suggestions for overcoming some of these

problems will be discussed below.

2 METHOD

Based on experience in the development of robotic

systems (Sokolov, 2022) and analysis of methods of

applied mathematics in various fields (Sudakov and

Zhukov, 2023), methods for developing a generally

very productive and promising scheme of the ALFUS

group based on replacing the weighted summation of

point estimates with fuzzy judgments are proposed.

Let a vector of fuzzy values for autonomy

assessment criteria be given:

𝑋=

(

𝑥





,𝑥





,𝑥





…𝑥





)

(1)

where 𝑥



- the value of the fuzzy j-th criterion is

characterized by the membership function:

𝜇



𝑥



, 𝑥



∈𝐷



(2)

where 𝐷



is the domain (set of possible values) of the

j-th criterion.

If the value of the criterion cannot be determined,

this is complete uncertainty:

∀𝑥



∈𝐷



,𝜇



𝑥



=0.5

(3)

If the criterion is “not fuzzy”, then 𝜇



𝑥



=1, for

some 𝑥



=𝑥



∗

and ∀𝑥



≠ 𝑥



∗

𝜇



𝑥



=0.

Any other 𝜇



𝑥



 specified on the coordinate grid

with the required degree of detail are acceptable.

Let the permissible levels of autonomy for objects 𝑂



𝑖=1,𝑛



be given. The solution to the problem of

determining the level of autonomy is based on the

construction of expert rules for the product type:

If some subset 𝑥



,𝑥



,𝑥



…𝑥



takes certain fuzzy

values, then 𝑋∈𝑂



This rule does not clearly assign an object to one

level of autonomy. It only redistributes the object’s

membership function to the set of all numbers of

autonomy levels. In fuzzy form, this implication

looks like this:

𝑝

(

𝑖,𝑥





,𝑥





,𝑥





…𝑥





)

=min





〈

𝜆



(

𝑥

)

,𝜇



(

𝑥

)〉

(4)

where 𝜆



(

𝑥

)

belongs the value x to level 𝑂



for

variable j.

Next, we can move to a “not fuzzy” statement by

choosing the most possible level of autonomy:

𝑖

∗

=argmax



𝑝

(

𝑖,𝑥





,𝑥





,𝑥





…𝑥





)

(5)

If the values of the characteristics are listed in

products through a fuzzy disjunction, then formula

(4) will take the form:

𝑝

(

𝑖,𝑥





,𝑥





,𝑥





…𝑥





)

=max





〈

𝜆



(

𝑥

)

,𝜇



(𝑥)

〉

(6)

Multicriteria Analysis of the Robotic Systems Autonomy Using Fuzzy Calculations

917

This method of specifying fuzzy membership

requires specifying expert judgments at the time of

estimating RS.

In addition to fuzzy class affiliation, it is

necessary to determine the degree of interest in the

corresponding levels of autonomy. This assessment

depends on the current environment in which the RS

operates, as well as on the priorities of the decision-

maker.

If the criteria are independent in preference, then

the standard fuzzy weighted summation procedure is

applied.

The level of autonomy is calculated using the

standard weighted sum formula:

𝑃



∗

=𝑊



𝑋







(7)

where 𝑋



is the fuzzy estimate of the i-th RS

according to criterion k, and 𝑊



is the fuzzy

importance of criterion k, 𝑃



∗

is the final interest in

objects of class i.

The rules for summing and multiplying fuzzy

numbers are carried out based on the principle of

communication. Membership function corresponding

to the operation:

𝜇

(

𝑦

∗

)

=sup





,



,…



(



,



,…



)

∗





𝜇



(𝑦



)



(8)

where η is the operation that needs to be applied (in

the case of calculating 𝑊



𝑋



is the product, and for

calculating

∑

𝑊



𝑋







is the sum), 𝑦



are the values

to which the required operation is applied, 𝜇



(𝑦



) is

the membership function of fuzzy values, 𝜇

(

𝑦

∗

)

the membership function for the result of applying the

operation η. Θ is the intersection operation for

membership functions. In this work, this is min, but

there are other varieties of this operation. Let us

denote the membership function of the resulting fuzzy

weighted sum as 𝜑



∗

(𝑦).

To calculate the final level of autonomy, the clip

function is calculated for all possible levels, taking

into account their priorities, and their further fuzzy

combination:

𝜌

(

𝑦

)

=max



min

〈

𝑝

(

𝑖,𝑥



,𝑥





,𝑥



…𝑥



)

,𝜑



∗

(𝑦)

〉

(9)

If fuzzy estimates of autonomy levels are obtained

for several RSs that need to be compared, then a fuzzy

comparison procedure is constructed or

defuzzification is performed:

𝑦





𝑦𝜌

(

𝑦

)

𝑑𝑦



𝜌

(

𝑦

)

𝑑𝑦

(10)

This approach will allow:

• provide decision makers with tools for

formalizing qualitative judgments about the degree of

autonomy in tasks with a high dimension of the

criterion;

• allows further assessment of the degree of

autonomy in automatic mode, i.e., the RS ranking

process will occur quickly and, possibly, without the

involvement of the decision maker, but taking into

account his preferences;

• take into account preference dependencies

between the components of the vector criterion. As a

result, it is possible to identify and eliminate

situations where RS with an unacceptable level

according to one criterion receives a high integral

assessment at the expense of other criteria;

• ensure the distinguishability of RS in the case

when criterion scales are subjected to artificial

discretization in order to replace continuous scales

with point scores.

To the complex “autonomy space” proposed by

the ALFUS group, we add scalar indicators:

qualification (mandatory program - admission to

evaluation, comparison or competition) of a robot:

what does a qualified device have?

statement of the problem: division into classes:

which class of problems can be solved (with what

amount of a priori information);

quality of problem solution (consumed computing

resources, time, accuracy, reliability (response to an

unforeseen situation)).

In terms of developing methods and plans for

testing and confirming the levels of autonomy of

unmanned vehicles, we are preparing an approximate

program and methodology for testing technical vision

systems in the task of providing information support

for targeted movements of autonomous vehicles.

To set the mission of an unmanned vehicle with

subsequent decomposition and automatic translation

into functional tasks, we create lists of tasks - basic,

typical scenarios, or precedents for the use of

unmanned vehicles. In our work, we focus on ground

mobile robots (Sokolov, 2022). An example of use

cases for an autonomous mobile robot:

- surveillance of the area or object;

- reconnaissance of a given area;

- search for specific objects of interest, their

identification and precise localization;

- work with detected objects of interest.

The basic list of technological operations (TO) of

a robot is determined by the above precedents. TO

include:

- TO robot safety systems (preventing collisions

with obstacles and robot overturning);

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

918

- calculated TO (construction of trajectories,

localization of objects of interest, construction and

correction of the map);

- TO practice commands or information-motor

actions (ensuring the fulfillment of specified modes

of operation of the robot).

One of the most challenging issues in assessing

and making comparisons of unmanned vehicle

autonomy is defining or establishing a domain-

specific level of autonomy model for selected

programs. An equally complex problem is the

problem of a weighted assessment of the totality of

quantitative assessments in the space of “autonomy”.

Assessing the combination of mission complexity

and environmental complexity is subject to a high

degree of uncertainty and complexity, which

significantly limits the application of quantitative

methods for comparative analysis. Therefore, in this

part of autonomy assessments, the use of a well-

developed cognitive modeling apparatus is proposed.

Fuzzy cognitive maps are a way of representing real

dynamic systems in a form that corresponds to the

human perception of such processes.

3 CONCLUSIONS

The degree of autonomy depends on the tasks that ned

to be performed. Qualitatively, the scope defined by

a set of parameters within which the system can make

decisions and act independently to achieve its goals

can be called the degree of autonomy. Robots with

full autonomy are preferred in many applications,

such as space exploration, logistics, and others. As the

analysis shows, an objective and constructive

assessment of the degree of autonomy of RS requires

the serious collective efforts of the entire robotics

community.

The use of fuzzy logic, methods of multicriteria

analysis of alternatives, and decision-making theory

will allow us to take into account the vagueness of the

judgments of experts and decision-makers in

problems of comparing different RSs in terms of the

degree of autonomy and other criteria, including

economic and technical indicators. On this path, in

particular, lies the construction of ontologies for

subject areas - areas of application of RS. The use of

ontologies will allow customers and users to unify

coordinate units along the axes of the “autonomy

space”, objectively and quantitatively compare the

degrees of intellectual and information autonomy of

RS and RS designers to quickly assemble successful

solutions among themselves.

REFERENCES

IFR. (2015). Service Robots. http://www.ifr.org/service-

robots/, date of access 12.11.2023.

Kaysner, E., Raffо, D., Wunsch-Vincent, S. (2016).

Robotics: breakthrough technologies, innovations,

intellectual property. Foresight, 2 (2).

Nesmelov, P. A. (2022). Stages of robotics development:

selection criteria and their characteristics. Young

scientist, 19(414). pp. 26-28.

Ermolov, I.E. (2012). Expanding the functionality of mobile

technological robots by increasing their level of

autonomy using hierarchical integrated processing of

onboard data. Dissertation for the degree of Doctor of

Technical Sciences.

Murphy, R. (2020) Introduction to robotics with artificial

intelligence / Robin R. Murphy. (Intelligent robotics

and autonomous agents. Bradford's Book.) The Press of

the Massachusetts Institute of Technology. Cambridge,

Massachusetts, London, England.

SAE. (2023). Society of Automotive Engineers - American

Association of Automotive Engineers https://www.

sae.org/standards/j3016_202104, date of access

12.11.2023.

PROXIMA. (2023). https://www.gisproxima.ru/avtonomn

yye_bpla, date of access 12.11.2023.

ALFUS. (2004). Autonomy Levels for Unmanned Systems

Framework, Volume I: Terminology, Version 1.1,

Huang, H. Ed., NIST Special Publication 1011,

National Institute of Standards and Technology,

Gaithersburg, MD, September.

Huang, H., Messina, E., Albus, J. (2003). Autonomy Level

Specification for Intelligent Autonomous Vehicles:

Interim Progress Report. PerMIS Workshop,

Gaithersburg, MD.

Sokolov, S.M. (2022). Ontological approach in the creation

of robotic complexes with an increased degree of

autonomy 11 News of the SFU. Technical Sciences, l,

pp.42-59.

Sudakov, V., Zhukov A. (2023). Fuzzy Domination Graphs

in Decision Support Tasks. Mathematics 11(13): 2837.

Multicriteria Analysis of the Robotic Systems Autonomy Using Fuzzy Calculations

919