Subjective Markov Process with Fuzzy Aggregations
Eugene Kagan
1,2
, Alexander Rybalov
2
and Ronald Yager
3
1
Department of Industrial Engineering, Ariel University, Ariel, Israel
2
LAMBDA Lab, Tel-Aviv University, Tel-Aviv, Israel
3
Machine Intelligence Institute, Iona College, New York, NY, U.S.A.
Keywords: Fuzzy Logic, Markov Process, Decision-making, Subjective Reasoning.
Abstract: Dynamical models of autonomous systems usually follow general assumption about rationality of the systems
and their judgements. In particular, the systems acting under uncertainty are defined using probabilistic
methods with the reasoning based on minimization or maximization of the expected payoffs or rewards.
However, in the systems that deal with rare events or interact with human usually demonstrating irrational
behaviour correctness of the use of probability measures and of the utility functions is problematic. In order
to solve this problem, in the paper we suggest a Markov-like process that is based on a certain type of
possibility measures and uninorm and absorbing norm aggregators. Together these values and operators form
an algebraic structure that, on one hand, extends Boolean algebra and, on the other hand, operates on the unit
interval as arithmetic system. We demonstrate the basic properties of the suggested subjective Markov process
that go in parallel to the properties of usual Markov process, and stress formal differences between two
models. The actions of the suggested process are illustrated by the simple model of search that clarifies the
differences between Markov and subjective Markov processes and corresponding decision-making.
1 INTRODUCTION
Usually the models of autonomous systems acting
under uncertainties are based on the Markov
processes that define the evolution of the probabilities
of the system states. The decision-making in such
systems deals with the choice of the system’s
activities in each state.
In spite of a wide variety of methods and
algorithms used in such models, the starting point of
these probabilistic techniques is an assumption about
rationality of the systems and their judgements (Luce
and Raiffa, 1964; Raiffa, 1968). Consequently, the
reasoning in such systems is based on minimization
or maximization of the expected payoffs or rewards
(White, 1993).
However, in the models those consider the systems
with rare events or deal with the systems interacting
with humans, who usually demonstrate irrational
behaviour (Kahneman and Tversky, 1979),
application of probabilistic measures and
minimization/maximization criterions are rather
problematic. Subjective factors in such models
usually are considered on the base of certain utility
functions that represent preferences of the observer
(Friedman and Savage, 1948), and by extending usual
Markov processes (MP) up to partially observable or
hidden Markov processes (HMP) (Monahan, 1982;
Rabiner, 1989) or hierarchical HMP (HHMP) (Fine,
Singer and Tishbi, 1998). Such techniques allow
effective modelling of many types of particular
systems, but as direct successors of the Markov
decision processes (MDP) (White, 1993), these
techniques are also based on probabilistic methods
with no concern to the existence or correctness of the
required probabilistic measures.
In order to resolve these problems, in the paper we
suggest subjective MP (μP) that goes in parallel to
usual MP, but instead of probabilities is defined on a
certain type of possibility measures (Dubois and
Prade, 1988) that represent beliefs of the observer.
Together with the uninom (Yager and Rybalov, 1996)
and the absorbing norm (Batyrshin, Kaynak and
Rudas, 2002), these measures form an algebraic
structure that, on one hand, extends Boolean algebra
and, on the other hand, operates on the unit interval
as usual arithmetic system (Kagan, Rybalov,
Siegelmann and Yager, 2013). In addition to clear
definition of the process, such property allows
consideration of the μP in comparison with the MP
and stressing their similarities and differences.
386
Kagan, E., Rybalov, A. and Yager, R.
Subjective Markov Process with Fuzzy Aggregations.
DOI: 10.5220/0008915203860394
In Proceedings of the 12th International Conference on Agents and Artificial Intelligence (ICAART 2020) - Volume 2, pages 386-394
ISBN: 978-989-758-395-7; ISSN: 2184-433X
Copyright
c
2022 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
In particular, in the paper we present a basic
classification of the μP states that goes in parallel to
the classification of the MP states and demonstrates
formal differences between two models. The actions
of the μP are illustrated by the simple model of search
(Pollock, 1970; Kagan and Ben-Gal, 2013) that
clarifies the differences between μP and MP and
corresponding decision-making.
2 UNINORM AND ABSORBING
NORM
Let us briefly recall the definitions of the uninorm
(Yager and Rybalov, 1996) and absorbing norm
(Batyrshin, Kaynak and Rudas, 2002).
Consider the truth values that, in contrast to the
Boolean logic, are drawn from the interval
0,1
. In
the theory of fuzzy sets and in fuzzy logic (Bellman
and Giertz, 1973; Zade, 1965), such truth values are
associated with the “grades of membership” 𝜇
𝑑
)
of
the points d of some domain 𝐷 to the set 𝐴⊂𝐷. The
function 𝜇
:𝐷→
0,1
is, respectively, called the
membership function.
For the non-binary truth values are defined the
multivalued “and” ⋏, “or” ⋎ and “not” ~ operators,
such that for any 𝑥,𝑦 ∈
0,1
also
𝑥⋏𝑦
)
∈
0,1
,
𝑥⋎𝑦
)
∈
0,1
and
~𝑥
)
∈
0,1
. These operators
go in parallel to the Boolean “and” ∧, “or” ∨ and
“not” ¬ operators and coincide with them such that
for binary truth values 𝑥,𝑦 ∈
0,1
it holds true that
𝑥⋏𝑦=𝑥∧𝑦, 𝑥⋎𝑦=𝑥∨𝑦 and ~𝑥=¬𝑥. In the
most applications, they are also associated with
statistical triangular norms (
Klement, Mesiar and Pap,
2000)
(conjunction ⋏ with 𝑡-norm and disjunction ⋎
with 𝑡-conorm) or are defined arithmetically (Dubois
and Prade, 1985).
Later (Yager and Rybalov, 1996), conjunction ⋏
and disjunction ⋎ operators were united into a single
uninorm aggregator ⊕
:
0,1
×
0,1
→
0,1
with
neutral element 𝜃∈
0,1
such that for θ=1 it is
𝑥⊕
𝑦=𝑥⋏𝑦 and for 𝜃=0 it is 𝑥⊕
𝑦=𝑥⋎𝑦.
In parallel, there was introduced (Batyrshin,
Kaynak and Rudas, 2002) an absorbing norm
aggregator ⊗
:
0,1
×
0,1
→
0,1
with
absorbing element 𝜗∈
0,1
; this aggregator extends
the Boolean 𝑛𝑜𝑡 xor operator.
Both uninorm ⊕
and absorbing norm ⊗
are the
functions that specify aggregation of the variables
𝑥,𝑦 ∈
0,1
resulting in
𝑥⊕
𝑦
)
∈
0,1
and
𝑥⊗
𝑦
)
∈
0,1
, and for all 𝑥,𝑦,𝑧 ∈
0,1
they
meet the commutative and associative properties:
𝑥⊕
𝑦=𝑦⊕
𝑥,
(1a)
𝑥⊗
𝑦=𝑦⊗
𝑥,
(1b)
𝑥⊕
𝑦
)
⊕
𝑧=𝑥⊕
𝑦⊕
𝑧
)
and
(2a)
𝑥⊗
𝑦
)
⊗
𝑧=𝑥⊗
𝑦⊗
𝑧
)
;
(2b)
for the uninorm it also holds true that
𝑥≤𝑦 implies 𝑥⊕
𝑧≤𝑦⊕
𝑧.
(3)
Neutral 𝜃 and absorbing 𝜗 elements play a role of
zero for their operators that is
𝜃⊕
𝑥=𝑥 and
(4a)
𝜗⊗
𝑥=𝜗.
(4b)
Interpretation of the aggregators ⊕
and ⊗
is
the following (Rybalov and Kagan, 2017). The
uninorm is an operator such that its truth value is
defined by the extent (called also true by extent), to
which both its arguments are true, and the absorbing
norm is an extension of the not xor comparison and
specifies the grade of similarity between its
arguments. In the other situations, these aggregators
can be considered as parameterized logical operators
and applied for design of logical schemes (Rybalov,
Kagan and Yager, 2012), or even as a tool for
modelling operations in quantum information theory
(Rybalov, Kagan, Rapoport and Ben-Gal, 2014).
Together with formal properties of the
aggregators ⊕
and ⊗
, it was also proven (Fodor,
Yager and Rybalov, 1997; Fodor, Rudas and Bede,
2004) that for any 𝑥,𝑦 ∈
0,1
there exist functions 𝑢
and 𝑣 called generator functions such that
𝑥⊕
𝑦=𝑢
𝑢
𝑥
)
+𝑢
𝑦
)
, (5a)
𝑥⊗
𝑦=𝑣
𝑣
𝑥
)
∙𝑣
𝑦
)
. (5b)
while the inverse functions 𝑢
and 𝑣
considered
on the open interval
0,1
)
are probability
distributions (Kagan, Rybalov, Siegelmann and
Yager, 2013). Such equivalence between inverse
generator functions and probability distributions
demonstrates deep relation between probabilistic and
fuzzy logics (Kagan, Rybalov, Siegelmann and
Yager, 2013; Kagan, Rybalov and Yager, 2014) that,
however, requires additional considerations.
The truth values from the interval
0,1
together
with the uninorm ⊕
and absorbing norm ⊗
aggregators form an algebraic structure. In the next
section we define this structure.
Subjective Markov Process with Fuzzy Aggregations
387
3 ALGEBRAIC STRUTURE WITH
UNINORM AND ABSORBING
NORM
Let ⊕
be a uninorm with the neutral element 𝜃 and
⊗
be an absorbing norm with the absorbing element
𝜗. As operators on the interval
0,1
, uninorm ⊕
defines monoid ℳ
⨁
=
〈
0,1
,⊕
,𝜃
〉
with a unit 𝜃,
and absorbing norm ⊗
defines monoid ℳ
⊗
=
〈
0,1
,⊗
,𝜗
〉
with a unit 𝜗.
The algebraic structure 𝒜=
〈
0,1
,⊕
,⊗
〉
on
the interval
0,1
with uninorm ⊕
and absorbing
norm ⊗
aggregators is defined as a triple that joins
monoids ℳ
⨁
and ℳ
⊗
. It is clear that this structure
extends Boolean algebra 𝐵=
〈
0,1
,∧,∨
〉
defined
for the operators ∧ and ∨, and to its multivalued
version ℬ=
〈
0,1
,⋏,⋎
〉
defined for the 𝑡-norm ⋏
and 𝑡-conorm ⋎ and acts both as a multivalued logical
system and as an arithmetic system on the interval
0,1
.
The basic properties of the structure 𝒜 are the
following.
− if 𝑢
𝑥
)
=𝑣
𝑥
)
for all 𝑥∈
0,1
, then 𝜃=
𝜗;
− the value λ=𝑣
1
)
is an identity element
of the absorbing norm that is λ⊗
𝑥=𝑥; below this
value will be denoted by 𝕀
⊗
.
Moreover 0 (Fodor, Rudas and Bede, 2004):
− if 𝜃=𝜗, then absorbing norm is
distributive with respect to the uninorm, that is
𝑥⊕
𝑦
)
⊗
𝑧=
𝑥⊗
𝑧
)
⊕
𝑦⊗
𝑧
)
;
(6)
− for any 𝑥∈
0,1
there exists an opposite
element ⊖
𝑥=𝑢
−𝑢
𝑥
)
∈
0,1
such that
𝑥⊕
⊖
𝑥
)
=𝑥⊖
𝑥=𝜃;
(7)
− for any 𝑥∈
0,1
, 𝑥≠𝜗, there exists an
inverse element 𝜆⊘
𝑥=𝑣
1𝑢
𝑥
)⁄)
∈
0,1
)
such that
𝑥⊗
λ
⊘
𝑥
)
=𝑥⊘
𝑥=
λ
.
(8)
In addition, notice that (Kagan, Rybalov,
Siegelmann and Yager, 2013)
− if 𝑢
𝑥
)
=𝑣
𝑥
)
, 𝑥∈
0,1
and so 𝜃=𝜗,
then the structure 𝒜 is a commutative ring
isomorphic to the ring of real numbers; otherwise, the
structure 𝒜 is a non-distributive algebra such that
𝑥⊕
𝑦
)
⊗
𝑧≠
𝑥⊗
𝑧
)
⊕
𝑦⊗
𝑧
)
.
(9)
In the other words, the structure 𝒜=
〈
0,1
,⊕
,⊗
〉
defines formal algebra on the interval
0,1
, where the aggregator ⊕
is considered as an
operation of summation and the aggregator ⊗
– as
an operation of multiplication. In addition to these
operations, there are obviously defined the operators
of subtraction ⊖
and of division ⊘
such that for
any 𝑥,𝑦 ∈
0,1
are
𝑥⊖𝜃𝑦=𝑢−1𝑢𝑥−𝑢𝑦,
(10)
and
𝑥⊘𝜗𝑦=𝑣−1𝑣𝑥/𝑣𝑦,
(11)
In the further considerations, we will need the
following properties of the uninorn and absorbing
norm:
uninorm ⊕
:
− if 𝑥,𝑦 > 𝜃 then 𝑥⊕
𝑦>𝜃;
− if 𝑥,𝑦 < 𝜃 then 𝑥⊕
𝑦<𝜃;
− if 𝑥>𝜃 and 𝑦<𝜃 and
|
𝑥−𝜃
|
>
|
𝑦−𝜃
|
then 𝑥⊕
𝑦>𝜃;
absorbing norm ⊗
:
− if 𝑥,𝑦 > 𝜗 or 𝑥,𝑦 < 𝜗 then 𝑥⊗
𝑦>𝜗;
− if 𝑥>𝜗 and 𝑦<𝜗 then 𝑥⊗
𝑦<𝜗.
In order to prove these properties it is enough to
exhibit a continuous monotonically increasing
function 𝜏
:
0,1
→
−1,1
with parameter 𝜐 (that
stands for 𝜃 in the uninorm ⊕
and for 𝜗 in the
absorbing norm ⊗
) such that 𝜏
0
)
=−1, 𝜏
𝜐
)
=
0 and 𝜏
1
)
=1. The simplest example of the
required function 𝜏
is a partially linear function
𝜏
𝑥
)
=
𝑥− 1, 𝑥<𝜐,
1−
𝑥+
, 𝑥>𝜐,
(12)
and such that 𝜏
𝑥
)
=−1 when 𝜐=0 and 𝜏
𝑥
)
=
1 when 𝜐=1.
Then aggregation using uninorm and absorbing
norm are equivalent to the normalized summation and
multiplication in the interval
−1,1
, for which the
required properties hold.
Notice that since the properties of aggregation
using uninorm and absorbing norm are equivalent to
the summation and multiplication in the interval
−1,1
, operations using these norms can be
considered as a multivalued extension of the
operations of the three-valued logic.
Finally, in the further considerations we need the
following values (Kagan, Rybalov and Ziv, 2016;
Kagan, Rybalov and Yager, 2018)
𝕆
⨁
=𝑢
−1
)
, 𝕀
⨁
=𝑢
1
)
,
(13a)
𝕆
⊗
=𝑣
−1
)
, 𝕀
⊗
=𝑣
1
)
.
(13b)
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
388
The values 𝕆
⨁
and 𝕆
⊗
are called subjective false
and the values 𝕀
⨁
and 𝕀
⊗
are called subjective true
(both with respect to ⊕
and ⊗
). These values are
certainly differ from neutral element 𝜃=𝑢
0
)
and
absorbing element 𝜗=𝑣
0
)
. Moreover, from the
properties of the generator functions for both
aggregators it immediately follows that
0<𝕆
⨁
<𝜃<𝕀
⨁
<1,
(14a)
0<𝕆
⊗
<𝜗<𝕀
⊗
<1
.
(14b)
where 0 and 1 represent Boolean false and true values
that are limiting values for subjective false and
subjective true, respectively.
Using the presented properties of algebra 𝒜 in the
next section we define the Markov-like process called
subjective Markov process.
4 Markov AND SUBJECTIVE
Markov PROCESSES
We define subjective Markov process (μP) as a
Markov process (MP) in algebra 𝒜. In order to
demonstrate similarities and differences between μP
and MP, we start with recalling the definition of MP
and then define the μP in parallel to MP. Since we are
interested in decision-making, we will consider only
the discrete time processes with finite number of
states that are the Markov chains.
4.1 Markov Process
Let 𝑆=
𝑠
,𝑠
,…,𝑠
be a finite set of some abstract
states, and consider a system that in time 𝑡 can be in
one of the states from this set such that an exact state
𝑠
𝑡
)
∈𝑆 is unknown. In order to handle this
uncertainty, assume that for the unknown system state
𝑠
𝑡
)
at time 𝑡 and each state 𝑠
from the indicated set
𝑆 of abstract states there is defined the probability
𝑝
𝑡
)
=𝑃𝑟
𝑠
𝑡
)
=𝑠
(15)
that the state 𝑠
𝑡
)
is equal to the state 𝑠
∈𝑆, 𝑖=
1,2,…,𝑛.
The dynamics of the system is defined using
conditional probabilities 𝜌
, 𝑗,𝑘=1,2,…,𝑛, such
that for each pair 𝑠
,𝑠
∈𝑆×𝑆 of abstract states
probability 𝜌
represents the chance of transition
from the state 𝑠
to the state 𝑠
. In the other words, if
it is known that at time t the system is in the state
𝑠
𝑡
)
=𝑠
, then the probability that at the next time
𝑡+1 it will be in the state 𝑠
𝑡+1
)
=𝑠
is
𝜌
=𝑃𝑟𝑠
𝑡+1
)
=𝑠
|𝑠
𝑡
)
=𝑠
.
(16)
Starting from the initial state probabilities 𝑝
0
)
,
𝑖=1,2,…,𝑛, defined at time 𝑡=0, evolution of the
system is formally defined by the product of the
probabilities vector 𝑝
𝑡
)
=𝑝
𝑡
)
,𝑝
𝑡
)
,…,𝑝
𝑡
)
and the transition matrix 𝜌=𝜌
×
𝑝
𝑡+1
)
=𝑝
𝑡
)
∙𝜌=𝑝
0
)
∙𝜌
.
(17)
The resulting state probabilities form a basis for
making decision about the action that should be
conducted at time 𝑡+1 and about possible rewards at
this time. Repetition of such multiplication allows
prediction of the system’s state for some future time
and correction of the decisions according to the
predicted future rewards.
However, as indicated above, definition of the
state probabilities is problematic; even at the initial
time it requires consideration of internal and external
parameters of the system that usually are not
available. Correct definition of the transition
probabilities, in its turn, requires deep analysis of the
system and its behaviour. But if such analysis was
already conducted, then the behaviour of the system
is known and its probabilistic modelling becomes
meaningless. Additional problem rises because of the
assumption about rationality of the system’s
behaviour since usually the real-world systems
interacting with humans follow irrational judgements
based on the subjective factors.
In order to resolve these problems, we suggest to
use a μP that is a Markov process in algebra 𝒜.
4.2 Subjective Markov Process
As above, let 𝑆=
𝑠
,𝑠
,…,𝑠
be a finite set of
abstract states, and assume that at time 𝑡 the system is
in the state 𝑠
𝑡
)
∈𝑆 that is unknown to the observer.
However, for each abstract state 𝑠
∈𝑆, 𝑖=1,2,…,𝑛,
the observer can ask the question: “Is it true that at
time 𝑡 the system is in the state 𝑠
?” and can conclude
that the truth level of the answer: “At time t the
system is in the state 𝑠
” is 𝜇
𝑠
,𝑡
)
=𝜇
𝑡
)
∈
0,1
.
For convenience, we consider this truth level as an
observer’s belief that state 𝑠
𝑡
)
is 𝑠
and denote it as
𝜇
𝑡
)
=𝐵𝑒𝑙
𝑠
𝑡
)
=𝑠
.
(18)
Boundary value 𝜇
𝑡
)
=0 means that the
statement “at time t the system is in the state 𝑠
” is
false and boundary value 𝜇
𝑡
)
=1 means that this
statement is true. In the other words, belief 𝜇
𝑡
)
=1
is interpreted as an exact knowledge about the
occurrence of the event and belief 𝜇
𝑡
)
=0 is
interpreted as an exact knowledge about non-
Subjective Markov Process with Fuzzy Aggregations
389
occurrence of the event. The intermediate truth values
represent the grades of the observer’s belief that at
time t the system is in the state 𝑠
, and belief 𝜇
𝑡
)
=
0.5 means an absence of any knowledge whether the
event occurred or not.
Denote by 𝜔
∈
0,1
, 𝑗,𝑘=1,2,…,𝑛, the truth
value, which for each pair 𝑠
,𝑠
∈𝑆×𝑆 of abstract
states represents the belief that from the state 𝑠
the
system transits to the state 𝑠
. In the other words, if
the observer asks the question: “Is it true that the
system transits from the state 𝑠
to the state 𝑠
?”, then
𝜔
is the truth level of the answer: “The system
transits from the state 𝑠
to the state 𝑠
”. In the other
interpretation the value 𝜔
can be considered as a
possibility of transition from the state 𝑠
to the state
𝑠
. Such interpretation allows application of 𝜔
in
the analysis of coincidentia oppositorium (Rybalov
and Kagan, 2017; Rybalov and Kagan, 2018) and
consider it as a truth value of the statement that the
system is both in the state 𝑠
and in the state 𝑠
(that
happens when the system is transiting from 𝑠
to 𝑠
:
at some moment it is both in 𝑠
and in 𝑠
, or neither
in 𝑠
nor in 𝑠
).
Dynamics of the system is defined in the algebra
𝒜 as follows. Let 𝜇
𝑡
)
=𝜇
𝑡
)
,𝜇
𝑡
)
,…,𝜇
𝑡
)
be
a vector of the states’ 𝑠
∈𝑆 truth values, 𝑖=
1,2,…,𝑛, and by 𝜔=𝜔
×
a matrix of
transition possibilities 𝜔
, 𝑗,𝑘=1,2,…,𝑛. Then, in
parallel to MP, the update of the state truth values is
specified as
𝜇
𝑡+1
)
=𝜇
𝑡
)
⊗
𝜔.
(19)
where the product of vector 𝜇
𝑡
)
and matrix ω in the
algebra 𝒜 is defined by application of the aggregators
⊕
and ⊗
following usual “the row to the column”
rule: for each 𝑖=1,2,…,𝑛
𝜇
𝑡+1
)
=
𝜇
𝑡
)
⊗
𝜔
)
⊕
𝜇
𝑡
)
⊗
𝜔
)
⊕
…
⊕
𝜇
𝑡
)
⊗
𝜔
)
.
(20)
This process considers behaviour of the system
from the observer’s point of view, and by changing
the values of neutral 𝜃 and absorbing 𝜗 elements the
observer’s beliefs and preferences can be tuned. For
decision-making, the values 𝜇
𝑡
)
, 𝑖=1,2,…,𝑛, can
be used either directly (such as in the example in
section 4) or can be transformed into the states
probabilities using the means of possibility theory
(Dubois and Prade, 1985) or of probabilistic logic
(Nilsson, 1986; Kagan, Rybalov and Yager, 2014).
4.3 Basic Types of the States in
Subjective Markov Process
Using the properties of algebra 𝒜 we can consider the
basic types of the μP states. In parallel to usual MP,
the states of μP are classified according to the
corresponding beliefs and transition possibilities that
allow prediction of possible states and beliefs of the
observer.
Let 𝑆=
𝑠
,𝑠
,…,𝑠
be a set of states, and
denote by 𝑠
𝑡
)
∈𝑆 the state of the system at time 𝑡.
Then, in parallel to the probabilities that characterize
MP, for μP we introduce the following beliefs and
possibilities:
− the first passage belief
𝛽
)
=𝐵𝑒𝑙
𝑠
𝑡+𝑙
)
=𝑠
,
𝑠
𝑡+𝑚
)
≠𝑠
,0<𝑚<𝑙
|𝑠
𝑡
)
=𝑠
(21)
is a belief that if at time 𝑡 the system is in the state 𝑠
,
then at first time it will be in the state 𝑠
in l steps,
𝑖,𝑗= 1,2,…,𝑛.
− the 𝑙-step transition belief
𝜓
)
=𝐵𝑒𝑙𝑠
𝑡+𝑙
)
=𝑠
| 𝑠
𝑡
)
=𝑠
(22)
is a belief that if at time 𝑡 the system is in the state 𝑠
,
then it will reach the state 𝑠
, 𝑗,𝑘= 1,2,…,𝑛, in
exactly l steps.
It is clear that by definition, 1-step transition
belief is equivalent to the transition possibility that is
𝜓
)
=𝜔
=𝐵𝑒𝑙𝑠
𝑡+1
)
=𝑠
| 𝑠
𝑡
)
=
𝑠
.
(23)
Following usual notation, denote by
𝛽
=⊕
𝛽
)
(24)
the belief that starting from the state 𝑠
in some time
the system will reach the state 𝑠
. Then, we say that
the state 𝑠
− is believed to be persistent (or recurrent) if
𝕀
⨁
≤𝛽
≤1,
− is believed to be transient if 𝕆
⨁
<𝛽
<
𝕀
⨁
, and
− is believed to be separate (non-persistent
and non-transient or non-recurrent and non-transient)
if 0≤𝛽
≤𝕆
⨁
.
It means that the state is persistent if the observer
highly believes that the system will sooner or later
return to this state, is transient if the observer’s belief
about return to this state is low, and is separate if the
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
390
observer highly beliefs that the system will never
return to this state. In addition, notice that, in contrast
to MP, in μP belief 𝛽
can change its value such that
persistent state will become separate state and
backwards. Such state is called oscillating state and
represents the observer’s hesitations regarding this
state. Finally, the state 𝑠
− is believed to be periodic (with period 𝑇) if
these exists an integer number 𝑇 such that if 𝑙=𝑘𝑇,
𝑘=1,2,3,…, then 𝜓
)
≠𝜗, otherwise 𝜓
)
=𝜗.
Formula “the state is believed to be…” is used for
avoiding unambiguousness and stresses that the
values 𝛽
)
, 𝛽
and 𝜓
)
are beliefs and makes sense
only in the context of observer’s knowledge; also it
allows distinguishing the values and states used in μP
from the probabilities and corresponding states used
in MP.
Relation between the first passage belief 𝛽
)
and
the 𝑙-steps transition belief 𝜓
)
is similar to the
relation between first passage and 𝑙 steps transition
probabilities in MP and is defined as follows:
𝜓
)
=⊕
𝛽
)
⊗
𝜓
)
,
(25)
where 𝛽
)
=𝕆
⊗
(it is believed that in zero steps the
system will not move from the state 𝑖 to the state 𝑗,
𝑗≠𝑖), 𝜓
)
=𝕀
⊗
(it is believed that during time unit
the system will stay in its current state), 𝜓
)
=𝕆
⊗
(it is believed that in zero steps the system will not
move from the state 𝑖 to the state 𝑗, 𝑗≠𝑖), and 𝛽
)
=
𝜔
(belief that starting from state 𝑖 the system will
reach state 𝑗 at first time in one step is equivalent to
the possibility of transition from state 𝑖 to state 𝑗),
𝑖,𝑗,𝑘=1,2, …,𝑛.
Relation between first passage belief 𝛽
)
and 𝑙-
steps transition belief 𝜓
)
for persistent, transient and
separate states in μP is the following. The state 𝑠
− is believed to be persistent (or recurrent) if
and only if there are no any hesitations about the
possibility of return to the state, that is
⊕
𝜓
)
=1;
(26)
− is believed to be transient if and only if
there exists some possibility of return but this
possibility is not exact, that is
0<⊕
𝜓
)
<1;
(27)
− is believed to be separate (non-persistent
and non-transient or non-recurrent and non-transient)
if and only if it is exactly known that there is no any
possibility to return to this state, that is
⊕
𝜓
)
=0.
(28)
The proofs of these propositions are based on
direct application of the monotonicity of the uninorn
and of the convergence of its results to 0 or 1 for the
terms less than or greater than 𝜃, respectively. The
other way to prove these propositions is based on the
application of the function 𝜏
and its reverse that
allows consideration of the propositions in the
interval
−1,1
with usual arithmetic operations
(together with normalization of sum).
The formulated properties of the states in μP go
in parallel to the properties of the states proven for
MP (Feller, 1970). However, it is seen that both the
meaning and formal characteristics of these states are
different.
In order to stress this difference and to illustrate
the actions of μP in the next section we consider the
simple model of search (Pollock, 1970; Kagan and
Ben-Gal, 2013) using both models.
5 SIMPLE MODEL OF SEARCH
WITH MP AND μP
We clarify the actions of μP and the difference
between MP and μP by running example of classical
Pollock model of search. In this model, the target
moves between two boxes and the observer should
catch the target by checking one of the boxes: if the
target is in the chosen box, then the search terminates
and if not, then the search continues (Pollock, 1970;
Kagan and Ben-Gal, 2013).
Below we do not address the optimization issues
and do not compare MP and μP from this point of
view; our goal is only to demonstrate that μP provides
additional information about the considered system
and can lead to decisions that differ from the
decisions led by MP.
Assume that the set 𝑆=
𝑠
,𝑠
includes only two
states that are associated with the boxes. At each time
𝑡=0,1,2,…, the target can be in one of the boxes 𝑠
and 𝑠
with the probabilities (𝑖=1,2)
𝑝
𝑡
)
=𝑃𝑟
𝑠
𝑡
)
=𝑠
,
(29)
𝑝
𝑡
)
=1−𝑝
𝑡
)
,
these probabilities are called location probabilities.
The chances of movements between the boxes 𝑠
and 𝑠
are defined by the transition probabilities
(𝑗,𝑘=1,2)
Subjective Markov Process with Fuzzy Aggregations
391
𝜌
=𝑃𝑟𝑠
𝑡+1
)
=𝑠
|𝑠
𝑡
)
=𝑠
,
(30)
𝜌
+𝜌
=1, 𝜌
+𝜌
=1,
where the probabilities 𝜌
and 𝜌
represent the
chances that the target stays in its current box 1 or 2,
respectively.
Dynamics of the target is governed by MP as
follows
𝑝
𝑡+1
)
,𝑝
𝑡+1
)
=
𝑝
𝑡
)
,𝑝
𝑡
)
∙
𝜌
𝜌
𝜌
𝜌
.
(31)
On the base of probabilities 𝑝
𝑡+1
)
and
𝑝
𝑡+1
)
the searcher decides which box should be
checked at time 𝑡+1: if the target is found, then the
search is terminated, and if it is not found, then the
location probabilities are updated (for the checked
box it is set to zero and to the other box – to one) and
the search continues. The one-step decision is clear
and prescribes to choose the box with maximal
location probability; however, since the unsuccessful
decision leads to the probabilities update, the long-
term decision-making is rather nontrivial problem.
Now let us describe the process using the
suggested μP. For the same set 𝑆=
𝑠
,𝑠
of states,
let
𝜇
𝑡
)
=𝐵𝑒𝑙
𝑠
𝑡
)
=𝑠
, i=1,2,
(32)
be truth values that represent the beliefs of the
searcher that the target is located in the boxes 𝑠
and
𝑠
; for briefness we call these values the location
beliefs. The beliefs that the target can move from one
box to another are represented by the transition
possibilities (𝑗,𝑘=1,2)
𝜔
=𝐵𝑒𝑙𝑠
𝑡+1
)
=𝑠
|𝑠
𝑡
)
=
𝑠
,
(33)
where the values 𝜔
and 𝜔
represent the beliefs
that the target stays in its current box 1 or 2,
respectively.
Then, the system is described directly from the
searcher’s point of view as a search process that is
governed by μP such that
𝜇
𝑡+1
)
,𝜇
𝑡+1
)
=
𝜇
𝑡
)
,𝜇
𝑡
)
⊗
𝜔
𝜔
𝜔
𝜔
.
(34)
Similar to MP, in the obtained μP the searcher
decides which box should be checked at time 𝑡+1
following location beliefs 𝜇
𝑡+1
)
and 𝜇
𝑡+1
)
.
As indicated above, it can be done either directly or
by the means of possibility theory or of the
probabilistic logic.
However, the values of the beliefs obtained in the
μP, in general, differ from the values of the location
probabilities and lead to the decision that differs from
the decision made in the MP. The long-term decision-
making also differs; since the beliefs represent the
observer’s subjective point of view, they do not
updated and the next step beliefs are calculated using
the current beliefs with no concern to the observation
result.
In order to illustrate the difference between MP
and μP, we implement the distributive version of
algebra 𝒜 with the aggregators ⊕
and ⊗
defined
by equations (5) with equivalent generator functions
𝑢=𝑣=𝑤, where 𝑤 is the inverse of Cauchy
distribution
𝑤
𝑥
)
=𝑚+𝛼tan𝜋
𝑥−
. (35)
Then
𝑤
𝜉
)
=
+
arctan
, (36)
where 𝜉,𝑚 ∈
−∞,∞
)
and 𝛼>0. From the
requirement 𝑤
𝜃
)
=𝑤
𝜗
)
=0 it also follows that
𝜃=𝜗=
−
arctan
, (37)
In addition, we assume that parameters of the
distribution are 𝑚=0 and 𝑎=1; thus 𝜃=𝜗=
.
Consider the first step of the process. For
convenience, we assume that the initial values of the
location probabilities and location beliefs are equal
and are
𝜇
0
)
=𝑝
0
)
=
0.8,0.2
)
,
(38)
Direct calculations result in the following. Let
transition matrices (transition beliefs and transition
probabilities) be
𝜔=𝜌=
0.4 0.6
0.6 0.4
. (39)
Then
𝑝
1
)
=
0.44,0.56
)
,
(40)
𝜇
1
)
=
0.27,0.73
)
.
It is seen that the probability 𝑝
1
)
=0.44 that
the target will be at the first box is smaller than the
probability 𝑝
1
)
=0.56 that it will be in the second
box and the same is true for the beliefs that are
𝜇
1
)
=0.27 and 𝜇
1
)
=0.73. Then, in both cases
the searcher should check box 2. However, the
difference between the probabilities 𝑝
1
)
and 𝑝
1
)
is essentially smaller than the difference between the
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
392
beliefs 𝜇
1
)
and 𝜇
1
)
that is the belief that the
target is in the box 2 is greater than the probability
that it is there.
The further iterations of the processes
demonstrate that in the MP the relation p
t
)
<p
t
)
remains until reaching the steady state p
t
)
=
p
t
)
=0.5. In the μP, in contrast, each iteration
changes the relation between 𝜇
𝑡
)
and 𝜇
𝑡
)
such
that 𝜇
0
)
<𝜇
0
)
, 𝜇
1
)
>𝜇
1
)
, 𝜇
2
)
<𝜇
2
)
,
𝜇
3
)
>𝜇
3
)
, … up to reaching the steady state
𝜇
𝑡
)
=𝜇
𝑡
)
=0.5.
Thus, in the MP the searcher should all times
check box 2, while in the μP the searcher should
change the checked box at each step.
Now assume that transition matrices (transition
beliefs and transition probabilities) are
𝜔=𝜌=
0.4 0.6
0.3 0.7
. (41)
Then the picture essentially changes and
𝑝
1
)
=
0.38,0.62
)
,
(42)
𝜇
1
)
=
0.66,0.34
)
.
Here location probability 𝑝
1
)
=0.38 for the first
box is again smaller than the location probability
𝑝
1
)
=0.62 for the second box, but the belief
𝜇
1
)
=0.66 that the target will be in the first box is
greater that the belief 𝜇
1
)
=0.34 that it will be in
the second box. Consequently, in the first case the
searcher should check box 2, but in the second case –
box 1.
In the further iterations both relations 𝑝
𝑡
)
<
𝑝
𝑡
)
and 𝜇
𝑡
)
>𝜇
𝑡
)
remain until reaching the
steady states 𝑝
𝑡
)
=
, 𝑝
𝑡
)
=
and 𝜇
𝑡
)
=
𝜇
𝑡
)
=0.5. This state in the MP prescribes to
continue checking box 2 and in the μP it prescribes to
choose the box by random.
In addition notice that in both cases of transition
matrices in the MP the steady state is reached faster
than in the μP, thus the μP provides more information
for making decision about the box for check.
It is clear that the considered model is the simplest
one and is used only as an example. However, even
such simple model stresses the difference between the
MP and μP and demonstrates that the decisions made
in μP can differ from the decisions made in MP.
More complex processes and decisions are
obtained by the use of non-distributive version of the
algebra 𝒜, where in the aggregators ⊕
and ⊗
the
elements 𝜃 and 𝜗 differ or even differ generation
functions 𝑢 and 𝑣; but these issues we remain for
further research.
6 CONCLUSIONS
The suggested subjective Markov process (μP) goes
in parallel to the usual Markov process (MP), but, in
contrast to MP, it acts in the recently constructed
algebra 𝒜 that implements uninorm and absorbing
norm aggregators and combines logical and
arithmetical operations.
The values, with which μP deals, are considered
as observer’s beliefs about the system’s states and can
be associated with the grades of membership or with
possibilities of the system to be in certain states. Such
definition allows to use the suggested μP instead or in
parallel to the MP for analysis of the systems that
include rare events or follow subjective irrational
decisions.
For the suggested process, we considered the
basic types of the states with respect to the transition
beliefs that specify the possibilities of transitions
among the states. The essential role in this
consideration play recently introduced concepts of
subjective false and subjective true that allow precise
and meaningful classification of the states.
The difference between the suggested μP and
usual MP is illustrated by running example of the
Pollock model of search. It was shown that even in
such simple model (with maximization of the
probabilities of finding the target) μP provides
additional information and leads to the decisions that
can differ from the decisions prescribed by MP.
REFERENCES
Batyrshin I, Kaynak O, Rudas I., 2002. Fuzzy modeling
based on generalized conjunction operations. IEEE
Trans. Fuzzy Syst. 10(5), 678-683.
Bellman R, Giertz M., 1973. On the analytic formalism of
the theory of fuzzy sets. Inform. Sci., 5, 149-156.
Dubois D, Prade H. 1985. A Review of Fuzzy Set
Aggregation Connectives. Inform. Sci., 36, 85-121.
Dubois D, Prade H. 1988. Possibility Theory. Plenum, New
York.
Feller W. 1970. An Introduction to Probability Theory and
its Applications. 3rd edition. Vol I., John Willey &
Sons, New York, etc.
Fine S, Singer Y, Tishbi N. 1998. The hierarchical hidden
Markov model: analysis and applications. Machine
Learning, 32, 41-62.
Fodor J, Rudas I, Bede B. 2004. Uninorms and absorbing
norms with applications to image processing. In Proc.
SISY.
Fodor J, Yager R, Rybalov A. 1997. Structure of uninorms.
Int. J. Uncert., Fuzziness and Knowl. Based Syst., 5,
411-427.
Subjective Markov Process with Fuzzy Aggregations
393
Friedman M., Savage L.J. 1948. The utility analysis of
choices involving risk. J. Political Economy, 56(4),
279-304.
Kagan E., Ben-Gal I., 2013. Probabilistic Search for
Tracking Targets: Theory and Modern Applications.
John Wiley &Sons, Chichester, UK.
Kagan E, Rybalov A, Siegelmann H, Yager R. 2013.
Probability generated aggregators. Int. J. Intel. Syst.,
28, 709-727.
Kagan E, Rybalov A, Yager R., 2014. Multi-valued logic
based on probability-generated aggregators. In Proc.
IEEE Conv. EEEI.
Kagan E, Rybalov A, Yager R., 2018. Complex-valued
logic for neural networks. In Proc IEEE Conf ICSEE.
Kagan E, Rybalov A, Ziv H., 2016. Uninorm-based neural
network and its application for control of mobile robots.
In Proc IEEE Conf ICSEE.
Kahneman D., Tversky A. 1979. Prospect theory: an
analysis of decision under risk. Econometrica, 47(2),
263-291.
Klement E, Mesiar R, Pap E., 2000. Triangular Norms.
Kluwer Academic Publishers, Dordrecht.
Luce R. D., Raiffa H., 1964. Games and Decisions.
Introduction and Critical Survey. John Wiley & Sons,
New York.
Monahan G. E. A, 1982. Survey of Partially Observable
Markov Decision Processes: Theory, Models, and
Algorithms. Management Science, 28(1), 1-16.
Nilsson N., 1986. Probabilistic logic. Artif. Intel.. 28, 71-
87.
Pollock S. M., 1970. A Simple Model of Search for a
Moving Target. Operations Research, 18, 883-903.
Rabiner L. R. 1989. A Tutorial on Hidden Markov Models
and Selected Applications in Speech Recognition.
Proceedings of IEEE, 77(2), 257-286.
Raiffa H., 1968. Decision Analysis. Introductory Lectures
on Choices under Uncertainty. Addison-Wesley,
Reading, Massachusetts.
Rybalov A, Kagan E.,, 2018. Non-distributive implications
for theological judgments. In Proc. 2nd World Congr
Logic and Religion.
Rybalov A, Kagan E., 2018. Mysticism and coincidentia
oppositorum: logical resolution in complex fuzzy logic.
Judaica Petropolitana, 9, 88-96.
Rybalov A, Kagan E, Yager R. 2012. Parameterized
uninorm and absorbing norm and their application for
logic design. In Proc IEEE Conv EEEI.
Rybalov A, Kagan E, Rapoport A, Ben-Gal I., 2014. Fuzzy
implementation of qubit operators. J. Comp. Sci. Syst.
Biol., 7(5), 163-168.
White D. J., 1993. Markov Decision Processes. John Wiley
& Sons, Chichester, UK, etc..
Yager R, Rybalov A., 1996. Uninorm aggregation
operators. Fuzzy Sets Syst., 80, 111-120.
Zade L., 1965. Fuzzy sets.
Inform. Cont., 8, 338-353.
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
394
