CONSENSUS PROBLEM OF MULTI-AGENT SYSTEMS WITH
MARKOVIAN COMMUNICATION FAILURE
Yuebing Hu, James Lam
Department of Mechanical Engineering, University of Hong Kong, China
Jinling Liang
Department of Mathematics, Southeast University, Nanjing 210096, China
Keywords:
Consensus control, Markov chain, Mean-square stability, Communication failure, Multi-agent systems.
Abstract:
This paper studies the consensus problem of multi-agent systems with Markovian communication failure
which may be caused by limited communication capacity. The occurrence of the failures is modeled by
a discrete-time Markov chain. A consensus sufficiency condition is established in terms of linear matrix
inequalities (LMIs). Based on this condition, a new controller design method is provided. A numerical
example is utilized to illustrate the effectiveness of the proposed approach.
1 INTRODUCTION
In the past few years, multi-agent system (MAS)
has sparked the interest of researchers. Up to now,
results of networked MASs have been broadly ap-
plied to biology, physics and engineering, such as the
study of swarming behavior (Liu and Passino, 2004),
automated highway systems (AHSs) (Bender, 1991)
and congestion control in communication (Paganini
et al., 2005). In the cooperative behaviors, consensus,
which means making a group of agents to reach an
agreement on certain quantity of interest that depends
on the states of all agents, is a fundamental topic in
MASs fields and has been studied recently (Olfati-
Saber et al., 2007).
Due to the special property of MASs, the intercon-
nected communication network among agents plays
an important role in the consensus reaching problem
and is usually described by Laplacian graph. Depend-
ing on applications, the network topologies of multi-
ple agents are either fixed or switched, while the lat-
ter is more practical due to the limited or imperfect
communication channel, noises or some special ob-
jectives. Results on switching topology have been
provided in recent articles such as (Olfati-Saber and
Murray, 2004).The physical systems are usually of
big complexity, thus some dynamic processes are de-
scribed by time-varying linear model. Particularly,
for systems subject to randomly changing parameters,
Markov jump linear system (MJLS), which is a hybrid
system composed of a finite number of subsystem
modes, is an appropriate class of models and has been
extensively studied (Xiong and Lam, 2007). How-
ever, to the best of our knowledge, although MASs
are usually treated as networked systems, the issue of
Markovian topology switching processes has not been
fully investigated and fruitful results of MJLS were
not applied to MASs until now.
In this paper, we investigate the consensus control
problem of MASs with communication failure. By
modeling the communication process in a Markovian
process, a new sufficiency condition of the consensus
problem is established in terms of linear matrix in-
equalities (LMIs) which can be easily solved. Based
on this condition, a state-feedback controller is de-
signed such that the consensus of the closed-loop sys-
tem is mean square stable (MSS) with known com-
munication failure processes.
Notation. Throughout this paper, R
n
, R
n×m
, S
n×n
represent the n-dimensional Euclidean space, the set
of all n×m real matrices and the n×n real symmet-
ric positive definite matrices, respectively; Z
+
is the
set of non-negativeintegers; (Ω, F , P) denotes a com-
plete probability space; the superscript “T” represents
the transpose; for Hermitian matrices X = X
T
∈R
n×n
and Y = Y
T
∈R
n×n
, the notation X ≥Y (respectively,
X > Y) means that the matrix X −Y is positive semi-
373
Hu Y., Lam J. and Liang J. (2010).
CONSENSUS PROBLEM OF MULTI-AGENT SYSTEMS WITH MARKOVIAN COMMUNICATION FAILURE.
In Proceedings of the 7th International Conference on Informatics in Control, Automation and Robotics, pages 373-376
DOI: 10.5220/0002939803730376
Copyright
c
 SciTePress

definite (respectively, positive definite); I
n
is the n×n
identity matrix; E(·) denotes the expectation opera-
tor with respect to some probability measure; (M)
ij
refers to the ith row, jth column element of matrix M;
k·k represents the Euclidean norm for a vector and
the spectral norm for a matrix; the symbol ⊗ denotes
the Kronecker product; trace(·) is the trace of ma-
trix; diag(M
1
, M
2
, . . . , M
N
) is a block-diagonal matrix
with diagonal blocks M
1
, M
2
, . . . , M
N
; 1
N
is defined as
1
N
= (1, 1, . . . , 1)
T
∈ R
N
and 0
N
is defined similarly,
that is, 0
N
= [0, 0, . . . , 0]
T
∈ R
N
.
2 PRELIMINARIES
A directed graph G = (V , E ) consists of a finite ver-
tex set V and an edge set E ⊂ V
2
. Suppose there
are n vertices in V , then the graph has an order n and
each vertex can be uniquely labeled by an integer i
belonging to a finite index set I = {1, 2, . . . , n} . Each
edge can be denoted by an ordered pair of distinct ver-
tices (v
i
, v
j
) where v
j
is the head and v
i
is the tail, that
is, the edge points from v
i
to v
j
with no self-loop.
Each edge (v
i
, v
j
) ∈ E corresponds to the information
transmission from agent j to agent i. The graph with
the property that for any (v
i
, v
j
) ∈ E ⇔ (v
j
, v
i
) ∈ E
is said to be symmetric or undirected. The in (out)-
degree of v
i
, denoted by d
i
(v
i
) (d
o
(v
i
)), is the number
of edges with v
i
as its tail(head). If (v
i
, v
j
) ∈ E , then
v
j
is one of the neighbors of v
i
. The set of neighbors
of v
i
is denoted by N
i
= {v
j
∈ V : (v
i
, v
j
) ∈ E }. An
adjacency matrix of graph G with order n is an n×n
matrix A = {a
ij
} defined as
a
ij
=
(
1 if (v
i
, v
j
) ∈ E ,
0 otherwise.
An in-degree matrix of graph G with order n is an
n×n matrix D = diag{d
11
, d
22
, . . . , d
nn
} where d
ii
=
∑
v
j
∈N
i
a
ij
. A Laplacian matrix L of graph G with
order n is an n×n matrix defined as follows:
L = D −A .
Let us consider a MAS with n agents. The
discrete-time linear dynamics of agent i can be de-
scribed by the following equation:
x
i
(k+ 1) = Ax
i
(k) + Bu
i
(k), i ∈ I (1)
where x
i
(k) ∈R
m
is the system state, u
i
(k) ∈R
l
is the
control input, A ∈R
m×m
, B ∈R
m×l
, k ∈Z
+
is the time
step, x
i
(0) , x
i0
is the initial state.
Suppose the communication failure between
agents i and j of the MAS (1) behaves in an indepen-
dent way, that is, agent i can receive data from agent
j does not necessarily mean agent j can receive data
from agent i. The control input of the ith agent with
communication failure is
u
i
(k) = K
∑
v
j
∈N
i
γ
ij
(k)(x
i
(k) −x
j
(k)) (2)
where K ∈ R
l×m
is the controller gain to be designed,
γ
ij
(k) denotes the communication status from agent
j to i at time k (1 for successful communication, 0
for unsuccessful communication). The communica-
tion status process is assumed to be a discrete-time
homogeneous Markov chain taking values in a finite
set W = {0, 1} with transition probability matrix
Π
ij
=

1−β
ij
β
ij
α
ij
1−α
ij

, (3)
where 0 ≤Pr(γ
ij
(k+1) = 0 |γ
ij
(k) = 1) = α
ij
≤1 and
0 ≤Pr(γ
ij
(k+1) = 1 |γ
ij
(k) = 0) = β
ij
≤1 are called
the failure probability and the recovery probability,
respectively. To simplify the expression, A , (α
ij
),
B , (β
ij
) are used to denote the failure probability
matrix and recovery probability matrix, respectively.
Notice that the communication failure model in (2)
indicates that the error signal x
i
(k) −x
j
(k) will not be
employed by the controller u
i
at time k when the j
communication channel fails.
Under the above formulation, if there is no com-
munication channel from agent j to i, that is, no edge
(v
i
, v
j
) in the graph and a
ij
= 0 in the Laplacian ma-
trix, then the absence channel is treated as an ‘inef-
fective’ channel with γ
ij
(k) = 0 for all k, with the fail-
ure and recovery probabilities assigned to be α
ij
= 1
and β
ij
= 0, respectively. By treating the absent com-
munication channels this way, the original problem is
equivalent to considering the communication failure
problem of an MAS with a complete graph governed
by known communication failure probability γ
ij
(k)
in the communication channel from j to i at time k.
Consequently, the Laplacian matrix at time k can be
rewritten as
L(k) =


∑
j6=1
γ
1j
(k) −γ
12
(k) . . . −γ
1n
(k)
. . . . . . . . . . . .
−γ
n1
(k) −γ
n2
(k) . . .
∑
j6=n
γ
nj
(k)


(4)
where L(k) ∈L
0
, {L
1
, L
2
, . . . , L
d
}such that L
0
con-
tains all possible Laplacian matrices of the MAS.
Here, maxd = 2
˜
d
where
˜
d ,
∑
n
i=1
d
ii
is the total num-
ber of effective communication channels in the graph
(that is, the number of edges of the complete graph
subtracting those ‘ineffective’ edges).
Definition 1. A communication failure process is said
to be Markovian if it is a discrete-time homogenous
Markov chain defined in a complete probability space
ICINCO 2010 - 7th International Conference on Informatics in Control, Automation and Robotics
374

(Ω, F , P), and takes value in W with known transi-
tion probability matrix Π
1
, (λ
ij
) ∈ R
s
0
×s
0
.
Denote by X the concatenation of vectors
x
1
, x
2
, . . . , x
n
, that is, X = (x
T
1
, x
T
2
, . . . , x
T
n
)
T
, by (1) and
(2), the closed-loop dynamics of MASs (1) can be
rewritten in a matrix form
X(k+ 1) = (
¯
A+
¯
B(k))X(k), (5)
where
¯
A = I
n
⊗A,
¯
B(k) = L(k)⊗(BK), L(k) takes val-
ues from L
0
. In this paper, we always assume that
every agent could receive information of some other
agents with a non-zero probability during the whole
control process.
The mode transition probability π
L
a
L
b
from L
a
to
L
b
is given by
π
L
a
L
b
=
n
∏
i, j=1,i6= j
(Γ
L
a
L
b
)
ij
(6)
for all a, b = 1, . . . , d, which satisfies
∑
d
b=1
π
L
a
L
b
= 1,
where
(Γ
L
a
L
b
)
ij
=









1−α
ij
if (L
b
)
ij
= (L
a
)
ij
= −1,
1−β
ij
if (L
b
)
ij
= (L
a
)
ij
= 0,
α
ij
if (L
b
)
ij
= 0, (L
a
)
ij
= −1,
β
ij
if (L
b
)
ij
= −1, (L
a
)
ij
= 0.
3 STABILITY ANALYSIS AND
CONTROLLER SYNTHESIS
In this section, we first give definitions on stabil-
ity and consensus. Then we consider the consensus
reaching and controller design problems for MASs
with communication failure characteristics described
in Section 2.
Definition 2. MAS (5) with Markovian communica-
tion failure process (2) is said to be mean square sta-
ble (MSS) if
lim
k→∞
E(kX(k)k
2
| X(0)) = 0 (7)
for any initial state X(0) ∈ R
mn
.
Definition 3. Agents of MAS (5) with Markovian
communication failure process (2) are said to reach
consensus if
lim
k→∞
E(x
i
(k) −x
j
(k)) = 0 (8)
for all i, j ∈ I .
Now we are in the position to present the main
contribution of this paper.
Theorem 1. Consider MAS (5) with Markovian com-
munication failure (2), given the controller gain ma-
trix K, consensus is reached if there exist real matri-
ces P
U
∈ S
(n−1)m×(n−1)m
, U ∈ L
0
, such that
∑
V ∈L
0
π
U V
Φ
T
V
P
V
Φ
V
−P
U
< 0, ∀U ∈ L
0
, (9)
where Φ
V
= I
n−1
⊗A+Λ
V
⊗BK, Λ
V
= T
T
o
V T
o
and
T
o
is the orthogonal basis for the null space of 1
n
.
Proof. Construct orthogonal matrix T =
h
1
√
n
1
n
T
o
i
∈ R
n×n
, where T
o
is the orthogonal
complement of 1
n
satisfying T
T
o
T
o
= I
n−1
. Since T
o
is the orthogonal complement of 1
n
, then
1
T
n
T
o
= 0
T
n−1
, T
T
T =

1
√
n
1
T
n
T
T
o

h
1
√
n
1
n
T
o
i
= I
n
;
which means T is an orthogonal matrix and T
T
L T
is a similarity transformation. Partition the Laplacian
matrix L and matrix T
T
conformably:
T
T
=

1
√
n
1
√
n
1
T
n−1
T
T
o1
T
T
o2

,
then
T
T
L T =

0 A
L
0
n−1
Λ
L

, (10)
where A
L
=
1
√
n
1
T
n
L T
o
and Λ
L
= T
T
o
L T
o
.
If the graph is undirected, that is, L = L
T
. Thus
T
T
L T =

0 (
1
√
n
T
T
o
L 1
n
)
T
0
n−1
T
T
o
L T
o

=

0 0
T
n−1
0
n−1
Λ
L

,
Denote
˜
X(k) = (T ⊗I
m
)
T
X(k), the dynamics of
the closed-loop system can be described by
˜
X(k+ 1)
= (I
n
⊗A)
˜
X(k) +

0⊗I
m
A
L(k)
⊗(BK)
0
n−1
⊗I
m
Λ
L(k)
⊗(BK)

˜
X(k),
where A
L(k)
=
1
√
n
1
T
n
L(k)T
o
, Λ
L(k)
= T
T
o
L(k)T
o
.
Define
˜
X(k+ 1) =

X
1
(k+ 1)
X
2
(k+ 1)

,
where
X
1
(k+ 1) = AX
1
(k) + (A
L(k)
⊗(BK))X
2
(k),(11)
X
2
(k+ 1) = Φ
L(k)
X
2
(k). (12)
where Φ
L(k)
= I
n−1
⊗A+ Λ
L(k)
⊗BK. Let
z
i
(k) =
∑
v
j
∈N
i
γ
ij
(k)(x
i
(k) −x
j
(k)),
CONSENSUS PROBLEM OF MULTI-AGENT SYSTEMS WITH MARKOVIAN COMMUNICATION FAILURE
375

Z(k) = (z
T
1
(k), z
T
2
(k), . . . , z
T
n
(k))
T
,
then
Z(k) = (L(k) ⊗I
m
)X(k)
where L(k) is given by (4), and
(T
T
⊗I
m
)Z(k) =

A
L(k)
⊗I
m
Λ
L(k)
⊗I
m

X
2
(k),
where T
T
L(k)T =

0 A
L(k)
0
n−1
Λ
L(k)

. Thus, Z(k) is MSS
if X
2
(k) is MSS, that is, the consensus of MASs (5) is
reached if (12) is MSS.
Define a Lyapunov function as follows:
V(k, L(k)) = X
T
2
(k)P
L(k)
X
2
(k)
where P
L(k)
> 0 are matrices to be determined.
Let U = L(k), V = L(k+ 1), then
E(V(k+ 1, L(k + 1))|L(k) = U ) −V(k, U )
= E(X
T
2
(k+ 1)P
L(k+1)
X
2
(k+ 1)|L(k) = U )
−X
T
2
(k)P
U
X
2
(k)
= X
T
2
(k)



∑
V ∈L
0
j=1,...,s
0
π
U V
Φ
T
V
P
V
Φ
V
−P
U



X
2
(k)
< 0
for any X
2
(k) 6= 0 if inequality (9) holds. Hence
lim
k→∞
E(V(k, L(k))) = 0, which ensures
lim
k→∞
E(kX
2
(k;x
0
)k
2
) = 0,
that is, system (12) is MSS. This completes the proof.

Now, based on Theorem 1 and Schur complement
Lemma, a controller design method is readily derived
in the following theorem.
Theorem 2. Consider MAS (5) of n agents with
Markovian communication failure (2), consensus of
the closed-loop system is reached if there exist X
i
∈
S
(n−1)m×(n−1)m
, i = 1, 2, . . . , d, Y ∈ R
l×m
and G ∈
R
m×m
such that ∀i = 1, 2, . . . , d,

−I
n−1
⊗(G+ G
T
) + X
i
Θ
T
i
Θ
i
−Λ

< 0 (13)
where
Λ = diag(X
1
, X
2
, . . . , X
d
),
Θ
i
=

√
π
iL
1
Ξ
T
L
1
√
π
iL
2
Ξ
T
L
2
. . .
√
π
iL
d
Ξ
T
L
d

T
,
Ξ
V
= I
n−1
⊗AG + Λ
V
⊗BY ,
Λ
V
= T
T
o
V T
o
, V ∈ L
0
and T
o
is the orthogonal basis for the null space of 1
n
.
Moreover, a consensus controller gain matrix in (2) is
given by K = Y G
−1
.
Proof. Define
Ψ
i
,

√
π
iL
1
Φ
T
L
1
√
π
iL
2
Φ
T
L
2
. . .
√
π
iL
d
Φ
T
L
d

T
,
where Φ
V
= I
n−1
⊗A + Λ
V
⊗BK. Pre- and post-
multiplying inequality (13) by

Ψ
i
I
(n−1)md

and its
transpose, respectively, then
Ψ
i
X
i
Ψ
T
i
−Λ < 0,
that is,

−X
−1
i
Ψ
T
i
Ψ
i
−Λ

< 0. (14)
Thus, inequality (14) is equivalent to inequality (9) by
replacing P
i
by X
−1
i
. This completes the proof. 
4 CONCLUSIONS
In this paper, a consensus control problem of MASs
with communication failure between agents has been
studied. A sufficient condition on consensus reaching
problem is established in terms of the feasibility of
some LMIs. In addition, a state-feedback consensus
controller is designed to make the closed-loop sys-
tem reach consensus. A numerical example has been
given to demonstrate the effectiveness of the proposed
results.
ACKNOWLEDGEMENTS
The research was supported by HKU CRCG
200907176129.
REFERENCES
Bender, J. G. (1991). An overview of systems studies of au-
tomated highway systems. IEEE Trans. Veh. Technol.,
40(1):82–99.
Liu, T. F. and Passino, K. M. (2004). Stable social foraging
swarms in a noisy environment. IEEE Trans. Automat.
Control, 49(1):30–44.
Olfati-Saber, R., Fax, J. A., and Murray, R. M. (Jan, 2007).
Consensus and cooperation in networked multi-agent
systems. In Proceedings of the IEEE, volume 95,
pages 215–233.
Olfati-Saber, R. and Murray, R. M. (2004). Consensus
problems in networks of agents with switching topol-
ogy and time-delays. IEEE Trans. Automat. Control,
49(9):1520–1533.
Paganini, F., Wang, Z. K., Doyle, J. C., and Low, S. H.
(2005). Congestion control for high performance, sta-
bility, and fairness in general networks. IEEE/ACM
Trans. Networking, 13(1):43–56.
Xiong, J. and Lam, J. (2007). Stabilization of linear systems
over networks with bounded packet loss. Automatica,
43(1):80–87.
ICINCO 2010 - 7th International Conference on Informatics in Control, Automation and Robotics
376