provides means for fast composition of conclusions
among these levels. The result gives rise a to novel
framework for system health monitoring, exploiting
the intriguing aspects of both involved theories.
2.1 Principles of Bayesian Modelling
The principles of Bayesian modelling consist in spec-
ification of a probabilistic model for observable data
y and a prior distribution for this model’s unobserv-
able parameters θ. In other words, we assume that the
data y obey some distribution with a probability den-
sity function f (y|θ), or, under existence of observable
exploratory variables, f (y|x, θ).
The prior distribution π(θ) statistically summa-
rizes all a priori available information about the in-
ferred parameter θ. It can be obtained from past mea-
surements, from an expert or alternatively has a non-
informative form. The prior pdf is updated by new in-
formation provided by x and y according to the Bayes’
rule,
π(θ|x, y) =
f (y|x, θ)π(θ)
R
f (y|x, θ)π(θ)dθ
, (1)
where the integral
q(y|x) =
Z
f (y|x, θ)π(θ)dθ =
Z
f (y, θ|x)dθ (2)
is taken over the space of θ. It serves as a normalizing
constant, assuring that the resulting posterior distribu-
tion π(θ|x, y) is proper. By careful inspecting of (2) it
is possible to notice, that q(y|x) can play even more
fundamental role than only the normalizing one. It
is also a predictive density of y given x, which is ob-
tained as an expected value E
θ
[ f (y|x, θ)], that is, over
all admissible values of θ.
The resulting posterior pdf in (1), namely
π(θ|x, y), involves both the prior information and the
contribution from the observed data.
An important thought not always applicable prop-
erty related to the Bayesian modelling is conjugacy.
A model f (y|x, θ), being chosen from a suitable class
of distributions (called the exponential family), guar-
antees the existence of the conjugate prior pdf π(θ).
This ensures that the posterior pdf π(θ|x, y) lies in
the same class of distributions as π(θ), the Bayesian
update is analytically tractable, and the posterior can
serve as the prior for a subsequent update when new
data is obtained. This salient feature is clearly prac-
tical for real time modelling of dynamical systems.
Then, denoting t = 1, 2, . . . the time index,
π(θ|x
1:t
, y
1:t
) ∝ f (y
t
|x
t
, θ)π(θ|x
1:t−1
, y
1:t−1
), (3)
where y
1:t
= {y
1
, . . . , y
t
} (analogously for x
1:t
) and ∝
stands for proportionality.
More on Bayesian modelling can be found, e.g.,
in (Gelman et al., 2003); its application to dynamic
modelling is thoroughly treated in (Peterka, 1981).
Example 1. Assume the regression model y = x
0
θ + ε
where y ∈ R
n
is a regressand (dependent or response
variable), x ∈ R
n×m
is a regressor (independent ex-
planatory variable), θ ∈ R
m
denotes regression coef-
ficients and ε is a vector of independent identically
distributed noise terms from a 0-centered normal dis-
tribution with a known variance, N (0, σ
2
). Then, the
probabilistic model f (y|x, θ) for y has the form
y ∼ N (θ
|
x, σ
2
).
If the prior pdf π(θ) is normal, then the posterior pdf
π(θ|x, y) is also normal. That is, the form of the prior
pdf is preserved, prior pdf is conjugate to the model
and dynamic setting (3) is possible. The predictive
distribution with a pdf q(y|x) takes the form of a gen-
eralized Student’s t distribution.
2.1.1 Aspects of the Bayesian Approach
The Bayesian approach has many advantages, aris-
ing from its theoretical consistency and to a signifi-
cant degree balancing its frequent computational bur-
den. Some of the advantages, important for reliability
modelling, are:
Uncertainty – an inherent aspect of information –
is consistently involved in modelling. For instance,
the posterior distribution of θ naturally expresses our
uncertainty in terms of variance.
Asymptotics – while frequentist paradigm heavily
relies on asymptotic results, the Bayesian approach
does not. It yields results with any sample size. This
is possible due to the concept of uncertainty.
Dynamic Modelling – an important aspect con-
nected with conjugate priors. For instance, the
Kalman filter or autoregression have their Bayesian
interpretations. But there are many Bayesian models
without equivalent non-Bayesian counterparts. Dy-
namic modelling inevitably calls for parameter track-
ing. The Bayesian paradigm allows its consistent
treatment, e.g. (Dedecius et al., 2012).
Model Selection and Combination – it is eas-
ily possible to discriminate among several candidate
models or even combine their results in order to fur-
ther improve (e.g. stabilize) the whole modelling, see,
e.g. (Raftery et al., 2010).
ICINCO2014-11thInternationalConferenceonInformaticsinControl,AutomationandRobotics
134