![](bg2.png)
tic methods have been studied/proposed ranging from
static and dynamic belief networks e.g., Bayesian Net-
works in (Albrecht et al., 1998) based on the condi-
tional independence assumption to different markov
models based on the markov independence assump-
tion (at different orders) (see (Anderson et al., 2002)
for a short overview). Earlier work strongly focused
on different types of hidden Markov models (HMM)
(Rabiner, 1989). Although successful in different do-
main and applications they imposed strong limitations
and restrictions with regard to the representation but
also provided efficient learning and inference meth-
ods. More recent research has shown that the ex-
pressiveness of the representation can be extended in
different directions with limited cutting backs with
respect to efficiency and precision (e.g. (Fine and
Singer, 1998), (Kersting et al., 2006)).
In contrast to these previous approaches we fo-
cused our investigation on the problem of sparse train-
ing data and the problem of inference on different lev-
els of granularity. One approach that considers an ex-
tended representation and addresses these problems is
the relational markov model (Anderson et al., 2002).
This approach uses domain specific information (sim-
ilarity of states) to provide a faster learning rate and
therefore the ability to deal with sparse reference data.
In contrast to the hidden markov model, this approach
does not provide a sensor model to represent the un-
certainty of perceptions.
To provide probabilistic inference with the usage
of a relational structure, several methods have been
researched, e.g. probabilistic relational models
(PRM) (D’Ambrosio et al., 2003) and dynamic PRMs
(DPRM) (Sanghai et al., 2003). These methods are
similar to bayessian networks in the way that their
nodes can depend to a freely specifiable amount of
parents, other than the restricted structure of a HMM
for example. This leads to a very complex struc-
ture and an enormous inference effort. Due to this
the inference is done via an approximative inference
method, a Rao-Blackwellized particle filter (Murphy
and Russel, 2001).
The concept of attributes like in RMM has also
been applied to a kind of HMM: The logical HMM
(LoHMM, s. (Kersting et al., 2006)). In this case the
states of the LoHMM are capable of containing vari-
ables and therefore variable-bindings to provide re-
lational inference in another meaning than RMM, to
express a dependency of states over time rather than a
relation over granularity levels like this work does.
Instead of representing different levels of granularity
over states (like RMM resp. this work does) the Hier-
archical HMM (HHMM) offers a hierarchy over sev-
eral HMMs (s. (Fine and Singer, 1998)).
3 LEARNING AND PREDICTION
BASED ON RHMM
Before introducing the relational hidden markov
model, we give a brief description on HMMs and
RMMs that build the basis for this approach.
3.1 Prerequisites
The HMM (s. (Rabiner, 1989)) can be characterized
as a double stochastic process. The HMM is defined
as the tuple HMM =< S, E, A, B, π >, where π is the
initial state distribution. S is defined as a set of hidden
states, E denotes the set of possible evidences (visible
states). The transition probabilities A are given by a
quadratic matrix A = |S|x|S| and the two-dimensional
matrix B describes the emission probability of evi-
dences in dependency to the states with B = |S |x|E|.
A well-known application of the HMM is the calcu-
lation of the most probable (hidden) state transitions
based on a sequence of observations and to forecast a
probability distribution (prediction model).
The RMM uses in contrast to the HMM a tax-
onomy of states to improve the inference process
of a markov chain by smoothing. Furthermore, the
RMM depends only on state transitions (therefore
no emission probability / sensor model is given).
State transitions trained with limited or no refer-
ence data will be approximated by considering nearby
(more abstract) states’ data depending on the tax-
onomy. It has been shown in (Anderson et al.,
2002) that this approximation leads to better infer-
ence quality. Formally, the RMM is defined as a tuple
RMM =< D, R , A, π >, where D describes the set of
taxonomic relations (in Anderson et al. they are de-
noted as domains), R is a set of predicate-attribute re-
lations with Predicate(Attribute
1
, Attribute
2
, ...) that
specifies a set of states by instantiating correspond-
ing to the leaves of the taxonomy / domain of the at-
tributes.
To provide an explicit model of the sensory un-
certainty and the ability to deal with sparse reference
data the relational markov model and the hidden
markov model will be combined. The proposed
RHMM-method will provide flexible inference on
different levels of granularity. Similar to the HMM
the RHMM separates hidden and visible states and
each state is represented by a relation.
3.2 Structure
The relational hidden markov model is defined as a
tuple RHMM =< D, R ,E , A, B, π > with the set of
ICAART 2010 - 2nd International Conference on Agents and Artificial Intelligence
212