Image Representation: From Raw Data to Models
1
I. Gurevich, Vera Yashina
Dorodnicyn Computing Center, Russian Academy of Sciences
Vavilov str. 40, Moscow, 119333 Russian Federation
Abstract. A space of image representations used in Descriptive Theory for
Image Analysis is considered. The main types of image representations are
introduced in accordance with image and its reducing to a recognizable form.
1 Introduction
This paper describes research results related to the Descriptive Theory for Image
Analysis (DTIA), which was proposed and substantiated by I.B. Gurevich and is
being developed by members of his scientific school [1, 2]. The main contribution of
this paper is that it describes a system of concepts characterizing initial data (images)
in recognition problems and uniquely defines a hierarchical system of relations
introduced on classes of concepts. This system underlies the formal definition of
methods for synthesizing image models and descriptive image models intended for
image recognition problems.
At present, the crucial points in the development of image analysis and recognition
include the understanding of the nature of initial data (images); methods for image
representation and description that provide the construction of image models for
image recognition problems; a mathematical language for a unified description of
image models and their transformations ensuring the construction of image models
and the solution of image recognition problems; and models for solving image
recognition problems in the form of standard algorithmic schemes that, in the general
case, ensure transitions from the original image to its model and from the latter to a
desired solution.
DTIA specifies a unified conceptual structure for the development and implemen-
tation of such models and mathematical language. The main goal of DTIA is to
represent various methods, operations, and representations used in image analysis and
recognition in a structured and standardized form. DTIA provides the conceptual and
mathematical foundation for Image Mining. The axiomatics and formal structures of
DTIA provide methods and tools for representing and describing images for their
1
This work was partially supported by the Russian Foundation for Basic Research Grant № 08-01-00469,
by the project “Algorithmic schemes of descriptive image analysis” of the Program of Basic Research
“Algebraic and Combinatorial Techniques of Mathematical Cybernetics” of the Department of
Mathematical Sciences of the RAS and by the project of the Program of the Presidium of the Russian
Academy of Sciences “Fundamental Problems of Computer Science and Information Technologies” (the
project no. 2.14)
Gurevich I. and Yashina V. (2009).
Image Representation: From Raw Data to Models.
In Proceedings of the 2nd International Workshop on Image Mining Theory and Applications, pages 20-29
DOI: 10.5220/0001962900200029
Copyright
c
SciTePress
subsequent analysis and estimation.
Overall, the following results were obtained by analyzing and adapting descriptive
image models (DIMs) for automated image analysis and recognition: (1) a system of
concepts was introduced that characterizes initial data (images) in image recognition
problems; (2) a system of concepts was introduced that characterizes and defines
DIMs for image recognition; (3) a hierarchical scheme was constructed that represents
and unifies the conceptual apparatus, definitions, mathematical objects, and transfor-
mations defining the construction of an image model in image recognition based on
DTIA; (4) a scheme for constructing image models was proposed; and (5) schemes
for constructing four classes of DIMs were introduced.
2 Images as Initial Data in Image Recognition Problems
To develop automated image recognition methods, we need techniques for effective
image formalization that reflects the image semantics and the information carried by
the internal image structure and the structure of external connections in the actual
world part (scene) reproduced by the image. No systematic mathematical methods for
image formalization and analysis are available at present. The overwhelming majority
of image-related methods are heuristic, and their merits are determined by how
effectively the pictorial nature of images is overcome by non-pictorial tools.
An image is an object with a complex information structure reproducing
information on the original scene using the brightness of discrete image elements
(pixels); configurations of image fragments and sets of pixels; and spatial and logical
relations between configurations, sets of pixels, and individual pixels. In contrast to
other ways of data representation, images are highly informative, illustrative,
structured, and naturally perceived by humans. An image is a mixture of original
(raw, "actual") data, their representations, and deformations arising in the formation
and transformation of digital images. Representations reflect the information and
physical nature of objects, events, and processes represented by images, while
deformations are caused by the technical characteristics of tools used to record, form,
and transform images in the construction of hierarchical representations. Thus, while
developing methods for a formal description of images, we have to take into account
not only the brightness of image pixels but also additional explicit and implicit
information associated with images.
A natural hypothesis underlying the formalization of image descriptions and its
conceptual apparatus is that an original image is specified not only by a set of its
digital realizations but also by contextual and semantic information associated with
the methods used for image recording and formation or with some specific aspects of
the image.
To construct a formalized description of an image, transformations admissible for
the given type of images have to be applied to the entire information available on the
image. Thus, it is a necessity to study (i) the types of information carried by the image
(the space of initial data) and (ii) the transformations to be applied to the original
image to reduce it to a form acceptable for recognition algorithms (the space of trans-
formations).
21
The descriptions of procedures for serial and/or parallel applications of transfor-
mations form the space of transformations to initial data from the space of initial data
constitute a set of schemes for constructing formal descriptions of images (the space
of image representations).
To ensure that recognition algorithms can be applied to the resulting formal image
descriptions, the schemes constructed (image representations) have to be imple-
mented; i.e., image models have to be constructed by reducing the original image
(with allowance for the entire information on it) to a form acceptable for recognition
algorithms. The space of image representations is intermediate between the space of
initial data and the space of image models.
Thus, the construction of image models involves the synthesis and application of
objects from the set of initial data (i.e., images), the set of image transformations for
Reducing Images to a Recognizable Form (RIRF), the set of image representations
(i.e., schemes for constructing formal image descriptions), and the set of image
models.
DTIA deals with three classes of admissible image transformations: procedural, pa-
rametric, and generating transformations (see Definitions 2-4 below). These classes of
transformations generate three classes of image representations and three classes of
image models, respectively.
The following concepts are used to characterize images in DTIA: initial data
(image as a whole with its legend), transformations of initial data, representations of
initial data (by a representation, we mean a formal scheme for describing an image
and the objects it involves), and models of initial data (by an image model, we mean
an image description acceptable for recognition algorithms).
Additional objects are introduced to define the types of representations through
which an original image goes in the course of image model construction and to estab-
lish the relations between these types. These additional objects include generating
rules, structuring elements, semantic and contextual information on an image, digital
image realizations, classes of image representations, realizations of image representa-
tions, classes of image models, and a correct image model.
The following relations between the objects have been revealed by analyzing the
basic concepts related to image description construction: 1) there are deterministic
(obvious) relations between the initial data (image) and: a) the transformations
applied to it; b) methods for obtaining its digital realizations; c) the results of applying
transformations to digital image realizations; 2) there are relations inherent in DTIA:
a) between the classes of image transformations and the classes of admissible image
representations; b) between the classes of admissible image representations and the
classes of image descriptions in a form acceptable for recognition algorithms (classes
of image models); 3) special relations were revealed: a) between some classes of
image models; b) between some class of image models and initial data; c) between the
results of applying transformations to digital image realizations and the classes of
image models.
The study of these relations has led to the construction of a hierarchy of DTIA con-
cepts. According to the hierarchical scheme shown in Fig. 1, the concepts introduced
can be structured so that they can be used to develop algorithmic schemes for image
analysis and recognition and to describe images with the help of DTIA. Based on the
hierarchy, several axioms of DTIA were formulated in [2].
The scheme in Fig.1 reflects several levels of relations between DTIA concepts.
22
I. An open arrow denotes the relation whereby object 1 is associated with objects 2,
3, …: the original image I is associated with the following three sets: 1) the set of
transformations
{}O
(the set of structuring elements
{}S
is auxiliary for
{}O
), 2) the
set of initial data
0
{}
I
; 3) the set of image models
{}
M
.
II. The application of
{}O
to
0
{}
I
generates a set
{}
M
of correct image models
(this assertion is proved in Theorem 1).
III. A solid heavy arrow denotes the relation "object 1 consists of object 2, object
3…" 1) The set of transformations
{}O
consists of three subsets: procedural
transformations
{}
T
O
, parametric transformations
{}
P
O
, and generating transforma-
tions
{}
G
O
. Moreover,
{}O
(as well as its subsets
{}
T
O
,
{}
P
O
,
{}
G
O
) is specified
together with the set
{}S
of structuring elements, which can be applied to the image
together with transformations. The rules for applying generating transformations from
{}
G
O
to initial data from
0
{}
I
are described by a set of generating rules
{}R
. Thus, a
subset of
{}
G
O
is associated with a fixed subset of
{}R
. 2) The set of image models
{}
M
consists of four subsets: procedural image models
{}
T
M
, parametric image
models
{}
P
M
, generating image models
{}
G
M
, and image I-model
{}
I
M
. 3) The
initial data
0
{}
I
contain contextual and semantic information
{}B
on the image and
the set of realizations
{}
I
I
′′
∈
of I that represent the given object or scene.
IV. A dashed line denotes the relation "an object generates another object”. 1) The
three classes
{}
T
O
,
{}
P
O
,
{}
G
O
of transformations generate the following three
classes
Fig. 1. Hierarchy of DTIA concepts.
23
of image representations: procedural representations
{()}
T
ρ
ℜ
, parametric
representations
{()}
P
ρ
ℜ
, and generating representations
{()}
G
ρ
ℜ
. 2) It can be
proved that any image T-model
{}
TT
MM∈
generates an image realization
{}
T
IM
′
∈
(see Proposition 1).
V. The dashed circle in the scheme stresses that the image realizations
{'}
I
are re-
lated to the semantic and context information
{}B
. This relation means that the
various types of initial data are used together in solving problems.
3 Descriptive Image Models
In DTIA it is assumed that an image is described by a set of initial data
0
{}I
. The
composition of this set is determined below.
Lemma 1. The set
0
{}
I
of initial data consists of two subsets
{}
I
′
and
{}B
: (1) the
set of realizations
{}
I
I
′′
∈
of I representing the given object or scene such that
{( , ( ))}
f
x
D
Ixfx
∈
′
=
is the set of points
x
lying in the domain
f
D
of the image
realization and the set of values
()
f
x
at each point of
f
D
; and (2) semantic and con-
textual information
{}B
on the image.
Definition 1. An I-model of an image is any element
I
′
of a set
{}
I
′
of image
realizations.
Consider a set of transformations
{}O
introduced over data given in the form of
images. Below we define the basic classes of image transformations (procedural, pa-
rametric, and generating) and introduce the related concepts of a structuring element,
a generating rule, and a correct generating transformation.
Definition 2. A Procedural Transformation
{}
TT
OO∈
of arity r over a set of
images
1...
{}
ir
I
is an operation such that its application to
1...
{}
ir
I
transforms it into
another set of images, into an image, or into image fragments.
Accordingly, a procedural transformation
{}
TT
OO∈
of arity r over a set of image
I-models
1...
{}
ir
I
′
is an operation such that its application to
1...
{}
ir
I
′
transforms it into
another set of image I-models, into an image I-model, or into a set of I-models of
image fragments. The operands of this operation can be I-models of a single original
image or I-models of several different original images.
Definition 3. A Parametric Transformation
{}
P
P
OO∈
over an image I is an
operation such that its application to I transforms it into a numerical characteristic p
that correlates with the properties of geometric objects, brightness characteristics, or
configurations formed by regular repetitions of the geometric objects and brightness
characteristics of the original image.
To calculate a numerical characteristic p of I, we can use both the set of image
realizations and semantic or contextual information on the image.
24
Definition 4. A Generating Transformation
{}
GG
OO∈
over an image I is an
operation generating a particular representation of I that reflects some specific
properties of I.
For the definition of a representation, see Definition 8.
Examples of such transformations are functions describing curves, the conjunction
function, the disjunction function, and image coding functions.
Definition 5. A Generating Rule
R
is a rule for constructing an image model that
determines a strict sequence of generating transformations applied to the image in
order to construct its model.
For the definition of an image model, see Definition 9.
Definition 6. A Generating Transformation
G
O
is correct for a given image if and
only if there are generating rules according to which
G
O
ensures the construction of a
generating image model.
Note that a generating model of an image (image G-model) is constructed using a
realization of a generating image representation (G-representation). For the
definitions of a generating representation and a realization of a generating
representation, see Definitions 17 and 18.
Definition 7. A Structuring Element
{}SS∈
is a two-dimensional spatial object
whose convolution with an image yields a partition of the image into a system of
fragments suitable for local analysis. A structuring element is specified by parameters
defining its form and numerical and geometric characteristics.
Definition 8. An Image Representation
()I
ℜ
is a formal scheme for obtaining a
standardized formal description of the surfaces, point configurations, and shapes
forming the image and the relations between them.
Definition 9. An Image Model
()
M
I
is a formal image description generated by a
realization of an image representation
()I
ℜ
.
Definition 10. A realization of an image representation is the application of the
representation to realizations of the original image with particular parameter values
specified for the transformations involved in the representation.
{}O
1
{}
I
′
{}B
{}S
{}{}SI
′
∗
{()}
ρ
ℜ
{}
M
2
{}I
′
Fig. 2. Construction of an image model.
25
The construction of an image model with the help of the objects and concepts
introduced above is shown schematically in Fig. 2.
Definition 11. A Correct Representation of an Image I is an element of the set of
image representations constructed from contextual and semantic information
{}B
by
applying transformations
{}O
and structuring elements
{}S
, where the sets
{}B
,
{}O
, and
{}S
are associated with I and the set
{}S
may be empty.
Definition 12. A Correct Image Model is an element of a set of image models
generated by implementing correct image representations on the set of initial data
0
{}
I
.
Theorem 1. Any element m of the set
{}
M
generated by applying transformations
from
{}O
to the set of initial data
0
{}
I
is a correct image model.
Corollary. Given an image I, the set of correct models of I is closed under trans-
formations from
{}O
as applied to the set of initial data
0
{}
I
with the use of struc-
turing elements from the set
{}S
associated with I.
The proof of this corollary is based on Definitions 11 and 12.
All transformations applied to an image model or an image are introduced to
achieve one of the following goals: the construction of a new model; RFSR; or the
construction of an aggregate model estimate, i.e., the transition from the space of ini-
tial data to a space of estimates on which classifying decision making procedures are
implemented in image recognition.
Schemes 1 and 2 illustrate the relations between image representations and image
models.
{()} {,}:{} { }
I
OS I Mℜ= ⇒
(1)
12 1
{()}() { , ,..., , }():{}{} { }
n
Ip OO OSpI I M M
′′
ℜ= ∈⇒∈
(2)
Definition 13. A T-representation
(, )
T
ημ
ℜ
of an image I is a formal scheme for
deriving a standardized formal description of I. This scheme is constructed from con-
textual and semantic information
{} {}BB⊂
by applying procedural transformations
{()}{}
TT
OO
η
⊂
and structuring elements
{( )} {}SS
μ
⊂
(where
,
η
μ
are parameters
of the procedural transformations and the structuring elements, respectively).
The set of all correct T-representations is denoted by
{(,)}
T
η
μ
ℜ
.
Scheme 3 illustrates Definition 13.
{
}
{}
{}
{},{ } {( )},{ ( )} ( , )
B
TT
SO S O
μη ημ
⎯⎯⎯→⎯⎯→ℜ
(3)
Definition 14. A Realization of a T-representation
(, )
T
ημ
ℜ
of an image I is the
application of
00
(, )
T
ημ
ℜ
to realizations
{} {}
I
I
′
′
⊂
of I, where (
00
,
η
ημμ
=
=
)
are parameter values chosen for the transformations involved in
(, )
T
ημ
ℜ
. Scheme 4
26
illustrates Definition 14.
00 0 0
(, ){}{(), ( )}{'}
TT
I
SO I
η
μημ
′
ℜ∗= ∗
(4)
Proposition 1. Any T-representation
(, ) { (, )}
TT
η
μημ
ℜ∈ℜ
of an image generates a
set of image T-models
00
{(,}
T
M
ημ
by setting parameter values
0
ηη
=
and
0
μμ
=
for the procedural transformations and structuring elements.
The set of all correct image Т-models is denoted by
{}
T
M
.
Proposition 2. Any image T-model
{}
TT
MM∈
generates an image realization I',
i.e., an image I-model
{}{'}
IT
M
IM I
′
≡∈ ≡
.
Definition 15. A P-representation
(, )
P
ημ
ℜ
of an image I is a formal scheme for
deriving a standardized formal description of I. This scheme is constructed from the
contextual and semantic information
{} {}BB⊂
by applying parametric transforma-
tions
{()}{}
P
P
OO
η
⊂
and structuring elements
{( )} {}SS
μ
⊂
(where
,
η
μ
are
parameters of the parametric transformations and the structuring elements, respec-
tively).
The set of all correct P-representations is denoted by
{(,)}
P
η
μ
ℜ
. Scheme 5
illustrates Definition 15.
{}
{}
{},{ } {( )},{ ( )} ( , )
{}
SO S O
B
PP
μ
ηημ
⎯⎯⎯→⎯⎯→ℜ
(5)
Definition 16: A realization of a P-representation
(, )
P
ημ
ℜ
of an image I is the
application of
00
(, )
P
ημ
ℜ
to realizations
{} {}
I
I
′
′
⊂
of I, where (
00
,
η
ημμ
=
=
)
are parameter values chosen for the transformations involved in
(, )
P
ημ
ℜ
.
Scheme 6 illustrates Definition 16.
00 0 0
(, ){}{(), ()}{'}
PP
ISO I
ημ η μ
′
ℜ∗= ∗
(6)
Proposition 3. Any P-representation
(, ) { (, )}
PP
η
μημ
ℜ∈ℜ
generates a set of image
P-models
00
{(,}
P
M
ημ
by setting parameter values
0
ηη
=
and
0
μμ
=
for the
parametric transformations and structuring elements, respectively.
The set of all correct P-models of an image is denoted by
{}
P
M
.
Definition 17. A G-representation
(,, )
G
λ
ημ
ℜ
of an image I is a formal scheme for
deriving a standardized formal description of I. This scheme is constructed from the
contextual and semantic information
{} {}BB⊂
according to generating rules
{()}R
λ
that completely define the sequence of generating transformations
{()}{}
GG
OO
η
⊂
and structuring elements
{( )} {}SS
μ
⊂
applied to I (here,
,,
λ
ημ
are parameters of the generating rules, generating transformations, and structuring
elements, respectively).
27
The set of all correct G-representations is denoted by
{(,,)}
G
λ
ημ
ℜ
. Scheme 7
illustrates Definition 17.
{}
{}
{},{},{} {()},{()},{()} (,,)
{}
RSO R S O
B
GPG
λ
μη λημ
⎯⎯⎯→⎯⎯→ℜ
(7)
Definition 18. A realization of a G-representation
(,, )
G
λ
ημ
ℜ
of an image I is the
application of
00 0
(,, )
P
λ
ημ
ℜ
to realizations
{} {}
I
I
′
′
⊂
of I, where (
00 0
,,
λ
ληη μ μ
===
) are parameter values chosen for the transformations in-
volved in
(,, )
G
λ
ημ
ℜ
.
Scheme 8 illustrates Definition 18.
(,, ){}{(),(), ( )}{'}
00 0 0 0 0
I
RSO I
GG
λη μ λ η μ
′
ℜ∗= ∗
(8)
Proposition 4. Any G-representation
(,, ) { (,,)}
GG
λ
ημ λημ
ℜ∈ℜ
generates a set of
image G-models
00 0
{(,,}
G
M
λ
ημ
by setting parameter values
0
λ
λ
=
,
0
ηη
=
, and
0
μμ
=
for the generating rules, generating transformations, and structuring elements,
respectively.
The set of all correct image G-models is denoted by
{}
G
M
.
The interrelations between the concepts discussed in Section 2 are shown in Fig. 1.
4 Conclusions
At present, automated extraction of information from images is a major strategic goal
of fundamental research in image analysis, recognition, and understanding.
DTIA was proposed and is being developed as a conceptual and logical foundation
for image analysis and recognition. It includes a collection of methods for image
analysis and recognition, methods for RIRF, a system of concepts for image analysis
and recognition, classes of DIMs, various settings in image analysis and recognition,
and a base image recognition model.
The above description of DTIA concepts and the formal apparatus introduced for
descriptive image models provide the prerequisites for the formulation of the basic
DTIA axioms. Below are examples of such axioms.
Axiom 1. Any image I can be uniquely associated with a collection of sets (
0
{}
I
,
{}O
,
{}
M
), where
0
{}
I
is the set of initial data,
{}O
is the set of transformations applied
to
0
{}
I
, and
{}
M
is the set of results produced by applying
{}O
to
0
{}
I
.
Scheme (9) illustrate Axiom 1:
{}:( { }) { }
o
OI I M≅⇒
(9)
Axiom 2. The set of transformations
{}O
is specified by a set
{}S
of structuring
elements, by a set
{}R
of generating rules, and by three subsets of transformations
28
{}
T
O
,
{}
P
O
, and
{}
G
O
, namely, (1) procedural transformations
{}
T
O
, (2)
parametric
transformations
{}
P
O
, and (3) generating transformations
{}
G
O
.
To be continued…
References
1. I.B. Gurevich: The Descriptive Framework for an Image Recognition Problem. Proceedings
of the 6th Scandinavian Conference on Image Analysis, Vol.1, Pattern Recognition Society
of Finland (1989) – 220 – 227
2. I.B. Gurevich, V.V. Yashina: Descriptive Approach to Image Analysis: Image Models.
Pattern Recognition and Image Analysis: Advances in Mathematical Theory and Applica-
tions, Vol.18, No.4. MAIK "Nauka/Interperiodica"/Pleiades Publishing, Inc. (2008) 518-541
29
