KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED
LEARNING ALGORITHMS
Image understanding based on automated knowledge refinement
Eckhard Büscher
Institute of Theoretical Communication Technique, Hannover University,Heckenweg 6 ,32584 Löhne , Germany
Keywords: Multimedia Applications, Signal Processing for Data
Storage, Image and Video Databases, Neural
Networks, Self Organising/Learning, Knowledge refinement and diagnosis
Abstract: This paper discusses the design and implementation of a knowledge based Modelling system KMS, which
combines semantic and rule based approaches in the modelling process. The design and implementation of
the semantic concepts are controlled dynamically to achieve an optimal degree of reality and to employ
efficient interactivity and accessibility for the user. The model-based controlling module is developed to
achieve efficiency and consistence in the basic analysis process, and to avoid the static structure that
frequently occurs in data driven systems. By using a hypothesis and verification scheme in order to ensure
interactivity and accessibility without sacrificing efficiency the KMS evokes the important task of merging
the use of heuristic knowledge in form of a knowledge base with domain specific requirements. By
detecting contradicting and inconsistent rules and by performing tests in the knowledge base and finally by
creating new hypothesis to solve the problems, the controlling process also provides the decision module
with a concept for automated knowledge refinement. This paper focuses on the implementation and
Multimedia adaptation of the learning processes in correlation with the linked databases.
1 INTRODUCTION
The goal of knowledge based modelling and
construction systems is high quality, robustness,
error tolerance and good capabilities for all kinds of
extension. This means within the controlling of a
system temporary or local solutions have to be
avoided, and additionally learning and evaluation
feedback must be performed especially as long as
the learning process proceeds so that complex
problems can be resolved. Consequently this process
results to learning across multiple levels with close
relationship and efficient consumption of knowledge
inside every learning phase. This feature enables the
propagation of efficient learning inside every level
for its subsequent stage, which provides the
controlling process with important decision and
evaluation criterions for learning strategies in the
higher levels. Therefor each resulting model has the
capability of processing different sensor or data
sources with an individual evaluation and
verification instance. Due to the complex domain
and scene specific requirements the integrated MEN
(Mixed Expert Network) system must operate as
data fusion controller. The individual model parts
decide with independent controlling mechanisms,
but create new information for the knowledge base
with extensive correlation on the output side, where
linking and assignment between the different models
is performed, so that the global system reliability can
be evaluated and performance can be increased as
often as it is necessary.
In construction and modelling systems the
rel
ations between experience and knowledge rules
are strongly dependent from the scene and domain
specific requirements. When the domain specific
requirements change, the rules (or behaviours) have
to be changed accordingly. The supervised learning
method based on experience knowledge rules
assignment provides a construction and modelling
system with the malleability for environmental
changes. This feature enables the decision part of
the system to overcome the problem of domain
specific changes (i. e., noisy environments) and to
learn the effect of altering scene specific
requirements. Unlike the behaviour of genetic
knowledge systems, the malleability of synthetic
computer systems is quite limited. Modifications in
algorithmic computer software almost leads to
356
Büscher E. (2004).
KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS - Image understanding based on automated knowledge
refinement .
In Proceedings of the First International Conference on E-Business and Telecommunication Networks, pages 356-362
DOI: 10.5220/0001399203560362
Copyright
c
SciTePress
sensor
acquisition
processing
evaluation
updating
controlling
analysis
Figure 1: Architecture of the knowledge processing
completely different functions and behaviour. This
may lead to the necessity that no alternate than
reprogramming becomes the only possible solution
in most of these cases independent from the degree
of modification within the requirements.
The influence of the environment in
combination with the complexity and structure of the
data can be considered and integrated if the scene
specific and domain specific restrictions can be
expressed in terms of the inference machine
algorithms. In order to obtain the benefit of learning
based knowledge systems concerning reliability and
function this approach aims to provide the
construction and modelling process with the rules of
decision strategies in learning systems. In addition
high level image interpretation can be demonstrated
by help of genetic algorithms, which have to be
derived from the kernel concepts. Accordingly
biological inherit mechanisms which are based on
continuous evolutionary learning could be integrated
into the decision part of the system. In addition self-
learning concepts, genetic decision controlled
algorithms as well as inherit by clustering can be
applied. By help of these important strategies the
system process structure can be organised with the
ability to synchronise and optimise the learning
modules in specific jobs.
However, most of the feature extraction systems
in related work are based on classification methods
(Mosterman, 1999), whereas this proposal focuses
the field of shape detection by merging of active
sensors e. g. SAR (Synthetic Aperture Radar) and
the resulting features of optical image processing to
implement the feature extraction.. Furthermore the
proposed concept does not require any geometrical
information like matching points or height. in
opposite to (Fink 1987), where interferometric data
is required.
2 RELATED WORK AND SYSTEM
STRUCTURE
The operating system distinguishes between three
processing modules. The basic module analyses the
reliability of the active sensors as well as the passive
sensor acquisition and employs methods to
transform data between each other. Processing and
controlling of the active sensor result data is
performed in the second module, especially the
feature extraction of regions with higher density. In
the subsequent module the data analysis and
evaluation for decision is implemented, where the
results of the basic modules are considered. The
outgoing data of each module will be evaluated
under the aspect of emphasising the hypothesis that
fulfils the given constraints. In Fig. 1 the structure of
the whole system connecting the different modules
is displayed.
2.1 Architecture of the knowledge
based controlling and modelling
system
The knowledge based module combines controlling,
processing and behaviour models of the system as
described in (Canton 1983). This module is invoked
if one of the processing parts fails to detect decision
states in a given situation. In several processes it can
KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS
357
be used to detect contradictions in the module states,
and support engineers and modelling experts in
correcting and modifying the system kernel. The
initial rely value for new errors is kept low however,
when modifying or extending the domain specific or
scene specific database with new rules or semantic
elements. When the same restrictions are detected
again, the rely value of these rules will be evaluated
and they will be considered as temporary. They will
only get a continuous attribute, if their rely value
ranges above a given limit which results from
several analysis processes of the same occurrence.
New tests may be performed within the different
modules of the database, but only after precise
validation by construction and modelling experts.
Efficient and reliable databases are built up in this
way for the domain specific and scene specific
knowledge with optimal premises for analysis and
synthesis within the whole modelling process. An
example object is displayed in Fig. 2 and has been
chosen to verify the proposed concept in the
following paragraphs.
2.2 Hierarchical process structure
In most controlling and modelling systems the
processes are organised in a sequential hierarchical
order. There are three types of processes which
generate and update the results: Basic processes
employ the physical data, which are initialised by
the start up process. In the case of signal processing
e. g. image processing the operators of feature
extraction are filled with initial parameters.
Verifying methods and improving processes take
existing result data and update them to generate
reliable and precise results. The concepts for altering
specific operators or even criterions for evaluation
can be performed by taking into account domain
specific knowledge as in knowledge-based systems,
rather than deriving decisions from data driven
algorithms or optimisation techniques, as in, for
example static modelling algorithms. Verifying
methods can take predefined or resulting data or
decisions as input and try to built up a hypothesis in
one or more system parts, or they can take decisions
that violate specific restrictions and try to eliminate
the constraint violations by help of "iterative repair".
Terminating processes remove inconsistent or non
reliable statements from the amount of hypothesis
and keep the total size of the hypothesis low. The
kernel of the knowledge base does not define the
function of the processes but only their possible
modifications. This gives us complete freedom to
use a broad range of methods encapsulated as
system processes. The connection and the working
structure of the different process modules is realised
by the working result of the other modules.
Figure 2: Example building inside the University of Dortmund, Germany.
ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS
358
3 OBJECT FEATURE
EXTRACTION USING DOMAIN
SPECIFIC CONSTRAINTS
When the acquisition module has terminated, the
primitives of the active sensor are transformed to the
grey level images by 3D projection assuming given
values for height and width as basic hypothesis.
Within the transformation only primitives with a
higher threshold level are taken into account and
missing object data is to be substituted by polygon
approximation or "splining" (Buescher 2000). The
precision grade of this process is sufficient since the
elements are small enough so that reliable solutions
can be expected. A representable subset of resulting
primitives has been extracted from our example
image and the result is displayed in Fig 3. After
passing this state successfully the resulting
primitives can be used to represent the associated
element part as stable fundamental 3D object
element. The goal of this step is the complete
assignment between object primitives and
transformed image primitives with a degree of
reliability that allows the elimination of
contradictions in lower states of the process. The
quality of the individual solutions is dependent on
several parametric constraints : degree of shape and
shading structure of the object, absence of planar
surfaces, texture intensity of the embedded object
elements on the whole scene. The assignment and
the evaluation of the controlling process is
performed by a special neural network with three
layers and four learning steps. The learning effort is
achieved by updating the weights with a subsequent
mean squared error minimisation based on update of
the membership functions.
3.1 Basic concept of the scene analysis
and interpretation
The implemented methods can be divided into two
groups : In the one group the methods can be
transacted rapidly but within a set of rigid
encapsulating envelopes. In the other group the
transaction speed is slower with more dynamic and
complicated structures on the objects. Common to
all of them is the processing of 3D-structures
obtained by feature extraction in the passive sensor
images in the following sequential method parts :
Performing the 3
rd
degree “Sobel“ edge
operator
Emphasizing the interesting object
structures
In order to employ vector orientation
for the different structures B-spline
approximation is applied
Among the different groups of passive sensor
images selection is performed considering domain
and scene specific knowledge.
The environment of every sensor
primitive is analysed regarding to the
viewing aspect. In this way hidden
regions will be excluded.
In the following process only the 3D-
structure elements with parallel or
right-angle orientation will be chosen.
In the next steps the group of selected object
structures as well as the resulting Sobel operator
images form the basic input data and information for
the subsequent modelling process.
3.2 Knowledge based methods for
relaxation and verification of the
scene interpretation
In this section we briefly describe some of the
methods that build up our controlling system. The
methods can be grouped into three categories :
process allocation methods that assign the model
primitives to the image primitives and create an
initial hypothesis of the scene for each aspect.
Structuring methods that take a process allocation as
input and transform the model primitive to every
aspect image. Verifying methods that project groups
of primitives on each aspect or between aspects to
generate better improvements.
3.2.1 Linear algorithms
Searching in its simplest way can be performed by
help of linear operators. The process whose results
fulfil several criterions with optimal values in a
given state s, is being performed and will be
repeated as far as a predefined goal has been
reached. In data driven modelling tasks it is helpful
to describe the best solution by help of an evaluation
function Qe(a,s) and success criterions of fulfilling
the restrictions. Qe(a,s) is to be derived from the
group of evaluation functions of finding the solution
a in s. The restrictions are dependent from the
different states and can be derived from the heuristic
knowledge base. This concept, which is based on
linear search is applied in the basic feature
extraction module. Linear search is easy to perform
and does not waste memory, but it has two
disadvantages : its results may evoke contradictions
or it may be aborted without any useful output. In
(Huang 1996) an uncomplicated modification is
suggested within the basic concept : Both
KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS
359
disadvantages can be avoided if the evaluation
process with its operators and functions can be
controlled dynamically.
3.2.2 Dynamic algorithms
The main goal of the dynamic controlled search
concepts is the creation and adaptation of functions,
that find the best solution for every transition
between the different states of the whole system.
These evaluation methods can be characterised by
the following deterministic cost function :
In dynamic programming, a deterministic control
problem is solved by finding the function V that
assigns the optimal cost V
*
(s) to each state s. A
common way for finding the function V is done by
an iterative method known as value iteration in
which estimates Vi are plugged into the right-hand
side of (1) so that an improved value function Vi +1
is obtained.
4 GENERIC KNOWLEDGE
CREATION PROCEDURE
This new 3D-modelling procedure is derived from a
new recursive genetic algorithm RGA ,explained in
section 5. The kernel of the concept is based on the
consumption that the MEN (Mixed Expert System)
is created step by step introducing additional
knowledge in every new state such that the resulting
knowledge base consists of a reliable part and a new
one, which has to be proofed. The next section
explains how the additional knowledge parts are
linked and proofed to the existing knowledge base
so that the complete structure forms a consistent
platform for reliable decision strategies within an
efficient modelling system. To improve and verify
the automatically generated hypothesis a new
analysis module is derived by applying supervised
learning strategies which optimise and improve the
final generation of the topologic structures. Within
every reasoning decision it is the goal of the self
learning system to add well fitting model parts and
new knowledge elements which satisfy the object
dependent criterions with the best evaluation. In
addition the verification of the knowledge
refinement process embedded in the learning module
is performed by generating the physical model with
the subsequent comparison to the reprojection in the
original images. The evaluation result data can be
transformed to comparable values as feedback of the
recursive algorithm.
[
]
)1()(),()(
*
)(
*
min
a
sAa
sVsacsV +=
Figure 3: Result of the model primitive extraction after the knowledge refinement process
ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS
360
5 EVALUATING THE RESULTS OF
THE LEARNING PROCESS
By updating the restrictions of the refinement
process the MEN can be applied within the model
structure generation in order to employ the necessary
feedback for the evaluation. In case of missing or
unreliable feedback caused by input data noise or
contradictions in the knowledge base a new analysis
by synthesis algorithm derived from the global
model optimisation process has been implemented to
overcome the assignment and correlation problem.
Based on the actual state of the hypothesis the
synthesis algorithm analyses the domain and scene
specific knowledge step by step considering every
update of incoming information until a satisfying
level is reached for every corresponding evaluation.
Because of the complex and dynamic behaviour of
the underlying restrictions (Hepner 1994) the
analysis by synthesis concept cannot directly be
derived from the MEN – algorithm structure
independent from the quality of the input
information. For the modelling and construction of
man made rigid objects considering data driven as
well as knowledge based methods the initial
estimation process could be verified in [Buescher
2000] applying several relaxation algorithms. Its
kernel concept however is derived from a static
dedicated evaluation process based on data driven
optimisation methods and therefor hard to
implement within the discussed modelling or
construction tasks
. The goal of this work is the
introduction and implementation of efficient
modelling concepts derived from the MEN method,
which can be controlled dynamically. Starting from
an initial state, in which every element of the
knowledge base will be considered with the same
reliability, the main modelling process, in which the
new relaxation and regression methods will be
applied, analyse and evaluate every subsequent state
of the model and knowledge base with the goal of
eliminating every contradiction between the
different rules of the knowledge system and
satisfying all given criterions as best as possible.
Applied to the input image in Fig. 1 the result of this
process is displayed in Fig. 3.
6 CONCLUSION
In this paper we described a knowledge-based
solution approach for controlling 3D - Modelling
and reconstruction for multiple different issues in the
presence of several restrictions and requirements.
The problem is of interest both to the academic
community (it generalises the problems studied
earlier) and to modelling experts. It captures
additional restrictions and knowledge that exist in
controlling systems. That means, it is not
inconsistent and will become unstable by what has
been modified during the learning process or due to
the extension of the knowledge process. Selflearning
and –organising methods are the benefit of the
whole system. This capability enables the kernel to
overcome the problems which arise from
contradictions and to reach the goal of a high quality
level. The concretising process and instantiation in
the main system ensure, that the knowledge
algorithms have to share their specific results for an
optimal learning effort.
To the best of our knowledge, this is the first
attempt to solve this general problem. Our solution
approach is to divide the knowledge base into
different levels, linking multiple solution strategies
with each other, building up rules and assigning
them to the functional behaviour. We have
implemented the controlling processes as part of a
knowledge based decision system for controlling 3D
modelling and reconstruction. The knowledge based
system is received in the specific marketplace and
currently being used on several CAD stations and
simulation systems in education and industry.
REFERENCES
Zadeh, L.A., 1994. Fuzzy logic, neural networks, and soft
computing. In Communications of the ACM. vol. 37,
pp. 77-84.
Jang, J.S.R., 1993. ANFIS adaptive network based fuzzy
inference systems. In IEEE Transactions on Systems,
Man, and Cybernetics. vol. 23, pp. 665-685.
Dague, P., Raiman, O., and Deves, P., 1987.
Troubleshooting: When modelling is the difficulty, In
Proceedings of 6th National Conference on Artificial
Intelligence, Seattle, WA, August, pp. 600–605.
Mosterman, P.J., and Biswas, G., 1999 .Diagnosis of
continuous valued systems in transient operating
regions, In IEEE Transactions on Systems, Man, and
Cybernetics, vol. 29, no. 6, pp. 554–565.
Fink, P.K., and Lusth, J.C., 1987 .Expert systems and
diagnostic expertise in the mechanical and electrical
domains, In IEEE Trans.on Systems, Man, and
Cybernetics, vol. SMC-17, pp. 340–349.
Canton, R., Pipitone, F., Lander, W., and M. Marrone,
1983 .Model-based probabilistic reasoning for
electronics troubleshooting, In Proceedings of 8th
IJCAI, pp. 207–211.
Patil, R.S., 1981 .Causal representation of patient illness
for electrolyte and acid-based diagnosis, In Technical
KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS
361
Report, no. 267, Massachussetts Institute of
Technology, Laboratory for Computer Science,
Cambridge, MA.
Huang, Y., and Miles, R., 1996 .Using case-based
techniques to enhance constraint satisfaction problem
solving, In Applied Artificial Intelligence, an
International Journal, vol. 10, no. 4.
Faugeras, O., 1995 .Stratification of 3-D Vision:
Projective, Affine, and Metric Representations, In J.
Optical Soc. Am. A, vol.12, pp. 465-484.
Büscher, E., 2000 .Generation of 3D – Scenes by
knowledge based Adaption of parametric Models, In
Proc. of the IASTED International Conference.
INTERNET AND MULTIMEDIA SYSTEMS AND
APPLICATIONS Nev. USA, pp. 411-417.
Bowyer, K. W. and Dyer, C. R., 1994 .Three-dimensional
shape representation, In Handbook of Pattern
Recognition and Computer Vision, Volume 2:
Computer Vision (T. Y Young, Ed.), New York,
Academic Press, pp.17-51.
Hepner, 1994 .Artificial Neural Network Classification
Using a Minimal Training Set: Comparison to
Conventional Supervised Classification, In
Photogrammetric Engineering and Remote Sensing
56: 469-473.
Kanatani, K., 1996 .Statistical Optimization for Geometric
Computation: Theory and Practice, In Amsterdam:
Elsevier.
ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS
362