KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED

LEARNING ALGORITHMS

Image understanding based on automated knowledge refinement

Eckhard Büscher

Institute of Theoretical Communication Technique, Hannover University,Heckenweg 6 ,32584 Löhne , Germany

Keywords: Multimedia Applications, Signal Processing for Data

Storage, Image and Video Databases, Neural

Networks, Self Organising/Learning, Knowledge refinement and diagnosis

Abstract: This paper discusses the design and implementation of a knowledge based Modelling system KMS, which

combines semantic and rule based approaches in the modelling process. The design and implementation of

the semantic concepts are controlled dynamically to achieve an optimal degree of reality and to employ

efficient interactivity and accessibility for the user. The model-based controlling module is developed to

achieve efficiency and consistence in the basic analysis process, and to avoid the static structure that

frequently occurs in data driven systems. By using a hypothesis and verification scheme in order to ensure

interactivity and accessibility without sacrificing efficiency the KMS evokes the important task of merging

the use of heuristic knowledge in form of a knowledge base with domain specific requirements. By

detecting contradicting and inconsistent rules and by performing tests in the knowledge base and finally by

creating new hypothesis to solve the problems, the controlling process also provides the decision module

with a concept for automated knowledge refinement. This paper focuses on the implementation and

Multimedia adaptation of the learning processes in correlation with the linked databases.

1 INTRODUCTION

The goal of knowledge based modelling and

construction systems is high quality, robustness,

error tolerance and good capabilities for all kinds of

extension. This means within the controlling of a

system temporary or local solutions have to be

avoided, and additionally learning and evaluation

feedback must be performed especially as long as

the learning process proceeds so that complex

problems can be resolved. Consequently this process

results to learning across multiple levels with close

relationship and efficient consumption of knowledge

inside every learning phase. This feature enables the

propagation of efficient learning inside every level

for its subsequent stage, which provides the

controlling process with important decision and

evaluation criterions for learning strategies in the

higher levels. Therefor each resulting model has the

capability of processing different sensor or data

sources with an individual evaluation and

verification instance. Due to the complex domain

and scene specific requirements the integrated MEN

(Mixed Expert Network) system must operate as

data fusion controller. The individual model parts

decide with independent controlling mechanisms,

but create new information for the knowledge base

with extensive correlation on the output side, where

linking and assignment between the different models

is performed, so that the global system reliability can

be evaluated and performance can be increased as

often as it is necessary.

In construction and modelling systems the

rel

ations between experience and knowledge rules

are strongly dependent from the scene and domain

specific requirements. When the domain specific

requirements change, the rules (or behaviours) have

to be changed accordingly. The supervised learning

method based on experience knowledge rules

assignment provides a construction and modelling

system with the malleability for environmental

changes. This feature enables the decision part of

the system to overcome the problem of domain

specific changes (i. e., noisy environments) and to

learn the effect of altering scene specific

requirements. Unlike the behaviour of genetic

knowledge systems, the malleability of synthetic

computer systems is quite limited. Modifications in

algorithmic computer software almost leads to

356

Büscher E. (2004).

KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS - Image understanding based on automated knowledge

reﬁnement .

In Proceedings of the First International Conference on E-Business and Telecommunication Networks, pages 356-362

DOI: 10.5220/0001399203560362

 SciTePress

sensor

acquisition

processing

evaluation

updating

controlling

analysis

Figure 1: Architecture of the knowledge processing

completely different functions and behaviour. This

may lead to the necessity that no alternate than

reprogramming becomes the only possible solution

in most of these cases independent from the degree

of modification within the requirements.

The influence of the environment in

combination with the complexity and structure of the

data can be considered and integrated if the scene

specific and domain specific restrictions can be

expressed in terms of the inference machine

algorithms. In order to obtain the benefit of learning

based knowledge systems concerning reliability and

function this approach aims to provide the

construction and modelling process with the rules of

decision strategies in learning systems. In addition

high level image interpretation can be demonstrated

by help of genetic algorithms, which have to be

derived from the kernel concepts. Accordingly

biological inherit mechanisms which are based on

continuous evolutionary learning could be integrated

into the decision part of the system. In addition self-

learning concepts, genetic decision controlled

algorithms as well as inherit by clustering can be

applied. By help of these important strategies the

system process structure can be organised with the

ability to synchronise and optimise the learning

modules in specific jobs.

However, most of the feature extraction systems

in related work are based on classification methods

(Mosterman, 1999), whereas this proposal focuses

the field of shape detection by merging of active

sensors e. g. SAR (Synthetic Aperture Radar) and

the resulting features of optical image processing to

implement the feature extraction.. Furthermore the

proposed concept does not require any geometrical

information like matching points or height. in

opposite to (Fink 1987), where interferometric data

is required.

2 RELATED WORK AND SYSTEM

STRUCTURE

The operating system distinguishes between three

processing modules. The basic module analyses the

reliability of the active sensors as well as the passive

sensor acquisition and employs methods to

transform data between each other. Processing and

controlling of the active sensor result data is

performed in the second module, especially the

feature extraction of regions with higher density. In

the subsequent module the data analysis and

evaluation for decision is implemented, where the

results of the basic modules are considered. The

outgoing data of each module will be evaluated

under the aspect of emphasising the hypothesis that

fulfils the given constraints. In Fig. 1 the structure of

the whole system connecting the different modules

is displayed.

2.1 Architecture of the knowledge

based controlling and modelling

system

The knowledge based module combines controlling,

processing and behaviour models of the system as

described in (Canton 1983). This module is invoked

if one of the processing parts fails to detect decision

states in a given situation. In several processes it can

KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS

357

be used to detect contradictions in the module states,

and support engineers and modelling experts in

correcting and modifying the system kernel. The

initial rely value for new errors is kept low however,

when modifying or extending the domain specific or

scene specific database with new rules or semantic

elements. When the same restrictions are detected

again, the rely value of these rules will be evaluated

and they will be considered as temporary. They will

only get a continuous attribute, if their rely value

ranges above a given limit which results from

several analysis processes of the same occurrence.

New tests may be performed within the different

modules of the database, but only after precise

validation by construction and modelling experts.

Efficient and reliable databases are built up in this

way for the domain specific and scene specific

knowledge with optimal premises for analysis and

synthesis within the whole modelling process. An

example object is displayed in Fig. 2 and has been

chosen to verify the proposed concept in the

following paragraphs.

2.2 Hierarchical process structure

In most controlling and modelling systems the

processes are organised in a sequential hierarchical

order. There are three types of processes which

generate and update the results: Basic processes

employ the physical data, which are initialised by

the start up process. In the case of signal processing

e. g. image processing the operators of feature

extraction are filled with initial parameters.

Verifying methods and improving processes take

existing result data and update them to generate

reliable and precise results. The concepts for altering

specific operators or even criterions for evaluation

can be performed by taking into account domain

specific knowledge as in knowledge-based systems,

rather than deriving decisions from data driven

algorithms or optimisation techniques, as in, for

example static modelling algorithms. Verifying

methods can take predefined or resulting data or

decisions as input and try to built up a hypothesis in

one or more system parts, or they can take decisions

that violate specific restrictions and try to eliminate

the constraint violations by help of "iterative repair".

Terminating processes remove inconsistent or non

reliable statements from the amount of hypothesis

and keep the total size of the hypothesis low. The

kernel of the knowledge base does not define the

function of the processes but only their possible

modifications. This gives us complete freedom to

use a broad range of methods encapsulated as

system processes. The connection and the working

structure of the different process modules is realised

by the working result of the other modules.

Figure 2: Example building inside the University of Dortmund, Germany.

ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS

358

3 OBJECT FEATURE

EXTRACTION USING DOMAIN

SPECIFIC CONSTRAINTS

When the acquisition module has terminated, the

primitives of the active sensor are transformed to the

grey level images by 3D projection assuming given

values for height and width as basic hypothesis.

Within the transformation only primitives with a

higher threshold level are taken into account and

missing object data is to be substituted by polygon

approximation or "splining" (Buescher 2000). The

precision grade of this process is sufficient since the

elements are small enough so that reliable solutions

can be expected. A representable subset of resulting

primitives has been extracted from our example

image and the result is displayed in Fig 3. After

passing this state successfully the resulting

primitives can be used to represent the associated

element part as stable fundamental 3D object

element. The goal of this step is the complete

assignment between object primitives and

transformed image primitives with a degree of

reliability that allows the elimination of

contradictions in lower states of the process. The

quality of the individual solutions is dependent on

several parametric constraints : degree of shape and

shading structure of the object, absence of planar

surfaces, texture intensity of the embedded object

elements on the whole scene. The assignment and

the evaluation of the controlling process is

performed by a special neural network with three

layers and four learning steps. The learning effort is

achieved by updating the weights with a subsequent

mean squared error minimisation based on update of

the membership functions.

3.1 Basic concept of the scene analysis

and interpretation

The implemented methods can be divided into two

groups : In the one group the methods can be

transacted rapidly but within a set of rigid

encapsulating envelopes. In the other group the

transaction speed is slower with more dynamic and

complicated structures on the objects. Common to

all of them is the processing of 3D-structures

obtained by feature extraction in the passive sensor

images in the following sequential method parts :

• Performing the 3

degree “Sobel“ edge

operator

• Emphasizing the interesting object

structures

• In order to employ vector orientation

for the different structures B-spline

approximation is applied

Among the different groups of passive sensor

images selection is performed considering domain

and scene specific knowledge.

• The environment of every sensor

primitive is analysed regarding to the

viewing aspect. In this way hidden

regions will be excluded.

• In the following process only the 3D-

structure elements with parallel or

right-angle orientation will be chosen.

In the next steps the group of selected object

structures as well as the resulting Sobel operator

images form the basic input data and information for

the subsequent modelling process.

3.2 Knowledge based methods for

relaxation and verification of the

scene interpretation

In this section we briefly describe some of the

methods that build up our controlling system. The

methods can be grouped into three categories :

process allocation methods that assign the model

primitives to the image primitives and create an

initial hypothesis of the scene for each aspect.

Structuring methods that take a process allocation as

input and transform the model primitive to every

aspect image. Verifying methods that project groups

of primitives on each aspect or between aspects to

generate better improvements.

3.2.1 Linear algorithms

Searching in its simplest way can be performed by

help of linear operators. The process whose results

fulfil several criterions with optimal values in a

given state s, is being performed and will be

repeated as far as a predefined goal has been

reached. In data driven modelling tasks it is helpful

to describe the best solution by help of an evaluation

function Qe(a,s) and success criterions of fulfilling

the restrictions. Qe(a,s) is to be derived from the

group of evaluation functions of finding the solution

a in s. The restrictions are dependent from the

different states and can be derived from the heuristic

knowledge base. This concept, which is based on

linear search is applied in the basic feature

extraction module. Linear search is easy to perform

and does not waste memory, but it has two

disadvantages : its results may evoke contradictions

or it may be aborted without any useful output. In

(Huang 1996) an uncomplicated modification is

suggested within the basic concept : Both

KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS

359

disadvantages can be avoided if the evaluation

process with its operators and functions can be

controlled dynamically.

3.2.2 Dynamic algorithms

The main goal of the dynamic controlled search

concepts is the creation and adaptation of functions,

that find the best solution for every transition

between the different states of the whole system.

These evaluation methods can be characterised by

the following deterministic cost function :

In dynamic programming, a deterministic control

problem is solved by finding the function V that

assigns the optimal cost V

(s) to each state s. A

common way for finding the function V is done by

an iterative method known as value iteration in

which estimates Vi are plugged into the right-hand

side of (1) so that an improved value function Vi +1

is obtained.

4 GENERIC KNOWLEDGE

CREATION PROCEDURE

This new 3D-modelling procedure is derived from a

new recursive genetic algorithm RGA ,explained in

section 5. The kernel of the concept is based on the

consumption that the MEN (Mixed Expert System)

is created step by step introducing additional

knowledge in every new state such that the resulting

knowledge base consists of a reliable part and a new

one, which has to be proofed. The next section

explains how the additional knowledge parts are

linked and proofed to the existing knowledge base

so that the complete structure forms a consistent

platform for reliable decision strategies within an

efficient modelling system. To improve and verify

the automatically generated hypothesis a new

analysis module is derived by applying supervised

learning strategies which optimise and improve the

final generation of the topologic structures. Within

every reasoning decision it is the goal of the self

learning system to add well fitting model parts and

new knowledge elements which satisfy the object

dependent criterions with the best evaluation. In

addition the verification of the knowledge

refinement process embedded in the learning module

is performed by generating the physical model with

the subsequent comparison to the reprojection in the

original images. The evaluation result data can be

transformed to comparable values as feedback of the

recursive algorithm.

[

]

)1()(),()(

)(

min

sAa

sVsacsV +=

∈

Figure 3: Result of the model primitive extraction after the knowledge refinement process

ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS

360

5 EVALUATING THE RESULTS OF

THE LEARNING PROCESS

By updating the restrictions of the refinement

process the MEN can be applied within the model

structure generation in order to employ the necessary

feedback for the evaluation. In case of missing or

unreliable feedback caused by input data noise or

contradictions in the knowledge base a new analysis

by synthesis algorithm derived from the global

model optimisation process has been implemented to

overcome the assignment and correlation problem.

Based on the actual state of the hypothesis the

synthesis algorithm analyses the domain and scene

specific knowledge step by step considering every

update of incoming information until a satisfying

level is reached for every corresponding evaluation.

Because of the complex and dynamic behaviour of

the underlying restrictions (Hepner 1994) the

analysis by synthesis concept cannot directly be

derived from the MEN – algorithm structure

independent from the quality of the input

information. For the modelling and construction of

man made rigid objects considering data driven as

well as knowledge based methods the initial

estimation process could be verified in [Buescher

2000] applying several relaxation algorithms. Its

kernel concept however is derived from a static

dedicated evaluation process based on data driven

optimisation methods and therefor hard to

implement within the discussed modelling or

construction tasks

. The goal of this work is the

introduction and implementation of efficient

modelling concepts derived from the MEN method,

which can be controlled dynamically. Starting from

an initial state, in which every element of the

knowledge base will be considered with the same

reliability, the main modelling process, in which the

new relaxation and regression methods will be

applied, analyse and evaluate every subsequent state

of the model and knowledge base with the goal of

eliminating every contradiction between the

different rules of the knowledge system and

satisfying all given criterions as best as possible.

Applied to the input image in Fig. 1 the result of this

process is displayed in Fig. 3.

6 CONCLUSION

In this paper we described a knowledge-based

solution approach for controlling 3D - Modelling

and reconstruction for multiple different issues in the

presence of several restrictions and requirements.

The problem is of interest both to the academic

community (it generalises the problems studied

earlier) and to modelling experts. It captures

additional restrictions and knowledge that exist in

controlling systems. That means, it is not

inconsistent and will become unstable by what has

been modified during the learning process or due to

the extension of the knowledge process. Selflearning

and –organising methods are the benefit of the

whole system. This capability enables the kernel to

overcome the problems which arise from

contradictions and to reach the goal of a high quality

level. The concretising process and instantiation in

the main system ensure, that the knowledge

algorithms have to share their specific results for an

optimal learning effort.

To the best of our knowledge, this is the first

attempt to solve this general problem. Our solution

approach is to divide the knowledge base into

different levels, linking multiple solution strategies

with each other, building up rules and assigning

them to the functional behaviour. We have

implemented the controlling processes as part of a

knowledge based decision system for controlling 3D

modelling and reconstruction. The knowledge based

system is received in the specific marketplace and

currently being used on several CAD stations and

simulation systems in education and industry.

REFERENCES

Zadeh, L.A., 1994. Fuzzy logic, neural networks, and soft

computing. In Communications of the ACM. vol. 37,

pp. 77-84.

Jang, J.S.R., 1993. ANFIS adaptive network based fuzzy

inference systems. In IEEE Transactions on Systems,

Man, and Cybernetics. vol. 23, pp. 665-685.

Dague, P., Raiman, O., and Deves, P., 1987.

Troubleshooting: When modelling is the difficulty, In

Proceedings of 6th National Conference on Artificial

Intelligence, Seattle, WA, August, pp. 600–605.

Mosterman, P.J., and Biswas, G., 1999 .Diagnosis of

continuous valued systems in transient operating

regions, In IEEE Transactions on Systems, Man, and

Cybernetics, vol. 29, no. 6, pp. 554–565.

Fink, P.K., and Lusth, J.C., 1987 .Expert systems and

diagnostic expertise in the mechanical and electrical

domains, In IEEE Trans.on Systems, Man, and

Cybernetics, vol. SMC-17, pp. 340–349.

Canton, R., Pipitone, F., Lander, W., and M. Marrone,

1983 .Model-based probabilistic reasoning for

electronics troubleshooting, In Proceedings of 8th

IJCAI, pp. 207–211.

Patil, R.S., 1981 .Causal representation of patient illness

for electrolyte and acid-based diagnosis, In Technical

KNOWLEDGE BASED 3D-MODELLING BY SELFORGANISED LEARNING ALGORITHMS

361

Report, no. 267, Massachussetts Institute of

Technology, Laboratory for Computer Science,

Cambridge, MA.

Huang, Y., and Miles, R., 1996 .Using case-based

techniques to enhance constraint satisfaction problem

solving, In Applied Artificial Intelligence, an

International Journal, vol. 10, no. 4.

Faugeras, O., 1995 .Stratification of 3-D Vision:

Projective, Affine, and Metric Representations, In J.

Optical Soc. Am. A, vol.12, pp. 465-484.

Büscher, E., 2000 .Generation of 3D – Scenes by

knowledge based Adaption of parametric Models, In

Proc. of the IASTED International Conference.

INTERNET AND MULTIMEDIA SYSTEMS AND

APPLICATIONS Nev. USA, pp. 411-417.

Bowyer, K. W. and Dyer, C. R., 1994 .Three-dimensional

shape representation, In Handbook of Pattern

Recognition and Computer Vision, Volume 2:

Computer Vision (T. Y Young, Ed.), New York,

Academic Press, pp.17-51.

Hepner, 1994 .Artificial Neural Network Classification

Using a Minimal Training Set: Comparison to

Conventional Supervised Classification, In

Photogrammetric Engineering and Remote Sensing

56: 469-473.

Kanatani, K., 1996 .Statistical Optimization for Geometric

Computation: Theory and Practice, In Amsterdam:

Elsevier.

ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS

362