OPTIMIZATION OF STRUCTURE OF FUZZY-NEURAL

SYSTEMS USING COEVOLUTIONARY ALGORITHM

Zikrija Avdagic, Samir Omanovic, Emir Buza and Belma Cardakovic

Department for Computer Science and Informatics, Faculty of Electrical Engineering in Sarajevo,

Zmaja od Bosne bb, Kampus Univerziteta, 71000, Sarajevo, Bosnia and Herzegovina

Keywords: Coevolutionary algorithm, Fuzzy logic, Neural network, Mamdani fuzzy system, Hybridization.

Abstract: This paper is related to a research of modelling fuzzy-neural systems using the coevolutionary algorithm,

and has the focus on advantages of using the coevolutionary algorithm for system structure optimization. In

the context of this work, the term fuzzy-neural system defines the system that can be used as the fuzzy

system with all its functionalities or as the neural network with all its functionalities. The hybridization of

fuzzy logic, neural networks and coevolutionary algorithm and its architecture are presented in general, and

the role of the coevolutionary algorithm in structure optimization is described in details. Results of testing

with Iris Database, from UCI Machine Learning Repository are also presented. Tests performed during the

research supports the conclusion that usage of the coevolutionary algorithm for the fuzzy-neural system’s

structure optimization is very efficient.

1 INTRODUCTION

This paper presents current results in research of

modelling fuzzy-neural systems using the

coevolutionary algorithm, based on the novel

hybridization of fuzzy logic, neural networks and

coevolutionary algorithm. Focus of this paper is on

advantages of usage of the coevolutionary algorithm

for fuzzy-neural system’s structure optimization. In

the context of this work, the term fuzzy-neural

system defines the system that can be used as the

fuzzy system with all its functionalities or as the

neural network with all its functionalities. In this

hybridization, the coevolutionary algorithm is used

as the primary mechanism for model optimization,

while specific backpropagation (Omanovic and

Avdagic, 2011) is used as the secondary mechanism

and is not explained here in details.

Basically, there are two main approaches in

using coevolutionary algorithm. In the first approach

coevolving species are complete models, while in

the second approach coevolving species are parts of

the one model. The hybridization used in this

research uses the second approach where coevolving

species are parts of the one model. There are similar

usages of the coevolutionary algorithm like the

coevolutionary fuzzy modelling (Pena-Reyes, 2002).

There are many other approaches in combining soft

computing techniques that do not use coevolutionary

algorithm, but have the same purpose (see Abraham,

2005; Kasabov, 2003; Cordón, 2001; Chen and

Abraham, 2010; and many others).

Each approach has its advantages and

disadvantages and this hybridisation try to use

advantages of the cooperative coevolution. The main

idea of the cooperative coevolution is that species

are helping each other in achieving the common

goal. The most important aspect of applying the

coevolutionary algorithm is creating the species in

the way that the model’s structure breakdown is

performed based on logical meanings of structure

elements.

2 COEVOLUTIONARY

ALGORITHM APPLICATION

The fuzzy-neural system’s structure and its logical

breakdown to ++2 coevolving species are

presented on Figure 1, where  is the number of

input fuzzy variables and  is the number of

membership functions of the output fuzzy variable.

125

Avdagic Z., Omanovic S., Buza E. and Cardakovic B..

OPTIMIZATION OF STRUCTURE OF FUZZY-NEURAL SYSTEMS USING COEVOLUTIONARY ALGORITHM.

DOI: 10.5220/0003642001250130

In Proceedings of the International Conference on Evolutionary Computation Theory and Applications (ECTA-2011), pages 125-130

ISBN: 978-989-8425-83-6

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

Figure 1: Architecture of the hybridization.

ECTA 2011 - International Conference on Evolutionary Computation Theory and Applications

126

As can be seen on Figure 1, fuzzy system and

neural network have a full structure mapping.





,…,



are species related to input fuzzy variables,

where one species have a chromosome that contains

all nodes (i.e. membership functions parameters)

related to one input fuzzy variable. 



,…,



are

species related to fuzzy rules. Antecedent parts of

rules with the same consequent part are encoded in

the same chromosome. Since the number of different

consequent parts is equal to the number of

membership functions in output fuzzy variable,

number of species related to fuzzy rules is equal to

the number of membership functions of the output

fuzzy variable. Chromosome for the species 



is composed of nodes (i.e. membership functions

parameters) of the output fuzzy variable. The last of

species – 



is related to active/inactive rules in

the model. This species is not marked on Figure 1

because it cannot be done in adequate way.

Chromosome size in each of species is variable

during the evolution of that species, because the

number of membership functions per fuzzy variable

and number of rules are changed dynamically.

From the rough description of the coevolving

species is possible to see that structure of the fuzzy-

neural system is split to coevolving species based on

logical interrelationships of elements. This is very

important for the coevolution efficiency, which is

shown in the tests results.

2.1 General Characteristics of the

Hybridisation

Hybridisation presented on Figure 1. uses only

triangular fuzzy sets. Fuzzy system is of Mamdani

type. Defuzzification type is centroid. The neural

network is five-layered with a clear mapping of the

fuzzy system into the neural network. The specific

backpropagation, not explained in details here, used

in this hybridisation can perform small adaptations

of the fuzzy-neural system and can be used after

system is built. Although the main optimization

mechanism is coevolutionary algorithm,

backpropagation is used partially during system

building. These two mechanisms interleave during

system’s model optimization as global and local

mechanism.

These characteristics are chosen to support

following goals:

 Fuzzy presentation of knowledge learned, that

enables its easier interpretability;

 Smaller size of the resulting fuzzy system’s model

comparing to Sugeno since “the number of the input

fuzzy sets and fuzzy rules needed by the Sugeno

fuzzy systems depend on the number and locations

of the extrema of the function to be approximated”

(Sivanandam et all, 2007);

 The resulting system can be used as fuzzy system

with all its functionalities or as neural network with

all its functionalities including the backpropagation.

2.2 Coevolution Characteristics

From the previous explanation of the species that are

in coevolution it is obvious that complexity of the

optimization problem is high. Besides that,

implementation this hybridisation generates the

fuzzy-neural system from data automatically, which

additionally makes the problem more difficult. But

this coevolution organization has species with

relatively small chromosomes comparing to similar

cases where less species is used for fuzzy system’s

model optimization.

Each optimization algorithm has its mechanisms

to skip local optimums and search for better

solution. In the case of this hybridization, the

coevolutionary algorithm is implemented with

following mechanisms and with following

behaviours:

 Coevolution of species. Coevolution of species is

implemented in the classical manner where each of

species has its own evolution up to the fitness

calculation. For fitness calculation, target member

and chosen cooperators from the rest of the species

are joined to form fuzzy-neural system. That fuzzy-

neural system (as neural network) is shortly trained

via the specific backpropagation to increase the

quality of the model and to measure its performance

with the training dataset. Behaviour of the

coevolution of species mainly depends on input

parameters, but also on additional mechanisms:

In the case of the very weak progress mechanism

of random generation of members is used along with

the basic mechanisms – crossover and mutation.

In the case of the weak progress, some of parameters

of the coevolution are change dynamically.

 Coevolution of two groups of species. This type of

coevolution is based on the analogy with the

coevolution of two species in coevolutionary fuzzy

modelling (Pena-Reyes, 2002). While in

coevolutionary fuzzy modelling there are only two

species – one related to fuzzy variables and one

related to fuzzy rules, in the coevolution of two

groups of species one of groups contains species

related to fuzzy variables while the other one contain

species related to fuzzy rules.

Except the analogy, the rest of behaviour is

different. Figure 2 shows how it works. At the start

OPTIMIZATION OF STRUCTURE OF FUZZY-NEURAL SYSTEMS USING COEVOLUTIONARY ALGORITHM

127

co-operators from all species are selected. Then, one

of two groups is randomly selected. All members,

from all of the species of the selected group, are

simultaneously mutated. Then members from all of

the species (mutated and non mutated) are integrated

to create fuzzy-neural system (FNS on figure 2).

This fuzzy-neural system (as neural network) is

shortly trained via the specific backpropagation to

increase the quality of the model and to measure its

performance with the training dataset. After that

Matlab fuzzy inference system (FIS on figure 2)

object (MathWorks, 2009) is created from the fuzzy-

neural system. That FIS object is then validated with

the validation dataset. Based on the results of the

short training and the validation, fitness calculation

is performed. At the end fitness is assigned to each

of the mutated members and new, mutated members

are placed in the population. If the fitness of the

newly created fuzzy-neural system is the best one,

then it is recorded as the best known.

Coevolution of species and coevolution of two

groups of species are combined in the way that the

coevolution of species is a basic mechanism and the

coevolution of two groups of species is an additional

mechanism that helps escaping from local

optimums.

Fitness calculation is performed in three levels.

Each of levels has its purpose and influence on the

optimization process. Levels are following:

 The first level controls the overfitting. It holds

percentage results of training and validation near to

each other. That means that training progress is not

allowed if it is not followed by validation progress.

 The second level is used to maximize the total

accuracy. It is a sum of the accuracy of the training

and the accuracy of the validation.

 Third level is used to minimize the total number of

fuzzy rules in the model and to minimize the

absolute average error.

Levels have importance, which means that the first

level is more important than the second, and the

second is more important than the third.

2.3 Tests

Implementation of this hybridization is done in

MATLAB. Tests were performed with databases

from UCI Machine Learning Repository

(http://archive.ics.uci.edu/ml). Irvine, CA:

University of California, School of Information and

Computer Science). In this paper are presented some

results of tests with Iris database. Table 1 shows one

of tests that were composed of ten (10) runs with sa-

Figure 2: Coevolution of two groups of species.

ECTA 2011 - International Conference on Evolutionary Computation Theory and Applications

128

Table 1: Results of one of tests with Iris database.

Run

Training+validation

result

Testing result Overall result

Number of

fuzzy rules

Comment

1. 128/130=98.46% 19/20=95.00% 147/150=98.00% 3

2. 125/130=96.15% 20/20=100.00% 145/150=96.67% 4

3. 122/130=93.85% 19/20=95.00% 141/150=94.00% 2

4. 128/130=98.46% 20/20=100.00% 148/150=98.67% 3

the best result

5. 126/130=96.92% 19/20=95.00% 145/150=96.67% 2

6. 130/130=100.00% 18/20=90.00% 148/150=98.67% 3

7. 126/130=96.92% 18/20=90.00% 144/150=96.00% 2

8. 121/130=93.08% 15/20=75.00% 136/150=90.67% 2 the worse result

9. 126/130=96.92% 20/20=100.00% 146/150=97.33% 4

10. 126/130=96.92% 18/20=90.00% 144/150=96.00% 2

Average 96.77% 93.00% 96.27% 2.7

same parameters. Since the Iris database is small, it

was split on training, validation and testing subsets

with ratios 11:2:2, respectively. Population size for

each of species was 50. Number of generations is

used as termination criteria and it was set to 1000

generations of the coevolution. This way all runs are

done under the same experimental conditions.

From the results in Table 1, and based on

characteristics of the hybridisation used in tests, and

parameters settings for this test, it is possible to note

the following:

 Obtained results in each run shows capability for

high classification of the generated fuzzy-neural

system. Having in mind that the number of

generations is limited, that only triangular

membership functions are used (less precise) and

that the fuzzy system is of Mamdani type (less

precise) then the classification result is high.

 Results were obtained in 1000 generations which

is near to similar usages of the coevolutionary

algorithm (Pena-Reyes, 2002) but without setting

any predefined element of the structure of the fuzzy-

neural system at the beginning. This shows the

strength of the presented approach in species

creation and automatic building of the system from

data.

 The fuzzy-neural system is build easily by setting

few parameters of the execution and providing the

dataset, without providing the initial structure of the

fuzzy-neural system. All is build from data

automatically. From the optimization point of view

this is harder problem than to optimize the structure

of the system that have some predefined elements.

 The overfitting is held well under control in most

of the runs, which is not easy with the small

database. This shows a good generalization

capability. The overfitting control results are a very

important fact for deciding which is the best build

system.

3 CONCLUSIONS

The hybridization of fuzzy logic, neural network and

coevolutionary algorithm is a very efficient

approach for the automatic generating the fuzzy-

neural system from data, whose main strength is in

usage of the cooperative coevolution in the way

presented earlier in this paper.

Cooperative coevolution implemented in this

way is a very good global search mechanism. When

more the two species are used in the coevolution

then mutation in one of the species makes a small

change and escaping from local optimums is not

fast, but with the coevolution of two groups of

species escaping from local optimums is faster

because several species simultaneously have a

mutation.

4 FURTHER WORK

Our special interests are hybridisations of soft

computing techniques, especially hybridisations that

include the cooperative coevolution. We will

continue to improve this hybridization of fuzzy

logic, neural network and coevolutionary algorithm,

and try to make some practical real-life applications

and build some systems in medicine and

bioinformatics. Coevolution of more than two

species is very interesting for parallel or distributed

implementation and that is also one of directions of

our interest.

OPTIMIZATION OF STRUCTURE OF FUZZY-NEURAL SYSTEMS USING COEVOLUTIONARY ALGORITHM

129

REFERENCES

Omanovic, S., Avdagic Z., 2011. Modeling of Fuzzy-

Neural Systems Using the Coevolutionary Algorithm.

In WSEAS’11, 12th International Conference on Fuzzy

Systems. WSEAS Press.

Pena-Reyes, C., 2002. Coevolutionary Fuzzy Modeling –

doctoral thesis, EPFL. Lausanne.

Nedjah, N., et al. (Eds.), 2005. Fuzzy System Engineering:

Theory and Practice, Chapter 3 - Adaptation of Fuzzy

Inference System Using Neural Learning (A.

Abraham), pp. 53-83, Springer. Verlag.

Kasabov, N., 2003. Evolving Connectionist Systems -

methods and applications in bioinformatics, brain

study and intelligent machines, Springer. Verlag.

Cordón, O., et al., 2001. Genetic Fuzzy Systems –

Evolutionary Tuning and Learning of Fuzzy

Knowledge Bases, World Scientific Publishing Co.

Pte. Ltd. Singapore.

Chen, Y., Abraham, A., 2010. Tree-Structure Based

Hybrid Computational Intelligence, Springer. Verlag.

Sivanandam, S. N., Sumathi, S., Deepa, S. N., 2007.

Introduction to Fuzzy Logic using MATLAB,

Springer. Verlag.

MathWorks, Inc., 2009. MATLAB Help.

ECTA 2011 - International Conference on Evolutionary Computation Theory and Applications

130