EVOLVING TAKAGI-SUGENO-KANG FUZZY SYSTEMS USING
MULTI POPULATION GRAMMAR-GUIDED GENETIC
PROGRAMMING
Athanasios Tsakonas and Bogdan Gabrys
Smart Technology Research Center, Dept. of Design, Engineering and Computing, Bournemouth University, Poole, U.K.
Keywords:
Genetic programming, Fuzzy rule based systems, Evolutionary computation.
Abstract:
This work proposes a novel approach for the automatic generation and tuning of complete Takagi-Sugeno-
Kang fuzzy rule based systems. The examined system aims to explore the effects of a reduced search space
for a genetic programming framework by means of grammar guidance that describes candidate structures of
fuzzy rule based systems. The presented approach applies context-free grammars to generate individuals and
evolve solutions through the search process of the algorithm. A multi-population approach is adopted for the
genetic programming system, in order to increase the depth of the search process. Two candidate grammars
are examined in one regression problem and one system identification task. Preliminary results are included
and discussion proposes further research directions.
1 INTRODUCTION
The application of fuzzy rule-based systems (FRBS)
has been proven effective in a wide area of domains
and problem tasks (Jang, 1997). Among their im-
plementations, Mamdani-FRBS have been used for
classification tasks, while regression and system iden-
tification application areas have been dominated by
the Takagi-Sugeno-Kang (TSK) -FRBS. In order to
exploit the power of an FRBS, commonly a train-
ing phase takes place with the neuro-fuzzy techniques
most often used. In their simplest approach, these sys-
tems require the pre-determination of the rule-base
size. Advanced approaches may adopt an ad-hoc or
heuristic incremental process to generate the fuzzy
rule base. Others make use of evolutionary techniques
aiming to provide efficient rule-base generation.
Genetic programming (GP) is a successful branch
of evolutionary computation effectively applied in
a wide range of tasks such as symbolic regression
and network design (Koza, 1992). In the past, GP
has been successfully tested for the production of
Mamdani-FRBS (Alba et al., 1996). Consequently,
the perspective for generating TSK-FRBS within GP
framework has always been regarded as a promising
research field, since such a system could combine the
attractive properties of both methods.
In response to the need for such a hybrid, GP
was effectively combined with TSK-FRBS in the past.
These approaches commonly involved partial contri-
bution of GP techniques, such as the assistance by
GP in locating proper membership functions for a
TSK-FRBS (Hoffman and Nelles, 2001) or the co-
evolution of several segments of a TSK-FRBS (Del-
gado et al., 2004). In this paper, we propose an
integrated approach, for the automatic design and
tuning of complete TSK-FRBS for regression and
system identification. To accomplish this, we use
grammars to describe the complete structure of TSK-
FRBS within a GP individual, thus mapping one GP-
individual to exactly one TSK-FRBS. Using this ap-
proach we are able to generate arbitrarily sized TSK-
FRBS which can be tuned using easy to formulate and
implement integrated approach. We furthermore en-
hance our search procedure by incorporating multi-
population architectures to the GP solution pool. We
test two variants of this system in one prediction and
one system identification task.
The content of this paper is organised as follows.
Next section describes the scientific background of
the methods to be used. In Section 3, the design of
the system and its implementation are presented. The
results and discussion related to the system effective-
ness are included in Section 4. The conclusions and
suggestions for further research are drawn in the Sec-
tion 5.
278
Tsakonas A. and Gabrys B..
EVOLVING TAKAGI-SUGENO-KANG FUZZY SYSTEMS USING MULTI POPULATION GRAMMAR-GUIDED GENETIC PROGRAMMING.
DOI: 10.5220/0003637702780281
In Proceedings of the International Conference on Evolutionary Computation Theory and Applications (ECTA-2011), pages 278-281
ISBN: 978-989-8425-83-6
Copyright
c
 2011 SCITEPRESS (Science and Technology Publications, Lda.)

2 BACKGROUND
Fuzzy sets are an extension to the classic (crisp) sets,
where the transition for a value from belonging to a
set and not belonging to the set is gradual and quanti-
fied by a suitably chosen membership function. When
performing fuzzy reasoning, we apply, in a general-
ized form, the traditional two-valued logic, the modus
ponens. More specifically, a fact can be more or less
true, based on the truth of another fact. The general
expression of a rule in a Takagi-Sugeno-Kang (TSK)
fuzzy rule based system has the following form (Jang,
1998):
R
i
: if x
1
is d
k
and ... then y = f(x
1
, ..., x
n
) withC (1)
(i = 1, 2, ..., m), (x
1
, ..., x
n
∈ X), C ∈ [0, 1]
In this formula, C is the certainty factor and X is the
input set.
Genetic programming - GP (Koza, 1992) is a
search methodology belonging to the evolutionary
computation family, which in its canonical form per-
mits the automatic generation of programs. Among
successful evolutionary computation approaches, GP
keeps a significant position due to its beneficial qual-
ities, such as the production of arbitrary-sized so-
lutions and the upholding of population diversity
throughout runs.
Multi population evolutionary models, or island
models, divide the population into sub-populations,
often called demes, and periodically exchange a num-
ber of individuals, simulating migration. In this pa-
per, we adopt a five island model, and we employ a
canonical setup, according to literature (Fern´andez et
al., 2003).
In canonical GP, the type of nodes is implicitly de-
fined by the number of each node’s arguments (Koza,
1992). Hence, usually there are two types of nodes
in the standard GP: a) terminals - non-function nodes;
and b) functions - non-terminal nodes. In order to
produce in an efficient way more complex structures
within GP, such as fuzzy rule based systems, we must
define a stricter hierarchy of the potential architec-
tures. To accomplish this, one of the most efficient
ways is the application of a context-free grammar
(Montana, 1995). The standard notation to express
context-free grammars is in the Backus Naur Form
(BNF).
The idea to incorporate fuzzy inference into GP is
not new. In (Alba et al., 1996) a model that uses a
context-free grammar for the production of Mamdani
fuzzy rule bases has been examined. The system has
proven competitive in various classification tasks. In
(Delgado et al., 2004), a genetic algorithm is proposed
for a co-evolutionary system that produces TSK fuzzy
systems. In (Chen, 2007), the generation of a specific
class of TSK fuzzy systems is accomplished using
evolutionary programming. In (Hoffman and Nelles,
2001), a genetic programming system has been used
to improve a greedy algorithm (LOLIMOT) in search
of data clusters by partitioning the search space.
Our approach aims at the production of TSK fuzzy
rule based systems in the context of multi popula-
tion grammar guided genetic programming. It has
several advantages in comparison to the previous at-
tempts. Firstly, it makes use of the genetic program-
ming framework which has a number of desirable
properties in terms of maintaining population diver-
sity and providing arbitrary solution size. Secondly,
our approach is comparatively simple to implement,
since only one population is maintained, and all fuzzy
rule base parameters are tuned within one system run.
Finally, it is an integrated approach, that generalizes
the applicability of GP for the production of fuzzy
systems by generating complete TSK rule bases for
regression and system identification. The structure of
the TSK fuzzy system is fully described within the GP
functions, and every tuning parameter is computed by
evolutionary means.
3 DESIGN AND
IMPLEMENTATION
We have evaluated the proposed system within two
data domains. The first domain is the Concrete Slump
Test (Yeh, 2008). We have retained 20% of the data
as test set with the remaining 80% used for learning.
The second problem addressed in this work is a sys-
tem identification task. We considered the Mackey-
Glass non-linear chaotic time series, as this is com-
monly used as a test for TSK models (Chen, 2007).
We generated 400 data points out of which 80 were
were used as the test set with the remaining data used
for training.
We examined two system configurations. The first
system evolves a rule base with the rules in the follow-
ing form:
R
i
: if x
1
is µ
k
1
[and x
2
is µ
k
2
...]
then y = f(x
c
) with C (2)
f(x
c
) = w
0
x
n
c
+ w
1
x
n−1
c
+ ... + w
n−1
x
c
+ w
n
(3)
(i = 1, ..., m), (x
1
, ..., x
n
∈ X), C ∈ [0, 1]
The second system evolves a rule base with the rules
in the following form:
R
i
: if x
1
is µ
k
1
[and x
2
is µ
k
2
...]
EVOLVING TAKAGI-SUGENO-KANG FUZZY SYSTEMS USING MULTI POPULATION GRAMMAR-GUIDED
GENETIC PROGRAMMING
279

then y =
f(x
c
1
) f(x
c
2
)
f(x
c
3
) f(x
c
4
)
+ f(x
c
5
) with C (4)
f(x
c
r
) = w
0
x
n
c
r
+ w
1
x
n−1
c
r
+ ... + w
n−1
x
c
r
+ w
n
(5)
(i = 1, ..., m), (x
1
, ..., x
c
n
∈ X), C ∈ [0, 1]
where C is the certainty factor, X is the input set, µ
k
n
are membership functions, and w
1
, ..., w
n
are arith-
metic expressions calculated during evolution. For
expressing the fuzzy relations, Gaussian membership
functions (MF) were selected. In this paper, we se-
lected to use 3 MFs per attribute. Tuning of these
MFs was also applied by evolutionary means. The se-
lection of the variables in each rule is also guided by
evolution. As fitness function, the mean square er-
ror (MSE) was selected. Table 1 summarizes the GP
training parameters.
Table 1: Genetic programming training parameters.
Parameter Value
GP System Grammar-Guided GP
Islands 5
Island topology Ring
Isolation time 50 generations
Migrants number 10 individuals
Migrants type Elite individuals
Total population 1,000 individuals
Selection Tournament
Tournament size 7
Elitism Yes
Crossover rate 0.7
Mutation rate 0.3
Max.individual size 4,000 nodes
Max.generations 500
4 RESULTS AND DISCUSSION
Our test results for the P1 and P2 systems in the Con-
crete Slump Test problem are shown in Table 2. Exact
reconstruction of the test data set was not possible, as
there is not related information in the literature (Yeh,
2008), however we selected to apply the same num-
ber of randomly chosen test records. Our results, in
this example, are given in terms of Pearson correla-
tion coefficient, to allow direct comparison with exist-
ing literature results. The highest reported accuracy in
previous research, for this data set is R
2
=0.922 (Yeh,
2008) where a neural network model was applied.
An example part of the derived rule base of the P1
system has the following form:
if Cement is Low then y = 1.4x
2
+ 1.5x − 9.3,
(x: Water)
if Cement is Medium then y = −2.3x
2
− 0.2x − 8.1,
Table 2: Results for the Concrete Slump Test.
Parameter P1 P2
Size (nodes) 2437 410
Size (rules) 202 9
R
2
0.9127 0.80794
(x: Coarse Aggr.)
if Water is Medium then y = −1.4x
2
− 1.7x − 3.3,
(x: Fly Ash)
As it can be seen, in this problem, the P1 system
compares well with the results reported in the liter-
ature, and in addition the output format allows for
easier interpretation of the results. Although the P2
system achieved similar to P1 scores in our training
and validation set, its performance for the test set was
lower.
The test results for P1 and P2 systems for the
Mackey-Glass data are summarised in Table 3.
Table 3: Results for the Mackey-Glass time series.
Parameter P1 P2
Size (nodes) 1867 457
Size (rules) 98 8
MSE 0.0033591 0.001336
As expected, the higher order P2 model was able
to provide a better approximation to this time-series.
As an example output, one of the rules of the acquired
solution by P2 system has the following form:
if x
4
is Low then y =
f
1
(x
3
) f
2
(x
3
)
f
3
(x
4
) f
4
(x
3
)
+ f
5
(x
2
) where:
f
1
(x
3
) = 1.14x
4
3
+ 0.15x
3
3
+ 0.44x
2
3
+ 0.24x
3
− 0.23,
f
2
(x
3
) = −1.24x
4
3
+0.15x
3
3
+0.719x
2
3
+1.23x
3
+0.73,
f
3
(x
4
) = 0.42x
4
4
− 0.36x
3
4
− 1.22x
2
4
− 0.74x
4
+ 1.11,
f
4
(x
3
) = 4.61x
4
3
− 0.8x
3
3
+ 1.28x
2
3
− 0.63x
3
− 0.25,
f
5
(x
2
) = −1.17x
4
2
+ 1.05x
3
2
+ 0.33x
2
2
− 0.63x
2
+ 0.37.
(x
2
: Slag, x
3
: Fly Ash, x
4
: Water).
Although direct comparison with previous literature
results is not possible due to inability to reconstruct
the exact training data set, past research is shown, for
reference reasons, in Table 4. As it can be seen in
the table, our approach compares well to other models
that do not employ local search methods.
5 CONCLUSIONS AND FURTHER
RESEARCH
This paper presented a system for the generation of
Takagi-Sugeno-Kang fuzzy rule based systems for re-
gression and system identification, by means of ge-
netic programming. The proposed approach carries
ECTA 2011 - International Conference on Evolutionary Computation Theory and Applications
280

Table 4: Results for the MacKey-Glass time series. Re-
gression scores from (Jang, 1997), (Kim and Kim, 1997),
(Wang, 1992) and (Lee and Kim, 1994).
System RMSE
Linear regression model 0.55
Auto regressive model 0.19
Sixth order polynomial 0.04
Back propagation NN 0.02
GA and fuzzy system (5 MFs) 0.049206
GA and fuzzy system (7 MFs) 0.042275
GA and fuzzy system (9 MFs) 0.037873
Wang Product T-norm 0.907
Wang Min T-norm 0.904
ANFIS 0.007
P2-TSK-GP (3 MFs) - this paper 0.036548
several advantages over past related research. Firstly,
the system is capable to automatically produce arbi-
trarily large and complex TSK fuzzy systems, accord-
ing to the needs of a specific problem. Secondly, it
provides flexibility in the selection of non-linear func-
tions fired per rule. Finally, the output model is inter-
pretable by humans in contrast to some other models
like MLPs, since it is in the form of fuzzy rules. In this
paper we have presented preliminary results of the ex-
periments which while focusing on certain character-
istics and capabilities of the GP algorithms produced
encouraging results warranting further investigations.
Further research will be primarily focused on an
advanced grammar design for efficient combinations
of polynomials. Increasing the number of the mem-
bership functions is also expected to improve the ac-
curacy of the system. Integration with ensemble sys-
tems will also be considered.
ACKNOWLEDGEMENTS
The research leading to these results has received
funding from the European Commission within the
Marie Curie Industry and Academia Partnerships and
Pathways (IAPP) programme under grant agreement
n. 251617.
REFERENCES
Alba E., Cotta C., Troya J. M., Evolutionary Design of
Fuzzy Logic Controllers Using Strongly-Typed GP, In
Proc. 1996 IEEE Int’l Symposium on Intelligent Con-
trol New York, NY., 127-132, 1996.
Chen Y., Yang B., Abraham A., Peng L., Automatic Design
of Hierarchical Takagi-Sugeno Type Fuzzy Systems
Using Evolutionary Algorithms IEEE Trans. on Fuzzy
Systems , 15:3, 2007.
Delgado M. R., Zuben F. V., Gomide F. , Coevolutionary
genetic fuzzy systems: a hierarchical collaborative ap-
proach Fuzzy Sets and Systems , Elsevier, 141, 89-106,
2004.
Fern´andez F., Tommassini M.,Vanneschi L., An Empiri-
cal Study of Multipopulation Genetic Programming
Genetic Programming and Evolvable Machines 4:1,
Kluwer, 2003.
Hoffman F., Nelles O., Genetic Programming for Model Se-
lection of TSK-Fuzzy Systems Information Sciences,
Elsevier, 136, 7-28, 2001.
Jang J-S. R., Sun C. T., Mizutani E.,Neuro-Fuzzy and Soft
Computing , Prentice Hall, NJ, 1997.
Jang J-S. R., Neuro-fuzzy modeling for nonlinear dynamic
system identification in: Ruspini E. H. , Bonissone P.
P. , Pedrycz W., eds., Handbook of fuzzy computation ,
Institute of Physics Publishing, Dirak House, Temple
Back, Bristol, UK 1998.
Kim D., Kim C., Forecasting time series with genetic fuzzy
predictor ensemble,IEEE Trans. Fuzzy Syst., 5:4, 523-
535, 1997.
Koza J. R. Genetic programming - On the programming of
computers by means of natural selection The MIT
Press, Cambridge, Massachussets, USA, 1992.
Lee S.-H., Kim I., Time series analysis using fuzzy learn-
ing, inProc. Int’l Conf. Neural Inform. Process., Vol.
6, Seoul, Korea, 1577-1582, 1994.
Montana D. J., Strongly Typed Genetic Programming, Evo-
lutionary Computation, 3:2, 1995.
Wang L. X., Mendel J. M.,Generating fuzzy rules by learn-
ing from examples,IEEE Trans. Systems, Man and Cy-
bernetics, 22:6, 1414-1427, 1992.
Yeh, I-C., Modeling slump of concrete with fly ash and su-
perplasticizer, Computers and Concrete, 5:6, 559-572,
2008.
EVOLVING TAKAGI-SUGENO-KANG FUZZY SYSTEMS USING MULTI POPULATION GRAMMAR-GUIDED
GENETIC PROGRAMMING
281