SOME SPECIAL HEURISTICS

FOR DISCRETE OPTIMIZATION PROBLEMS

Boris Melnikov

, Alexey Radionov

Department of Mathematics and Information Science,Togliatti State Univ., Belorusskaya str., 14, Togliatti, 445667, Russia

Viktor Gumayunov

Department of Telecommunication Software,Ulyanovsk State Univ., L.Tolstoy str., 42, Ulyanovsk,432700, Russia

Keywords: Anytime algorithm, discrete optimization problem, local heuristics, minimization, nondeterministic

finite automata, disjunctive normal forms

Abstract: In previous paper we considered some heuristic methods of decision-making for various discrete

optimization problems; all these heuristics should be considered as the combination of them and form a

common multi-heuristic approach to the various problems. And in this paper, we begin to consider local

heuristics, which are different for different problems. At first, we consider two problems of minimization:

for nondeterministic finite automata and for disjunctive normal forms.

Our approach can be considered

as an alternative to the methods of linear programming, multi-agent optimization, and neuronets.

1 INTRODUCTION

In previous papers (see (Melnikov, 2005), and also

some papers in Russian) we considered some

heuristic methods of decision-making for various

discrete optimization problems (DOP). In fact, all

these heuristics should be considered as the

combination of them and form a common multi-

heuristic approach to the various DOP. And in this

paper, we consider not the common heuristics

(which can be applied to various DOP), but so called

local heuristics, which are own for different

problems. Let us remark, that in (Melnikov, 2005)

we already considered also some local heuristics (for

a problem of considered in this paper, i.e., for

minimization of automata).

But at first, let us briefly describe the common

set of heuristics of (Melnikov, 2005). The object of

each of considered problems is programming

anytime algorithms.

• We use some modifications of truncated branch-

and-bound method (B&B).

• For the selecting immediate step, we apply

dynamic risk functions.

• Simultaneously, for the selection of coefficients

of the averaging-out, we use genetic algorithms.

• And the reductive self-learning by the same

genetic methods is used for the start of truncated

B&B.

Thus, this combination of heuristics represents a

special approach to construction of anytime-

algorithms for the discrete optimization problems,

which is an alternative to the methods of linear

programming, multi-agent optimization, and

neuronets.

2 CONSIDERED PROBLEMS

Thus, as we said before, the main object of this

paper is local heuristics, and each of them belongs to

its own area. Therefore let us describe considered

DOP.

At first, we consider some connected problems

of minimization for nondeterministic finite Rabin-

Scott automata (NFA). Probably, the main for them

is state-minimization, i.e., the problem of

constructing NFA, which defines the given regular

language and has minimum possible number of

Author was partially supported by the Russia

Foundation of the Basic Research (project No.04-01-

00863).

360

Melnikov B., Radionov A. and Gumayunov V. (2006).

SOME SPECIAL HEURISTICS FOR DISCRETE OPTIMIZATION PROBLEMS.

In Proceedings of the Eighth International Conference on Enterpr ise Information Systems - AIDSS, pages 360-364

DOI: 10.5220/0002469803600364

 SciTePress

states. Since (Kameda and Weiner, 1970), there are a

few changes in description of the exact algorithms

for this problem: all the algorithms are exponential

relative to the number of states of considered NFA.

The last argument is true because all the algorithms

need to construct equivalent automaton of canonical

form (or, maybe, some similar graphs or other

objects). Let us remark, that from the point of view

of the theory of complexity of algorithms, all the

published algorithms (Kameda and Weiner, 1970;

Jiang and Ravikumar, 1993; Melnikov, 2000; etc.)

are equivalent. However we hope, that the approach

of authors of this paper (Melnikov, 2000) allows

formulating some heuristics for anytime algorithms.

Second, it is the problem of minimization of

disjunctive normal forms (DNF). The exact

algorithms for this minimization are obtained for

ages (and are considered in the classical textbooks,

for example, in the Russian textbook for first-year

students (Yablonskiy, 1979), which is used more

than 25 years), however the computer programs

making on basis of such solutions cannot work in

real time even for the number of variables, which is

equal to 20, except, certainly, for a lot of trivial

cases. The author does not know books, where any

anytime algorithms for this problem are obtained,

however, if such papers do exist, the approach of

this paper, certainly, can be used in some alternative

versions of computer programs.

In this paper, we shall consider local heuristics

for two the described problems. However, let us

briefly describe the two other problems, for which

we are going to publish local heuristics in the next

papers.

Thus, the third problem is the classical travelling

salesman problem (TSP; see (Hromkovič, 2003),

etc.); certainly, universal methods for solving TSP

simply cannot exist. Some last years, authors of

papers for heuristic methods of TSP-solution

consider most often so called metric TSP. For their

solving, some methods of linear programming and

multi-agent optimization are used; (Hromkovič,

2003; Dorigo and Gambardella, 1997; Johnson and

McGeoch, 1997; etc). However, some variants of the

classical B&B can also be used not only for the

exact (optimal) solution of considered TSP, but also

for quasi-optimal heuristic solutions. At first, such

approach can be used for the quasi-metric TSP

(Melnikov and Romanov, 2001).

And the fourth problem is the special problem

for graph transformation algorithms. Considering

weighted oriented graphs, we formulate some

special rules for combining their vertices. And the

goal is to obtain graph having minimum possible

number of edges. For details, see (Belozyorova and

Melnikov, 2005) and the references from that paper.

3 SOME LOCAL HEURISTICS

FOR THE NFA-MINIMIZATION

PROBLEM

In this Section, we consider a heuristic algorithm for

forming quasi-optimum covering; let it be Q defined

in (Melnikov, 2000). For this thing, we shall select a

subset of blocks (grids) of a matrix, which cells are

corresponding to elements of special binary relation

#, and this relation can be construct on the base of

the given NFA. (See details also in (Melnikov,

2000).)

The considered algorithm is based on a special

modification of truncated B&B. It differs from the

classical truncated B&B that we do not divide the

considered problem (and, therefore, the searching

space) into the left and right ones. The whole

searching space corresponds to the whole set of

blocks, but for the first step, it is only the considered

matrix of binary relation #.

But the practical programming for the classical

truncated B&B gave the poor results. The main

obstacle is that we hardly can fixed the fact that the

considered block does not belong to the anytime

solution. Besides, we can estimate only the cells, not

the blocks (see such heuristics in (Melnikov, 2005)).

And procedure of constructing blocks is the

particular heuristic sub-problem; we are going to

describe corresponding algorithms in the next paper.

Therefore, we have to use the following

modification of the truncated B&B.

1. Considering the next problem of the

searching space, we select a cell using the

heuristics A (see below).

2. Using the heuristics B (see below), we

construct the set of blocks M. Each of

these blocks contains the selected cell. Let

the number of blocks given by algorithm

B be N.

3. The considered problem is divided into N

ones. In each of obtained problem, we

suppose that the next considered block

belongs to Q, and other N-1 blocks do not

belong to Q.

4. Returning to the step 1. (Or exiting, if the

searching space is empty.) ■

This heuristic algorithm is based on the

following example. Let us have the considered

problem T, and after the heuristics A and B (see

below), we obtain a set of blocks (let they be b1, b2

and b3) for the next branching. Then we divide the

problem T for 3 ones (T1, T2 and T3, branching by

b1, b2 and b3 correspondingly). And we use the fact

that b1 could hardly be included in the set of blocks

solving the problems T2 and T3, etc.

SOME SPECIAL HEURISTICS FOR DISCRETE OPTIMIZATION PROBLEMS

361

Heuristics A (selecting the cell). We select the

cell having the maximum possible sum of the

numbers of cells in the same row or in the same

column, which are not included in the current

answer. (Remark that in (Melnikov, 2005), we

considered some more complicated heuristics for

this thing.) ■

Heuristics B (selecting the block). Let us choose

the set of blocks containing the cell of the row r and

the column c.

1. First, let us construct the following quasi-

block.

a) Excluding columns, which have 0

in the position r. In the same way,

we also exclude rows.

b) Using algorithm A (or its

modification of (Melnikov, 2005))

for estimating remaining columns

and rows, such that the estimation

is the sum of their values given by

algorithm A. Certainly, we

consider only cells, which values

are equal to 1.

c) Proving that there exist two

columns or two rows, which have

the different estimations.

(Otherwise, we already obtained

the quasi-block, i.e., all the

corresponding cells have values 1.

In this case, we add this quasi-

block to the set of them and return

to step b.)

d) Excluding the row or the column

having the minimum estimation.

e) Returning to the step b.

2. For each constructed quasi-block, let us

extend the number of rows and/or

columns, such that all the corresponding

values are equal to 1. Thus, we can obtain

1 or 2 blocks.

3. All the blocks constructed of the step 2

form the final set of blocks. ■

Certainly, for the successful execution, B&B

needs not only heuristics for branching (i.e.,

heuristics A and B described before), but also some

more heuristics:

• for calculating the bounds (C1 and C2);

• for the quick addition for the set of blocks (D);

• and for quick transformating the quasi-block

into the block (E).

Let us remark in advance, that we do not describe

heuristics E in this paper, and heuristics C2 will be

described briefly. Their detailed description, and

also the detailed description of some complicated

heuristics which are the alternative to the simplest

heuristics C1 given below, is the subject of the next

paper.

Heuristics C1 (calculating upper bound). The

upper bound is the maximum possible value of

blocks, which are obtained for the ending of

calculating the considered problem.

We count the number of rows which contain at

least one value 1; also we count the number of such

columns. The answer is the minimum of two these

values. ■

Heuristics C2 (calculating lower bound).

Similarly to C1, the lower bound is the minimum

possible value of blocks, which are obtained for the

ending of calculating the considered problem.

For calculating this value, we use another

heuristics for constructing special set of cells, for

which each their pair cannot belong to the same

block. (Let us remark, that there exist, in general,

more than 1 such sets of cells, but the mentioned

heuristics constructs the only one.s) The number of

such cells is the lower bound. ■

Certainly, the solved subproblem can be called

unpromising if its lower bound is equal or more than

minimum of the upper bounds of all the existing

subproblems. Such subproblem can be excluded

from the set of subproblems to be considered.

Heuristics D (the quick addition for the set of

blocks). Two of the possible goals of applying this

heuristics are the following: to make the upper

bound; to make a sequence of the left problems (for

the last thing, see (Melnikov, 2005)).

Thus, the simplest heuristics is the same as

heuristics C1. ■

4 SOME LOCAL HEURISTICS

FOR THE DNF-MINIMIZATION

PROBLEM

Instead of blocks, we consider here the planes; each

plane has dimension in the interval from 1 to the

given number of variables N. At first sight, we have

to construct all the planes, for which all the values of

minimized function are equal to 1. (After this

constructing, B&B can start.) However, all the well-

known algorithms for constructing such planes are

too long (Birkhoff and Bartee, 1999; Lee and

Markus, 1967; Yablonskiy, 1979; etc) – unlike the

NFA-minimization problem.

Really, if we use algorithms which obtain planes

in decreasing order of their dimension, then we

obtain that the time is O(4

). Let we have N

variables and M sets of their values (corresponding

to the sets of coordinates), where the minimized

function is equal to 1. Then infilling the array

corresponding planes (the number of planes is 3

)

ICEIS 2006 - ARTIFICIAL INTELLIGENCE AND DECISION SUPPORT SYSTEMS

362

requires O(M

•2

) units of time. Because at the worst

M =2

N-1

, we obtain that the required time is O(4

Certainly, we exclude the planes belonging to

other ones. The operation of such excluding require

the time, which only linearly depends on N. But

such procedure does not solve formulating problem

completely.

Besides, if we use algorithms which obtain

planes in decreasing order of their dimension, then

the appearance of the first plane usually needs a lot

of time; but it is a practical result, it hardly could be

rigorously proven.

The algorithms which obtain planes in ascending

order of their dimension also do not solve the

problem, although the time estimating is here O(3

);

this estimating is simply obtained, e.g., by

realization algorithms of (Birkhoff and Bartee, 1999;

Yablonskiy, 1979). This time estimating is some

better, but also too long. However, the immediate

start of B&B is here uninteresting, because there is

unlikely that even the pseudo-optimal answer

(pseudo-optimal DNF) contains planes having little

dimensions.

However, the heuristics for the immediate start

of B&B does exist. The detailed description of this

heutristics is the subject of the special paper, we

shall describe it briefly.

Thus, we set for this thing the following local

goal: to construct the sets of considered subsets of

the given planes, which intersections are minimum

possible. The bound is here the power of intersection

sets. And the indication of anytime decision is the

absence of the sets of coordinates, for which the

value of the given function is equal to 1.

Then we select an arbitrary value 1 from the

given set of multidimensional cube corner. For it, we

construct the plane containing it and having

maximum possible dimension. (Such algorithm of

constructing plane is similar to considered in Section

3 for the selecting block.) Certainly, we can obtain

more than one planes. Then for each of these planes,

we exclude then sets of coordinates belonging to this

plane. It is important to remark, that each of them

will form the different sub-problem, and, therefore,

we can start truncated B&B before we have

constructed the whole set of planes. Besides, such

algorithm allow to obtain planes of big dimension,

e.g., we often do not consider intersections of them.

We use here the following estimation for the

bounds. The high bound is the number of planes in

the quasi-optimum DNF (i.e., of the best DNF of the

considered sub-problem). And the low bound is the

same value for the considered DNF.

And, as we said before, the heuristics for

minimization of DNF (unlike minimization of NFA)

are given here very briefly. We are going to describe

this thing more detailed in the next paperз.

5 SOME PRELIMINARY

PRACTICAL RESULTS

While testing, we set the time for our anytime

algorithm (see the tables). We also set the dimension

of the problem – i.e., the number of rows for NFA

(the number of columns depends of the last value

also by special variate) and the number of variables

for DNF – not the numbers of grids for NFA and

planes for DNF, the last values are also special

variates depending on previous ones.

The clock speed of the computer was about 2.0

GHz. If we choose the time under 10 minutes, we

make the averaging-out by 50 or more solutions.

And the values of cells have the following

meaning. For each cell, we made corresponding

tests. For each test, we set the number of

grids/planes for the given problem (certainly, we did

not use this information in the program) and obtain

the value of grids/planes found by anytime

algorithm. Then we counted comparative

improvement of this value (+) or the worsening (–).

The possibility of positive values is the corollary of

the fact, that, e.g. for the DNF, two planes of

dimension k could form one plane of dimension k+1.

The values were averaged; they are written in the

table in percents. (I.e., +0.20 means that the mean

value is better than the a priori given than 0.2%.)

Thus, below are the practical results.

NFA 20–23 40–45 60–65 80–90

01 sec –1.76 –0.58 –0.02 –0.02

10 sec –0.55 –0.17 +0.22 +0.20

01 min –0.03 +0.45 +1.00 +1.06

10 min 0 +1.06 +1.07 +1.20

01 h 0 +1.07 +1.17 +1.21

DNF 20–22 25–27 30–33

01 sec –8.5 –2.2 –1.9

10 sec –1.21 –0.70 –0.43

01 min –0.73 –0.65 –0.43

10 min –0.03 –0.01 –0.01

01 h –0.03 –0.01 0

Thus, the obtained results are near to 100%; this

fact shows that the approach proposed in this paper

could be applied in the future. And in the next

papers, we are going to give the practical results for

two other problems mentioned in Section 2.

REFERENCES

Belozyorova, A., and Melnikov, B., 2005. “Applying the

set of heuristics in the problem of constructing scheme

of nuclear transformations”, 2nd Conference

SOME SPECIAL HEURISTICS FOR DISCRETE OPTIMIZATION PROBLEMS

363

“Methods and instrument of the information

processing”, Russia, Moscow State Univ. Ed. (2005)

208–212 (in Russian).

Birkhoff, G., and Bartee, T., 1999. Modern Applied

Algebra, McGraw-Hill, N.Y., 1999.

Dorigo, M., and Gambardella, L., 1997. “Ant Colony

System: A Cooperative Learning Approach to the

Traveling Salesman Problem”, IEEE Transactions on

Evolutionary Computation, Vo.1, No.1 (1997) 53–66.

Hromkovič, J., 2003. Algorithms for Hard Problems,

Springer, 2003.

Jiang, T., and Ravikumar, B., 1993. “Minimal NFA

Problems are Hard”, SIAM J. Comput., Vo.22 (1993).

Johnson, D., and McGeoch, L., 1997. “The Traveling

Salesman Problem: A Case Study in Local

Optimization”, in “Local Search in Combinatorial

Optimization”, eds E.Aarts, J.Lenstra, John Wiley Ed.,

1997, 215–310.

Kameda, T., and Weiner, P., 1970. “On the State

Minimization of Nondeterministic Finite Automata”,

IEEE Trans.on Computers, C-19 (1970) 617–627.

Lee, E., and Markus, L., 1967. Foundation of Optimal

Control Theory. Wiley, 1967.

Melnikov, B., 2000. “Once more about the state-

minimization of the nondeterministic finite automata”,

The Korean Journal of Computational and Applied

Mathematics, Vo.7, No.3 (2000) 655–662.

Melnikov B., and Romanov, N., 2001. “Once more on the

heuristics for the traveling salesman problem”, Russia,

Saratov State Univ. Ed., “Theoretical informatics and

its applying”, Vo.4 (2001) 81–92 (in Russian).

Melnikov, B., 2005. “Discrete Optimization Problems –

Some New Heuristic Approaches”, Conference HPC-

Asia-2005, IEEE Computer Society Press Ed., 2005.

Yablonskiy, S., 1979. Introduction into discrete

mathematics, Moscow, Nauka Ed., 1979 (in Russian).

ICEIS 2006 - ARTIFICIAL INTELLIGENCE AND DECISION SUPPORT SYSTEMS

364