BINARY OPTIMIZATION: A RELATION BETWEEN THE DEPTH OF A LOCAL MINIMUM AND THE PROBABILITY OF ITS DETECTION

B. V. Kryzhanovsky, V. M. Kryzhanovsky, A. L. Mikaelian

2007

Abstract

The standard method in optimization problems consists in a random search of the global minimum: a neuron network relaxes in the nearest local minimum from some randomly chosen initial configuration. This procedure is to be repeated many times in order to find as deep an energy minimum as possible. However the question about the reasonable number of such random starts and whether the result of the search can be treated as successful remains always open. In this paper by analyzing the generalized Hopfield model we obtain expressions describing the relationship between the depth of a local minimum and the size of the basin of attraction. Based on this, we present the probability of finding a local minimum as a function of the depth of the minimum. Such a relation can be used in optimization applications: it allows one, basing on a series of already found minima, to estimate the probability of finding a deeper minimum, and to decide in favor of or against further running the program. The theory is in a good agreement with experimental results.

References

  1. Amit, D.J., Gutfreund, H., Sompolinsky, H., 1985. Spinglass models of neural networks. Physical Review A, v.32, pp.1007-1018.
  2. Fu, Y., Anderson, P.W., 1986. Application of statistical mechanics to NP-complete problems in combinatorial optimization. Journal of Physics A. , v.19, pp.1605- 1620.
  3. Hartmann, A.K., Rieger, H., 2004. New Optimization Algorithms in Physics., Wiley-VCH, Berlin.
  4. Hopfield, J.J. 1982. Neural Networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci.USA. v.79, pp.2554- 2558 .
  5. Hopfield, J.J., Tank, D.W., 1985. Neural computation of decisions in optimization problems. Biological Cybernetics, v.52, pp.141-152.
  6. Huajin Tang; Tan, K.C.; Zhang Yi, 2004. A columnar competitive model for solving combinatorial optimization problems. IEEE Trans. Neural Networks v.15, pp.1568 - 1574.
  7. Joya, G., Atencia, M., Sandoval, F., 2002. Hopfield Neural Networks for Optimization: Study of the Different Dynamics. Neurocomputing, v.43, pp. 219-237.
  8. Kryzhanovsky, B., Magomedov, B., 2005. Application of domain neural network to optimization tasks. Proc. of ICANN'2005. Warsaw. LNCS 3697, Part II, pp.397- 403.
  9. Kryzhanovsky, B., Magomedov, B., Fonarev, A., 2006. On the Probability of Finding Local Minima in Optimization Problems. Proc. of International Joint Conf. on Neural Networks IJCNN-2006 Vancouver, pp.5882-5887.
  10. Kwok, T., Smith, K.A., 2004. A noisy self-organizing neural network with bifurcation dynamics for combinatorial optimization. IEEE Trans. Neural Networks v.15, pp.84 - 98.
  11. Perez-Vincente, C.J., 1989. Finite capacity of sparcecoding model. Europhys. Lett., v.10, pp.627-631.
  12. Poggio, T., Girosi, F., 1990. Regularization algorithms for learning that are equivalent to multilayer networks. Science 247, pp.978-982.
  13. Salcedo-Sanz, S.; Santiago-Mozos, R.; Bousono-Calzon, C., 2004. A hybrid Hopfield network-simulated annealing approach for frequency assignment in satellite communications systems. IEEE Trans. Systems, Man and Cybernetics, v. 34, 1108 - 1116 Smith, K.A. 1999. Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research. INFORMS Journal on Computing v.11 (1), pp.15-34.
  14. Wang, L.P., Li, S., Tian F.Y, Fu, X.J., 2004. A noisy chaotic neural network for solving combinatorial optimization problems: Stochastic chaotic simulated annealing. IEEE Trans. System, Man, Cybern, Part B - Cybernetics v.34, pp. 2119-2125.
  15. Wang, L.P., Shi, H., 2006: A gradual noisy chaotic neural network for solving the broadcast scheduling problem in packet radio networks. IEEE Trans. Neural Networks, vol.17, pp.989 - 1000.
Download


Paper Citation


in Harvard Style

V. Kryzhanovsky B., M. Kryzhanovsky V. and L. Mikaelian A. (2007). BINARY OPTIMIZATION: A RELATION BETWEEN THE DEPTH OF A LOCAL MINIMUM AND THE PROBABILITY OF ITS DETECTION . In Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 978-972-8865-82-5, pages 5-10. DOI: 10.5220/0001621000050010


in Bibtex Style

@conference{icinco07,
author={B. V. Kryzhanovsky and V. M. Kryzhanovsky and A. L. Mikaelian},
title={BINARY OPTIMIZATION: A RELATION BETWEEN THE DEPTH OF A LOCAL MINIMUM AND THE PROBABILITY OF ITS DETECTION},
booktitle={Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2007},
pages={5-10},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001621000050010},
isbn={978-972-8865-82-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - BINARY OPTIMIZATION: A RELATION BETWEEN THE DEPTH OF A LOCAL MINIMUM AND THE PROBABILITY OF ITS DETECTION
SN - 978-972-8865-82-5
AU - V. Kryzhanovsky B.
AU - M. Kryzhanovsky V.
AU - L. Mikaelian A.
PY - 2007
SP - 5
EP - 10
DO - 10.5220/0001621000050010