# HIGH THROUGHPUT COMPUTING DUE TO NEAR-OPTIMAL EMERGENT MULTIAGENT COALITIONS FOR LOAD SHARING

### Leland Hovey, Mina Jung

#### Abstract

Grid CPU load-sharing is a subclass of computational grid resource management. Its purpose is to improve grid throughput – High Throughput Computing (HTC). The problem is load-sharing optimization state-space can be quite large. This is because of two factors: the load-sharing optimization problem is NP-complete, and a large volume of CPU-intensive loads can require thousands of Internet connected CPUs. Approximate models can find near-optimal solutions to NP-complete problems. Multiagent coalition formation (MCF) is a particular approximate game theoretic approach for these problems. We propose a new distributed MCF (DMCF) model for Grid CPU load-sharing, DMCF grouping genetic algorithm (DMCF-GGA). This paper presents the model in detail. It also compares this model with our existing model, DMCF-spatial. The comparison consists of a discussion of the models’ similarities and differences, and a comprehensive empirical evalution. The results of this study are the following: The optimization search cost of DMCF-GGA is significantly less than DMCF-spatial. DMCF-GGA has a linear relation between coalition size and search cost (for high throughput). We have found preliminary lower and upper bound estimates for the effective coalition size. We have also found the average job sizes required for the run time of DMCF-GGA to be 1% of the job execution time.

#### References

- Brucker, P. (2004). Scheduling, chapter Computational Complexity, pages 50-60. Springer, Osnabruck, Germany, 4th edition.
- Csari, B., Monostori, L., and Kadar, B. (2004). Learning and cooperation in a distributed market-based production control system. In Proceedings of the 5th International Workshop on Emergent Synthesis, pages 109- 116.
- Decker, K., Durfee, E., and Lesser, V. (1998). Evaluating Research in Cooperative Distributed Problem Solving. UMass Computer Science Technical Report 88-89.
- Fiala, J. and Paulusma, D. (2005). A complete complexity classification of the role assignment problem. Theor. Comput. Sci., 349:67-81.
- Foster, I. and (Eds.), C. K. (1999). The Grid: Blueprint for a Future Computing Infrastructure. Morgan Kaufmann Publishers.
- Garcia-Molina, H. (1982). Elections in a distributed computing system. IEEE Trans. Comput., 31:48-59.
- Hovey, L., Volper, D. E., and Oh, J. C. (2003). Adaptive dynamic load-balancing through evolutionary formation of coalitions. In Abraham, A., Koppen, M., and Franke, K., editors, Design and Application of Hybrid Intellient Systems, pages 194-203, Ohmsha. IOS Press.
- Ibaraki, T. and Katoh, N. (1988). Resource Allocation Problems: Algorithmic Approaches, chapter 1, pages 1-9. MIT Press, Cambridge, MA, USA.
- Kowalski, R. and Sadri, F. (1996). Towards a unified agent architecture that combines rationality with reactivity. In Pedreschi, D. and Zaniolo, C., editors, Logic in Databases, volume 1154 of Lecture Notes in Computer Science, pages 135-149. Springer Berlin / Heidelberg. 10.1007/BFb0031739.
- Kuhn, H. W. (1955). The Hungarian method for the assignment problem. Naval Research Logistic Quarterly, 2:83-97.
- Livny, M., Basney, J., Raman, R., and Tannenbaum, T. (1997). Mechanisms for high throughput computing. SPEEDUP Journal, 11(1).
- Michalewicz, Z. (1999). Genetic Algorithms + Data Structure = Evolution Programs, chapter 11, pages 251- 253. Springer-Verlag, New York, New York.
- Oliphant, M. (1994). Evolving cooperation in the noniterated prisoner's dilemma: The importance of spatial organization. In Brooks, R. and Maes, P., editors, Artificial Life IV: Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems, pages 349-352. MIT Press.
- Pisinger, D. (1999). An exact algorithm for large multiple knapsack problems. European Journal of Operational Research, 114:528-541.
- Raman, R., Livny, M., and Solomon, M. (1998). Matchmaking: Distributed resource management for high throughput computing. In Proceedings of the Seventh IEEE International Symposium on High Performance Distributed Computing (HPDC7), pages 28- 31, Chicago, IL.
- Sandholm, T. (1999). Distributed rational decision making. In Weiss, G., editor, Multiagent Systems. A modern approach to distributed artificial intelligence, volume 1 of Reviews in important subjects, chapter 5, pages 241-251. The MIT Press, Munich, Germany.
- Vazirani, V. V. (2004). Approximation Algorithms, chapter 1, pages 1-2. Springer.
- Weichhart, G., Affenzeller, M., Reitbauer, A., and Wagner, S. (2004). Modelling of an agent-based schedule optimisation system. In Proceedings of the IMS International Forum.
- Wu, A. S., Yu, H., Jin, S., and Lin, K.-C. (2004). An incremental genetic algorithm approach to multiprocessor scheduling. IEEE Trans. Parallel Distrib. Syst., 15(9):824-834. Member-Schiavone, Guy.

#### Paper Citation

#### in Harvard Style

Hovey L. and Jung M. (2012). **HIGH THROUGHPUT COMPUTING DUE TO NEAR-OPTIMAL EMERGENT MULTIAGENT COALITIONS FOR LOAD SHARING** . In *Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,* ISBN 978-989-8425-95-9, pages 295-305. DOI: 10.5220/0003733702950305

#### in Bibtex Style

@conference{icaart12,

author={Leland Hovey and Mina Jung},

title={HIGH THROUGHPUT COMPUTING DUE TO NEAR-OPTIMAL EMERGENT MULTIAGENT COALITIONS FOR LOAD SHARING},

booktitle={Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},

year={2012},

pages={295-305},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0003733702950305},

isbn={978-989-8425-95-9},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,

TI - HIGH THROUGHPUT COMPUTING DUE TO NEAR-OPTIMAL EMERGENT MULTIAGENT COALITIONS FOR LOAD SHARING

SN - 978-989-8425-95-9

AU - Hovey L.

AU - Jung M.

PY - 2012

SP - 295

EP - 305

DO - 10.5220/0003733702950305