Optimization of the Master/Slave Model in Supercomputing Global Storage Systems

Xiaobin He, Wei Xiao, Qi Chen, Xin Liu, Zuoning Chen

2022

Abstract

The global storage system is an important infrastructure for supercomputing systems, providing a globally consistent view of data for large-scale compute nodes. The Master/Slave model is used extensively in the software implementation of global storage to enhance the server-side capability of handling IO requests. In recent years, as network performance and IO concurrency of compute nodes have increased, the problem of high performance loss of the Master/Slave model in high concurrency scenarios has come to the fore. Taking LWFS, the data forwarding software of the Sunway supercomputers, as an example, this paper proposes two performance optimization methods: optimizing the request receiving capability of the Master based on a multi-threaded parallel processing approach, and reducing the lock overhead of multiple Slave synchronization based on a no-wait request queue mechanism. Tests in the Sunway E-class prototype validation system show that the peak bandwidth of 1M block sequential read/write and 4K block random read/write are increased by 16% and 90% respectively after the optimization methods are overlaid, and the peak bandwidth of 4K block random read/write is increased by 80% and 48% respectively, which shows that the optimized Master/Slave model has a significant increase in read/write bandwidth under high concurrency scenarios and can better adapt to the application IO requirements under high concurrency scenarios.

Paper Citation

in Harvard Style

He X., Xiao W., Chen Q., Liu X. and Chen Z. (2022). Optimization of the Master/Slave Model in Supercomputing Global Storage Systems. In Proceedings of the 3rd International Symposium on Automation, Information and Computing - Volume 1: ISAIC; ISBN 978-989-758-622-4, SciTePress, pages 416-421. DOI: 10.5220/0011949700003612

in Bibtex Style

@conference{isaic22,
author={Xiaobin He and Wei Xiao and Qi Chen and Xin Liu and Zuoning Chen},
title={Optimization of the Master/Slave Model in Supercomputing Global Storage Systems},
booktitle={Proceedings of the 3rd International Symposium on Automation, Information and Computing - Volume 1: ISAIC},
year={2022},
pages={416-421},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011949700003612},
isbn={978-989-758-622-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Symposium on Automation, Information and Computing - Volume 1: ISAIC
TI - Optimization of the Master/Slave Model in Supercomputing Global Storage Systems
SN - 978-989-758-622-4
AU - He X.
AU - Xiao W.
AU - Chen Q.
AU - Liu X.
AU - Chen Z.
PY - 2022
SP - 416
EP - 421
DO - 10.5220/0011949700003612
PB - SciTePress