Characterizing Speed Performance of Multi-Agent Reinforcement Learning

Samuel Wiggins, Yuan Meng, Rajgopal Kannan, Viktor Prasanna

2023

Abstract

Multi-Agent Reinforcement Learning (MARL) has achieved significant success in large-scale AI systems and big-data applications such as smart grids, surveillance, etc. Existing advancements in MARL algorithms focus on improving the rewards obtained by introducing various mechanisms for inter-agent cooperation. However, these optimizations are usually compute- and memory-intensive, thus leading to suboptimal speed performance in end-to-end training time. In this work, we analyze the speed performance (i.e., latency-bounded throughput) as the key metric in MARL implementations. Specifically, we first introduce a taxonomy of MARL algorithms from an acceleration perspective categorized by (1) training scheme and (2) communication method. Using our taxonomy, we identify three state-of-the-art MARL algorithms - Multi-Agent Deep Deterministic Policy Gradient (MADDPG), Target-oriented Multi-agent Communication and Cooperation (ToM2C), and Networked Multi-agent RL (NeurComm) - as target benchmark algorithms, and provide a systematic analysis of their performance bottlenecks on a homogeneous multi-core CPU platform. We justify the need for MARL latency-bounded throughput to be a key performance metric in future literature while also addressing opportunities for parallelization and acceleration.

Paper Citation

in Harvard Style

Wiggins S., Meng Y., Kannan R. and Prasanna V. (2023). Characterizing Speed Performance of Multi-Agent Reinforcement Learning. In Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA; ISBN 978-989-758-664-4, SciTePress, pages 327-334. DOI: 10.5220/0012082200003541

in Bibtex Style

@conference{data23,
author={Samuel Wiggins and Yuan Meng and Rajgopal Kannan and Viktor Prasanna},
title={Characterizing Speed Performance of Multi-Agent Reinforcement Learning},
booktitle={Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA},
year={2023},
pages={327-334},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012082200003541},
isbn={978-989-758-664-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA
TI - Characterizing Speed Performance of Multi-Agent Reinforcement Learning
SN - 978-989-758-664-4
AU - Wiggins S.
AU - Meng Y.
AU - Kannan R.
AU - Prasanna V.
PY - 2023
SP - 327
EP - 334
DO - 10.5220/0012082200003541
PB - SciTePress