Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication

Adrian Redder, Arunselvan Ramaswamy, Holger Karl

2022

Abstract

Distributed online learning over delaying communication networks is a fundamental problem in multi-agent learning, since the convergence behaviour of interacting agents is distorted by their delayed communication. It is a priori unclear, how much communication delay can be allowed, such that the joint policies of multiple agents can still converge to a solution of a multi-agent learning problem. In this work, we present the decentralization of the well known deep deterministic policy gradient algorithm using a communication network. We illustrate the convergence of the algorithm and the effect of lossy communication on the rate of convergence for a two-agent flow control problem, where the agents exchange their local information over a delaying wireless network. Finally, we discuss theoretical implications for this algorithm using recent advances in the theory of age of information and deep reinforcement learning.

Paper Citation

in Harvard Style

Redder A., Ramaswamy A. and Karl H. (2022). Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-547-0, pages 282-289. DOI: 10.5220/0010845400003116

in Bibtex Style

@conference{icaart22,
author={Adrian Redder and Arunselvan Ramaswamy and Holger Karl},
title={Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2022},
pages={282-289},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010845400003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication
SN - 978-989-758-547-0
AU - Redder A.
AU - Ramaswamy A.
AU - Karl H.
PY - 2022
SP - 282
EP - 289
DO - 10.5220/0010845400003116