Deep W-Networks: Solving Multi-Objective Optimisation Problems with Deep Reinforcement Learning

Jernej Hribar, Luke Hackett, Ivana Dusparic

2023

Abstract

In this paper, we build on advances introduced by the Deep Q-Networks (DQN) approach to extend the multiobjective tabular Reinforcement Learning (RL) algorithm W-learning to large state spaces. W-learning algorithm can naturally solve the competition between multiple single policies in multi-objective environments. However, the tabular version does not scale well to environments with large state spaces. To address this issue, we replace underlying Q-tables with DQN, and propose an addition of W-Networks, as a replacement for tabular weights (W) representations. We evaluate the resulting Deep W-Networks (DWN) approach in two widely-accepted multi-objective RL benchmarks: deep sea treasure and multi-objective mountain car. We show that DWN solves the competition between multiple policies while outperforming the baseline in the form of a DQN solution. Additionally, we demonstrate that the proposed algorithm can find the Pareto front in both tested environments.

Paper Citation

in Harvard Style

Hribar J., Hackett L. and Dusparic I. (2023). Deep W-Networks: Solving Multi-Objective Optimisation Problems with Deep Reinforcement Learning. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-623-1, pages 17-26. DOI: 10.5220/0011610300003393

in Bibtex Style

@conference{icaart23,
author={Jernej Hribar and Luke Hackett and Ivana Dusparic},
title={Deep W-Networks: Solving Multi-Objective Optimisation Problems with Deep Reinforcement Learning},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2023},
pages={17-26},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011610300003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Deep W-Networks: Solving Multi-Objective Optimisation Problems with Deep Reinforcement Learning
SN - 978-989-758-623-1
AU - Hribar J.
AU - Hackett L.
AU - Dusparic I.
PY - 2023
SP - 17
EP - 26
DO - 10.5220/0011610300003393