Shaping the Behavior Space with Counterfactual Agents in Multi-Objective Map Elites

Anna Nickelson, Nicholas Zerbel, Gaurav Dixit, Kagan Tumer

2023

Abstract

Success in many real-world problems cannot be adequately defined under a single objective and instead requires multiple, sometimes competing, objectives to define the problem. To perform well in these environ-ments, autonomous agents need to have a variety of skills and behaviors to balance these objectives. The combination of Multi-Objective Optimization (MOO) and Quality Diversity (QD) methods, such as in MultiObjective Map Elites (MOME), aim to provide a set of policies with diverse behaviors that cover multiple objectives. However, MOME is unable to diversify its search across the behavior space, resulting in significantly reduced coverage of the global Pareto front. This paper introduces Counterfactual Behavior Shaping for Multi-Objective Map Elites (C-MOME), a method that superimposes counterfactual agents onto the state space of a learning agent to more richly define the diversity of agent behaviors. Counterfactuals explicitly introduce new forms of diversity in agent behaviors, resulting in C-MOME’s effective coverage of behavioral niches; this provides a broader set of Pareto optimal policies. We show that C-MOME covers more than twice as much of the behavior space compared to MOME while increasing the hypervolume of the global Pareto front by over 40%.

Paper Citation

in Harvard Style

Nickelson A., Zerbel N., Dixit G. and Tumer K. (2023). Shaping the Behavior Space with Counterfactual Agents in Multi-Objective Map Elites. In Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: ECTA; ISBN 978-989-758-674-3, SciTePress, pages 41-52. DOI: 10.5220/0012164800003595

in Bibtex Style

@conference{ecta23,
author={Anna Nickelson and Nicholas Zerbel and Gaurav Dixit and Kagan Tumer},
title={Shaping the Behavior Space with Counterfactual Agents in Multi-Objective Map Elites},
booktitle={Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: ECTA},
year={2023},
pages={41-52},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012164800003595},
isbn={978-989-758-674-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: ECTA
TI - Shaping the Behavior Space with Counterfactual Agents in Multi-Objective Map Elites
SN - 978-989-758-674-3
AU - Nickelson A.
AU - Zerbel N.
AU - Dixit G.
AU - Tumer K.
PY - 2023
SP - 41
EP - 52
DO - 10.5220/0012164800003595
PB - SciTePress