Dynamically Choosing the Number of Heads in Multi-Head Attention

Fernando Fradique Duarte, Nuno Lau, Artur Pereira, Luís Reis

2024

Abstract

Deep Learning agents are known to be very sensitive to their parameterization values. Attention-based Deep Reinforcement Learning agents further complicate this issue due to the additional parameterization associated to the computation of their attention function. One example of this concerns the number of attention heads to use when dealing with multi-head attention-based agents. Usually, these hyperparameters are set manually, which may be neither optimal nor efficient. This work addresses the issue of choosing the appropriate number of attention heads dynamically, by endowing the agent with a policy πh trained with policy gradient. At each timestep of agent-environment interaction, πh is responsible for choosing the most suitable number of attention heads according to the contextual memory of the agent. This dynamic parameterization is compared to a static parameterization in terms of performance. The role of πh is further assessed by providing additional analysis concerning the distribution of the number of attention heads throughout the training procedure and the course of the game. The Atari 2600 videogame benchmark was used to perform and validate all the experiments.

Paper Citation

in Harvard Style

Fradique Duarte F., Lau N., Pereira A. and Reis L. (2024). Dynamically Choosing the Number of Heads in Multi-Head Attention. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 358-367. DOI: 10.5220/0012384500003636

in Bibtex Style

@conference{icaart24,
author={Fernando Fradique Duarte and Nuno Lau and Artur Pereira and Luís Reis},
title={Dynamically Choosing the Number of Heads in Multi-Head Attention},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2024},
pages={358-367},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012384500003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - Dynamically Choosing the Number of Heads in Multi-Head Attention
SN - 978-989-758-680-4
AU - Fradique Duarte F.
AU - Lau N.
AU - Pereira A.
AU - Reis L.
PY - 2024
SP - 358
EP - 367
DO - 10.5220/0012384500003636
PB - SciTePress