A Framework of Hierarchical Deep Q-Network for Portfolio Management

Yuan Gao, Ziming Gao, Yi Hu, Sifan Song, Zhengyong Jiang, Jionglong Su

2021

Abstract

Reinforcement Learning algorithms and Neural Networks have diverse applications in many domains, e.g., stock market prediction, facial recognition and automatic machine translation. The concept of modeling the portfolio management through a reinforcement learning formulation is novel, and the Deep Q-Network has been successfully applied to portfolio management recently. However, the model does not take into account of commission fee for transaction. This paper introduces a framework, based on the hierarchical Deep Q-Network, that addresses the issue of zero commission fee by reducing the number of assets assigned to each Deep Q-Network and dividing the total portfolio value into smaller parts. Furthermore, this framework is flexible enough to handle an arbitrary number of assets. In our experiments, the time series of four stocks for three different time periods are used to assess the efficacy of our model. It is found that our hierarchical Deep Q-Network based strategy outperforms ten other strategies, including nine traditional strategies and one reinforcement learning strategy, in profitability as measured by the Cumulative Rate of Return. Moreover, the Sharpe ratio and Max Drawdown metrics both demonstrate that the risk of policy associated with hierarchical Deep Q-Network is the lowest among all ten strategies.

Paper Citation

in Harvard Style

Gao Y., Gao Z., Hu Y., Song S., Jiang Z. and Su J. (2021). A Framework of Hierarchical Deep Q-Network for Portfolio Management.In Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-484-8, pages 132-140. DOI: 10.5220/0010233201320140

in Bibtex Style

@conference{icaart21,
author={Yuan Gao and Ziming Gao and Yi Hu and Sifan Song and Zhengyong Jiang and Jionglong Su},
title={A Framework of Hierarchical Deep Q-Network for Portfolio Management},
booktitle={Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2021},
pages={132-140},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010233201320140},
isbn={978-989-758-484-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - A Framework of Hierarchical Deep Q-Network for Portfolio Management
SN - 978-989-758-484-8
AU - Gao Y.
AU - Gao Z.
AU - Hu Y.
AU - Song S.
AU - Jiang Z.
AU - Su J.
PY - 2021
SP - 132
EP - 140
DO - 10.5220/0010233201320140