An Activation Function with Probabilistic Beltrami Coefficient for Deep Learning

Hirokazu Shimauchi

2022

Abstract

We propose an activation function that has a probabilistic Beltrami coefficient for deep neural networks. Activation functions play a crucial role in the performance and training dynamics of deep learning models. In recent years, it has been suggested that the performance of real-valued neural networks can be improved by adding a stochastic perturbation term to the activation function. Meanwhile, numerous studies have been conducted on activation functions of complex-valued neural networks. The proposed approach probabilistically deforms the Beltrami coefficient of complex-valued activation functions. The Beltrami coefficient represents the distortion by mapping at each point. In previous research, when dealing with complex numbers, adding a perturbation term meant applying probabilistic parallel translation from a geometric viewpoint. By contrast, our approach introduces a stochastic perturbation for rotation and scaling. Our experimental results show that the proposed activation function improves the performance of image classification tasks, implying that the suggested activation function produces effective representations during training.

Paper Citation

in Harvard Style

Shimauchi H. (2022). An Activation Function with Probabilistic Beltrami Coefficient for Deep Learning. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-547-0, pages 613-620. DOI: 10.5220/0010870500003116

in Bibtex Style

@conference{icaart22,
author={Hirokazu Shimauchi},
title={An Activation Function with Probabilistic Beltrami Coefficient for Deep Learning},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2022},
pages={613-620},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010870500003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - An Activation Function with Probabilistic Beltrami Coefficient for Deep Learning
SN - 978-989-758-547-0
AU - Shimauchi H.
PY - 2022
SP - 613
EP - 620
DO - 10.5220/0010870500003116