Author:
Hirokazu Shimauchi
Affiliation:
Hachinohe Institute of Technology, Hachinohe, Aomori, Japan
Keyword(s):
Activation Function, Neural Network, Beltrami Coefficient, Quasiconformal Mapping, Stochastic Perturbation.
Abstract:
We propose an activation function that has a probabilistic Beltrami coefficient for deep neural networks. Activation functions play a crucial role in the performance and training dynamics of deep learning models. In recent years, it has been suggested that the performance of real-valued neural networks can be improved by adding a stochastic perturbation term to the activation function. Meanwhile, numerous studies have been conducted on activation functions of complex-valued neural networks. The proposed approach probabilistically deforms the Beltrami coefficient of complex-valued activation functions. The Beltrami coefficient represents the distortion by mapping at each point. In previous research, when dealing with complex numbers, adding a perturbation term meant applying probabilistic parallel translation from a geometric viewpoint. By contrast, our approach introduces a stochastic perturbation for rotation and scaling. Our experimental results show that the proposed activation fu
nction improves the performance of image classification tasks, implying that the suggested activation function produces effective representations during training.
(More)