Searching for Idealized Prototype Learning for Interpreting Multi-Layered Neural Networks

Ryotaro Kamimura

2024

Abstract

The present paper aims to show that neural learning consists of two fundamental phases: prototype and non-prototype learning in an ideal state. The prototype refers to a network with the simplest configuration, ideally determined without the influence of inputs. However, in actual learning, prototype and non-prototype learning are mixed and entangled. To demonstrate the existence of these two phases in an ideal state, it is necessary to explicitly distinguish between networks that are exclusively focused on acquiring the prototype and those that target non-prototype properties. We use different activation functions, combined serially, to represent the prototype and non-prototype learning phases. By combining multiple different activation functions, it is possible to create networks that exhibit both prototype and non-prototype properties in an almost ideal state. This method was applied to a business dataset that required improved generalization as well as interpretation. The experimental results confirmed that the ReLU activation function could identify the prototype with difficulty, while the hyperbolic tangent function could more easily detect the prototype. By combining these two activation functions within one framework, generalization performance could be improved while maintaining representations that are as close as possible to those obtained during prototype learning, thus facilitating easier interpretation.

Paper Citation

in Harvard Style

Kamimura R. (2024). Searching for Idealized Prototype Learning for Interpreting Multi-Layered Neural Networks. In Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: NCTA; ISBN 978-989-758-721-4, SciTePress, pages 475-487. DOI: 10.5220/0013047600003837

in Bibtex Style

@conference{ncta24,
author={Ryotaro Kamimura},
title={Searching for Idealized Prototype Learning for Interpreting Multi-Layered Neural Networks},
booktitle={Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: NCTA},
year={2024},
pages={475-487},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013047600003837},
isbn={978-989-758-721-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computational Intelligence - Volume 1: NCTA
TI - Searching for Idealized Prototype Learning for Interpreting Multi-Layered Neural Networks
SN - 978-989-758-721-4
AU - Kamimura R.
PY - 2024
SP - 475
EP - 487
DO - 10.5220/0013047600003837
PB - SciTePress