Embedding Human Knowledge into Deep Neural Network via Attention Map

Masahiro Mitsuhara, Hiroshi Fukui, Yusuke Sakashita, Takanori Ogata, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi

2021

Abstract

The conventional method to embed human knowledge has been applied for non-deep machine learning. Meanwhile, it is challenging to apply it for deep learning models due to the enormous number of model parameters. In this paper, we propose a novel framework for optimizing networks while embedding human knowledge. The crucial factors are an attention map for visual explanation and an attention mechanism. A manually edited attention map, in which human knowledge is embedded, has the potential to adjust recognition results. The proposed method updates network parameters so that the output attention map corresponds to the edited ones. As a result, the trained network can output an attention map that takes into account human knowledge. Experimental results with ImageNet, CUB-200-2010, and IDRiD demonstrate that it is possible to obtain a clear attention map for a visual explanation and improve the classification performance.

Paper Citation

in Harvard Style

Mitsuhara M., Fukui H., Sakashita Y., Ogata T., Hirakawa T., Yamashita T. and Fujiyoshi H. (2021). Embedding Human Knowledge into Deep Neural Network via Attention Map. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 626-636. DOI: 10.5220/0010335806260636

in Bibtex Style

@conference{visapp21,
author={Masahiro Mitsuhara and Hiroshi Fukui and Yusuke Sakashita and Takanori Ogata and Tsubasa Hirakawa and Takayoshi Yamashita and Hironobu Fujiyoshi},
title={Embedding Human Knowledge into Deep Neural Network via Attention Map},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP},
year={2021},
pages={626-636},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010335806260636},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP
TI - Embedding Human Knowledge into Deep Neural Network via Attention Map
SN - 978-989-758-488-6
AU - Mitsuhara M.
AU - Fukui H.
AU - Sakashita Y.
AU - Ogata T.
AU - Hirakawa T.
AU - Yamashita T.
AU - Fujiyoshi H.
PY - 2021
SP - 626
EP - 636
DO - 10.5220/0010335806260636
PB - SciTePress