Target Evaluation for Neural Language Model using Japanese Case Frame

Kazuhito Tamura, Ikumi Suzuki, Kazuo Hara

2020

Abstract

Automatic text generation are widely used in various type of natural language processing systems. It is crucial to capture correct grammar for these systems to work. According to the recent studies, neural language models successfully acquire English grammar. However, it’s not thoroughly investigated why the neural language models work. Therefore, fine-grained grammatical or syntactic analysis is important to assess neural language models. In this paper, we constructed grammatical evaluation methods to assess Japanese grammatical ability in neural language models by adopting a target evaluation approach. We especially focus on case marker and verb match in Japanese case grammar. In experiments, we report the grammatical ability of neural language model by comparing n-gram models. Neural language model performed better even some information lacks, while n-gram performs poorly. Also, Neural language model exhibited more robust performance for low frequency terms.

Paper Citation

in Harvard Style

Tamura K., Suzuki I. and Hara K. (2020). Target Evaluation for Neural Language Model using Japanese Case Frame. In Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2020) - Volume 1: KDIR; ISBN 978-989-758-474-9, SciTePress, pages 251-258. DOI: 10.5220/0010137702510258

in Bibtex Style

@conference{kdir20,
author={Kazuhito Tamura and Ikumi Suzuki and Kazuo Hara},
title={Target Evaluation for Neural Language Model using Japanese Case Frame},
booktitle={Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2020) - Volume 1: KDIR},
year={2020},
pages={251-258},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010137702510258},
isbn={978-989-758-474-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2020) - Volume 1: KDIR
TI - Target Evaluation for Neural Language Model using Japanese Case Frame
SN - 978-989-758-474-9
AU - Tamura K.
AU - Suzuki I.
AU - Hara K.
PY - 2020
SP - 251
EP - 258
DO - 10.5220/0010137702510258
PB - SciTePress