Comparing Dependency-based Compositional Models with Contextualized Word Embeddings

Pablo Gamallo, Manuel Corral, Marcos Garcia

2021

Abstract

In this article, we compare two different strategies to contextualize the meaning of words in a sentence: both distributional models that make use of syntax-based methods following the Principle of Compositionality and Transformer technology such as BERT-like models. As the former methods require controlled syntactic structures, the two approaches are compared against datasets with syntactically fixed sentences, namely subject-predicate and subject-predicate-object expressions. The results show that syntax-based compositional approaches working with syntactic dependencies are competitive with neural-based Transformer models, and could have a greater potential when trained and developed using the same resources.

Paper Citation

in Harvard Style

Gamallo P., Corral M. and Garcia M. (2021). Comparing Dependency-based Compositional Models with Contextualized Word Embeddings.In Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-484-8, pages 1258-1265. DOI: 10.5220/0010391812581265

in Bibtex Style

@conference{icaart21,
author={Pablo Gamallo and Manuel Corral and Marcos Garcia},
title={Comparing Dependency-based Compositional Models with Contextualized Word Embeddings},
booktitle={Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2021},
pages={1258-1265},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010391812581265},
isbn={978-989-758-484-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Comparing Dependency-based Compositional Models with Contextualized Word Embeddings
SN - 978-989-758-484-8
AU - Gamallo P.
AU - Corral M.
AU - Garcia M.
PY - 2021
SP - 1258
EP - 1265
DO - 10.5220/0010391812581265