Comparing Dependency-based Compositional Models with Contextualized Word Embeddings

Pablo Gamallo, Manuel de Prada Corral, Marcos Garcia

2021

Abstract

In this article, we compare two different strategies to contextualize the meaning of words in a sentence: both distributional models that make use of syntax-based methods following the Principle of Compositionality and Transformer technology such as BERT-like models. As the former methods require controlled syntactic structures, the two approaches are compared against datasets with syntactically fixed sentences, namely subject-predicate and subject-predicate-object expressions. The results show that syntax-based compositional approaches working with syntactic dependencies are competitive with neural-based Transformer models, and could have a greater potential when trained and developed using the same resources.

Paper Citation

in Harvard Style

Gamallo P., Corral M. and Garcia M. (2021). Comparing Dependency-based Compositional Models with Contextualized Word Embeddings.In Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-484-8, pages 1258-1265. DOI: 10.5220/0010391812581265

in Bibtex Style

@conference{icaart21,
author={Pablo Gamallo and Manuel Corral and Marcos Garcia},
title={Comparing Dependency-based Compositional Models with Contextualized Word Embeddings},
booktitle={Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2021},
pages={1258-1265},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010391812581265},
isbn={978-989-758-484-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Comparing Dependency-based Compositional Models with Contextualized Word Embeddings
SN - 978-989-758-484-8
AU - Gamallo P.
AU - Corral M.
AU - Garcia M.
PY - 2021
SP - 1258
EP - 1265
DO - 10.5220/0010391812581265