![](bg7.png)
be taken with a grain of salt as our methodology de-
pends on the number and the quality of the training
samples. In this regard, the Blended mechanism does
not purport to replace other augmented solutions that
the AI community is currently utilizing but only to
help them as an additional strategy in their toolbox.
Table 3: Results of the Vanilla and the Blended approach.
Results
- Vanilla Blended
Round Accuracy Accuracy
1st 0.47 0.37
2rd 0.21 0.46
3rd 0.46 0.41
4th 0.45 0.46
5th 0.43 0.36
6th 0.48 0.47
7th 0.17 0.46
8th 0.58 0.52
9th 0.49 0.44
10th 0.48 0.50
Average 0.42 0.45
5 CONCLUSION
This study used a simplified technique to analyze
the relationship between dependency parsers and lan-
guage models as a step toward understanding the im-
pact of fine-tuning the latter with enhanced samples
designed upon word relations found by the former.
Given the successes of utilizing dependency parsers
in developing classic AI systems, we suspected that
further fine-tuning language models with semantics
found in parsed samples would improve their accu-
racy results, especially when we do not have access
to a large amount of training material.
Undertaken experiments revealed that enhancing
language models with semantic relations improves
their accuracy scores, indicating the robustness of this
method. The success highlights the importance of uti-
lizing dependency parsers when fine-tuning language
models, suggesting that they seem to help the models
generalize well when dealing with unseen classes or
sequences of texts that were not met before.
We hope our work will spur further research on
the relationship between parsers and language mod-
els. Note that we did not compare possible differences
in utilizing the syntactic output of parsers. As this
seems to be a rapidly evolving area of research, we
would encourage researchers to build upon this work
by examining the impact of utilizing and comparing
results based on the syntactic versus semantic analy-
sis of training samples.
REFERENCES
Bengio, Y. (2008). Neural net language models. Scholarpe-
dia, 3(1):3881.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D.,
Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G.,
Askell, A., et al. (2020). Language models are few-
shot learners. Advances in neural information pro-
cessing systems, 33:1877–1901.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.
(2018). Bert: Pre-training of deep bidirectional trans-
formers for language understanding. arXiv preprint
arXiv:1810.04805.
He, P., Liu, X., Gao, J., and Chen, W. (2020). Deberta:
Decoding-enhanced bert with disentangled attention.
arXiv preprint arXiv:2006.03654.
Isaak, N. and Michael, L. (2016). Tackling the Winograd
Schema Challenge Through Machine Logical Infer-
ences. In Pearce, D. and Pinto, H. S., editors, STAIRS,
volume 284 of Frontiers in Artificial Intelligence and
Applications, pages 75–86. IOS Press.
Isaak, N. and Michael, L. (2017). How the Availability of
Training Material Affects Performance in the Wino-
grad Schema Challenge. In Proceedings of the (IJCAI
2017) 3rd Workshop on Cognitive Knowledge Acqui-
sition and Applications (Cognitum 2017).
Isaak, N. and Michael, L. (2021). Experience and predic-
tion: a metric of hardness for a novel litmus test. Jour-
nal of Logic and Computation, 31(8):2028–2056.
K
¨
ubler, S., McDonald, R., and Nivre, J. (2009). Depen-
dency parsing. Synthesis lectures on human language
technologies, 1(1):1–127.
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma,
P., and Soricut, R. (2019). Albert: A lite bert for
self-supervised learning of language representations.
arXiv preprint arXiv:1909.11942.
Levesque, H. J. (2014). On Our Best behaviour. Artificial
Intelligence, 212:27–35.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D.,
Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov,
V. (2019). Roberta: A robustly optimized bert pre-
training approach. arXiv preprint arXiv:1907.11692.
McCarthy, J., Minsky, M. L., Rochester, N., and Shannon,
C. E. (2006). A Proposal for the Dartmouth Summer
Research Project on Artificial Intelligence, August 31,
1955. AI magazine, 27(4):12–12.
Michael, L. (2013). Machines with websense. In Proc.
of 11th International Symposium on Logical For-
malizations of Commonsense Reasoning (Common-
sense’13).
Nivre, J. (2005). Dependency grammar and dependency
parsing. MSI report, 5133(1959):1–32.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D.,
Sutskever, I., et al. (2019). Language models are un-
supervised multitask learners. OpenAI blog, 1(8):9.
Blending Dependency Parsers With Language Models
819