Chat Language Normalisation using Machine Learning Methods

Daiga Deksne

Research.Publish.Connect.

*Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Country:
Subject:

Advanced Search Affiliations Search

If you're looking for an exact phrase use quotation marks on text fields.

Proceedings

Proceedings Search *Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

Papers

Papers Search *Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

Authors

Authors Search *Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

Advanced Search

Paper

Chat Language Normalisation using Machine Learning Methods

In Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: NLPinAI, 965-972, 2019 , Prague, Czech Republic

Author: Daiga Deksne

Affiliation: Tilde, Vienibas Gatve 75a, Riga and Latvia

Keyword(s): Non-Canonical Language, Language Normalisation, Intelligent Virtual Assistants, Intent Detection.

Abstract: This paper reports on the development of a chat language normalisation module for the Latvian language. The model is trained using a random forest classifier algorithm that learns to rate normalisation candidates for every word. Candidates are generated using pre-trained word embeddings, N-gram lists, a spelling checker module and some other modules. The use of different means in generation of the normalisation candidates allows covering a wide spectre of errors. We are planning to use this normalisation module in the development of intelligent virtual assistants. We have performed tests to detect if the results of the intent detection module improve when text is pre-processed with the normalisation module.

CC BY-NC-ND 4.0

Guest: Register as new SciTePress user now for free.

SciTePress user: please login.

My Papers

You are not signed in, therefore limits apply to your IP address 3.133.149.208

In the current month:

Recent papers: 100 available of 100 total

2⁺ years older papers: 200 available of 200 total

Paper citation in several formats:

Deksne, D. (2019). Chat Language Normalisation using Machine Learning Methods. In Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: NLPinAI; ISBN 978-989-758-350-6; ISSN 2184-433X, SciTePress, pages 965-972. DOI: 10.5220/0007693509650972

@conference{nlpinai19,
author={Daiga Deksne},
title={Chat Language Normalisation using Machine Learning Methods},
booktitle={Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: NLPinAI},
year={2019},
pages={965-972},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007693509650972},
isbn={978-989-758-350-6},
issn={2184-433X},
}

TY - CONF

JO - Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: NLPinAI
TI - Chat Language Normalisation using Machine Learning Methods
SN - 978-989-758-350-6
IS - 2184-433X
AU - Deksne, D.
PY - 2019
SP - 965
EP - 972
DO - 10.5220/0007693509650972
PB - SciTePress