Generating Appropriate Question-Answer Pairs for Chatbots using Data Harvested from Community-based QA Sites

Wenjing Yang, Jie Wang

2017

Abstract

Community-based question-answering web sites (CQAW) contain rich collections of question-answer pages, where a single question often has multiple answers written by different authors with different aspects. We study how to harvest new question-answer pairs from CQAWs so that each question-answer pair addresses just one aspect that are suitable for chatbots over a specific domain. In particular, we first extract all answers to a question from a CQAW site using DOM-tree similarities and features of answer areas, and then cluster the answers using LDA. Next, we form a sub-question for each cluster using a small number of top keywords in the given cluster with the keywords in the original question. We select the best answer to the sub-question based on user ratings and similarities of answers to the sub-question. Experimental results show that our approach is effective.

Paper Citation

in Harvard Style

Yang W. and Wang J. (2017). Generating Appropriate Question-Answer Pairs for Chatbots using Data Harvested from Community-based QA Sites.In Proceedings of the 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, ISBN 978-989-758-271-4, pages 342-349. DOI: 10.5220/0006578603420349

in Bibtex Style

@conference{kdir17,
author={Wenjing Yang and Jie Wang},
title={Generating Appropriate Question-Answer Pairs for Chatbots using Data Harvested from Community-based QA Sites},
booktitle={Proceedings of the 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR,},
year={2017},
pages={342-349},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006578603420349},
isbn={978-989-758-271-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR,
TI - Generating Appropriate Question-Answer Pairs for Chatbots using Data Harvested from Community-based QA Sites
SN - 978-989-758-271-4
AU - Yang W.
AU - Wang J.
PY - 2017
SP - 342
EP - 349
DO - 10.5220/0006578603420349