An Improved Meta-Knowledge Prompt Engineering Approach for Generating Research Questions in Scientific Literature

Meng Wang, Zhixiong Zhang, Zhixiong Zhang, Hanyu Li, Hanyu Li, Guangyin Zhang

2024

Abstract

Research questions are crucial for the development of science, which are an important driving force for scientific evolution and progress. This study analyses the key meta knowledge required for generating research questions in scientific literature, including research objective and research method. To extract metaknowledge, we obtained feature words of meta-knowledge from knowledge-enriched regions and embedded them into the DeBERTa (Decoding-enhanced BERT with disentangled attention) for training. Compared to existing models, our proposed approach demonstrates superior performance across all metrics, achieving improvements in F1 score of +9% over BERT (88% vs. 97%), +3% over BERT-CNN (94% vs. 97%), and +2% over DeBERTa (95% vs. 97%) for identifying meta-knowledge. And, we construct the prompts integrate meta-knowledge to fine tune LLMs. Compared to the baseline model, the LLMs fine-tuned using metaknowledge prompt engineering achieves an average 88.6% F1 score in the research question generation task, with improvements of 8.4%. Overall, our approach can be applied to the research question generation in different domains. Additionally, by updating or replacing the meta-knowledge, the model can also serve as a theoretical foundation and model basis for the generation of different types of sentences.

Paper Citation

in Harvard Style

Wang M., Zhang Z., Li H. and Zhang G. (2024). An Improved Meta-Knowledge Prompt Engineering Approach for Generating Research Questions in Scientific Literature. In Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-716-0, SciTePress, pages 457-464. DOI: 10.5220/0013060900003838

in Bibtex Style

@conference{kdir24,
author={Meng Wang and Zhixiong Zhang and Hanyu Li and Guangyin Zhang},
title={An Improved Meta-Knowledge Prompt Engineering Approach for Generating Research Questions in Scientific Literature},
booktitle={Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2024},
pages={457-464},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013060900003838},
isbn={978-989-758-716-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - An Improved Meta-Knowledge Prompt Engineering Approach for Generating Research Questions in Scientific Literature
SN - 978-989-758-716-0
AU - Wang M.
AU - Zhang Z.
AU - Li H.
AU - Zhang G.
PY - 2024
SP - 457
EP - 464
DO - 10.5220/0013060900003838
PB - SciTePress