LLM Output Compliance with Handcrafted Linguistic Features: An Experiment

Andrei Olar

2025

Abstract

Can we control the writing style of large language models (LLMs) by specifying desired linguistic features? We address this question by investigating the impact of handcrafted linguistic feature (HLF) instructions on LLM-generated text. Our experiment evaluates various state-of-the-art LLMs using prompts incorporating HLF statistics derived from corpora of CNN articles and Yelp reviews. We find that LLMs demonstrate sensitivity to these instructions, particularly when tasked with conforming to concrete features like word count. However, compliance with abstract features, such as lexical variation, proves more challenging, often resulting in negative impacts on compliance. Our findings highlight the potential and limitations of utilizing HLFs for guiding LLM text generation and underscore the need for further research into optimizing prompt design and feature selection.

Paper Citation

in Harvard Style

Olar A. (2025). LLM Output Compliance with Handcrafted Linguistic Features: An Experiment. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 767-775. DOI: 10.5220/0013350400003890

in Bibtex Style

@conference{icaart25,
author={Andrei Olar},
title={LLM Output Compliance with Handcrafted Linguistic Features: An Experiment},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2025},
pages={767-775},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013350400003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - LLM Output Compliance with Handcrafted Linguistic Features: An Experiment
SN - 978-989-758-737-5
AU - Olar A.
PY - 2025
SP - 767
EP - 775
DO - 10.5220/0013350400003890
PB - SciTePress