Continual Optimization of In-Production Machine Learning Systems Through Semantic Analysis of User Feedback

Hemadri Jayalath, Ghadeer Yassin, Lakshmish Ramaswamy, Sheng Li

2023

Abstract

With the rapid advancement of machine learning technologies, a wide range of industries and domains have adopted machine learning in their key business processes. Because of this, it is critical to ensure the optimal performance of operationalized machine learning models. This requires machine learning models to be regularly monitored and well-maintained after deployment. In this paper, we discuss the benefits of getting human guidance during the machine learning model maintenance stage. We present a novel approach that semantically evaluates end-user feedback and identifies the sentiment of the users towards different aspects of machine learning models and provides guidance to systematize the fixes according to the priority. We also crawled the web and created a small data set containing user feedback related to language models and evaluated it using our approach and uncovered some interesting insights related to language models. Further, we compare the trade-offs of alternative techniques that can be applied in different stages in our proposed model evaluation pipeline. Finally, we have provided insights and the future work that can be done to broaden the area of machine learning maintenance with human collaboration.

Paper Citation

in Harvard Style

Jayalath H., Yassin G., Ramaswamy L. and Li S. (2023). Continual Optimization of In-Production Machine Learning Systems Through Semantic Analysis of User Feedback. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-623-1, pages 285-292. DOI: 10.5220/0011660300003393

in Bibtex Style

@conference{icaart23,
author={Hemadri Jayalath and Ghadeer Yassin and Lakshmish Ramaswamy and Sheng Li},
title={Continual Optimization of In-Production Machine Learning Systems Through Semantic Analysis of User Feedback},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2023},
pages={285-292},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011660300003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - Continual Optimization of In-Production Machine Learning Systems Through Semantic Analysis of User Feedback
SN - 978-989-758-623-1
AU - Jayalath H.
AU - Yassin G.
AU - Ramaswamy L.
AU - Li S.
PY - 2023
SP - 285
EP - 292
DO - 10.5220/0011660300003393