Real-time Statistical Log Anomaly Detection with Continuous AIOps Learning

Lu An, An-Jie Tu, Xiaotong Liu, Rama Akkiraju

2022

Abstract

Anomaly detection from logs is a fundamental Information Technology Operations (ITOps) management task. It aims to detect anomalous system behaviours and find signals that can provide clues to the reasons and the anatomy of a system’s failure. Applying advanced, explainable Artificial Intelligence (AI) models throughout the entire ITOps is critical to confidently assess, diagnose and resolve such system failures. In this paper, we describe a new online log anomaly detection algorithm which helps significantly reduce the time-to-value of Log Anomaly Detection. This algorithm is able to continuously update the Log Anomaly Detection model at run-time and automatically avoid potential biased model caused by contaminated log data. The methods described here have shown 60% improvement on average F1-scores from experiments for multiple datasets comparing to the existing method in the product pipeline, which demonstrates the efficacy of our proposed methods.

Paper Citation

in Harvard Style

An L., Tu A., Liu X. and Akkiraju R. (2022). Real-time Statistical Log Anomaly Detection with Continuous AIOps Learning. In Proceedings of the 12th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-570-8, pages 223-230. DOI: 10.5220/0011069200003200

in Bibtex Style

@conference{closer22,
author={Lu An and An-Jie Tu and Xiaotong Liu and Rama Akkiraju},
title={Real-time Statistical Log Anomaly Detection with Continuous AIOps Learning},
booktitle={Proceedings of the 12th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2022},
pages={223-230},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011069200003200},
isbn={978-989-758-570-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Real-time Statistical Log Anomaly Detection with Continuous AIOps Learning
SN - 978-989-758-570-8
AU - An L.
AU - Tu A.
AU - Liu X.
AU - Akkiraju R.
PY - 2022
SP - 223
EP - 230
DO - 10.5220/0011069200003200