A New Cluster Validation Index Based on Stability Analysis

Adane Nega Tarekegn, Bjørnar Tessem, Fazle Rabbi

2025

Abstract

Clustering is a frequently employed technique across various domains, including anomaly detection, recommender systems, video analysis, and natural language processing. Despite its broad application, validating clustering results has become one of the main challenges in cluster analysis. This can be due to factors such as the subjective nature of clustering evaluation, lack of ground truth in many real-world datasets, and sensitivity of evaluation metrics to different cluster shapes and algorithms. While there is extensive literature work in this area, developing an evaluation method that is both objective and quantitative is still challenging task requiring more effort. In this study, we proposed a new Clustering Stability Assessment Index (CSAI) that can provide a unified and quantitative approach to measure the quality and consistency of clustering solutions. The proposed CSAI validation index leverages a data resampling approach and prediction analysis to assess clustering stability by using multiple features associated within clusters, rather than the traditional centroid-based method. This approach enables reproducibility in data clustering and operates independently of the clustering algorithms used, which makes it adaptable to various methods and applications. To evaluate the effectiveness and generality of the CSAI, we have carried out an extensive experimental analysis using various clustering algorithms and benchmark datasets. The obtained results show that CSAI demonstrates competitive performance compared to existing cluster validation indices and effectively measures the quality and robustness of clustering results across multiple samples.

Paper Citation

in Harvard Style

Tarekegn A., Tessem B. and Rabbi F. (2025). A New Cluster Validation Index Based on Stability Analysis. In Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM; ISBN 978-989-758-730-6, SciTePress, pages 377-384. DOI: 10.5220/0013309100003905

in Bibtex Style

@conference{icpram25,
author={Adane Tarekegn and Bjørnar Tessem and Fazle Rabbi},
title={A New Cluster Validation Index Based on Stability Analysis},
booktitle={Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},
year={2025},
pages={377-384},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013309100003905},
isbn={978-989-758-730-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM
TI - A New Cluster Validation Index Based on Stability Analysis
SN - 978-989-758-730-6
AU - Tarekegn A.
AU - Tessem B.
AU - Rabbi F.
PY - 2025
SP - 377
EP - 384
DO - 10.5220/0013309100003905
PB - SciTePress