FEST: A Unified Framework for Evaluating Synthetic Tabular Data

Weijie Niu, Alberto Huertas Celdran, Karoline Siarsky, Burkhard Stiller

2025

Abstract

Synthetic data generation, leveraging generative machine learning techniques, offers a promising approach to mitigating privacy concerns associated with real-world data usage. Synthetic data closely resemble real-world data while maintaining strong privacy guarantees. However, a comprehensive assessment framework is still missing in the evaluation of synthetic data generation, especially when considering the balance between privacy preservation and data utility in synthetic data. This research bridges this gap by proposing FEST, a systematic framework for evaluating synthetic tabular data. FEST integrates diverse privacy metrics (attack-based and distance-based), along with similarity and machine learning utility metrics, to provide a holistic assessment. We develop FEST as an open-source Python-based library and validate it on multiple datasets, demonstrating its effectiveness in analyzing the privacy-utility trade-off of different synthetic data generation models. The source code of FEST is available on Github.

Paper Citation

in Harvard Style

Niu W., Celdran A., Siarsky K. and Stiller B. (2025). FEST: A Unified Framework for Evaluating Synthetic Tabular Data. In Proceedings of the 11th International Conference on Information Systems Security and Privacy - Volume 1: ICISSP; ISBN 978-989-758-735-1, SciTePress, pages 434-444. DOI: 10.5220/0013383700003899

in Bibtex Style

@conference{icissp25,
author={Weijie Niu and Alberto Celdran and Karoline Siarsky and Burkhard Stiller},
title={FEST: A Unified Framework for Evaluating Synthetic Tabular Data},
booktitle={Proceedings of the 11th International Conference on Information Systems Security and Privacy - Volume 1: ICISSP},
year={2025},
pages={434-444},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013383700003899},
isbn={978-989-758-735-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Information Systems Security and Privacy - Volume 1: ICISSP
TI - FEST: A Unified Framework for Evaluating Synthetic Tabular Data
SN - 978-989-758-735-1
AU - Niu W.
AU - Celdran A.
AU - Siarsky K.
AU - Stiller B.
PY - 2025
SP - 434
EP - 444
DO - 10.5220/0013383700003899
PB - SciTePress