CSP-DC: Data Cleaning via Constraint Satisfaction Problem Solving

Nibel Nadjeh, Sabrina Abdellaoui, Fahima Nader

2023

Abstract

In this paper, we present CSP-DC, a data cleaning system that integrates a new intelligent solution into the cleaning process to improve data accuracy, consistency, and minimize user involvement. We address three main challenges: (1) Consistency: Most repairing algorithms introduce new violations when repairing data, especially when constraints have overlapping attributes. (2) Automaticity: User intervention is time-consuming, we seek to minimize their efforts. (3) Accuracy: Most automatic approaches compute minimal repairs and apply unverified modifications to repair ambiguous cases, which may introduce more noise. To address these challenges, we propose to formulate this problem as a constraint satisfaction problem (CSP) allowing updates that always maintain data consistency. To achieve high performances, we perform a first cleaning phase to automatically repair violations that are easily handled by existing repair algorithms. We handle violations with multiple possible repairs with a CSP solving algorithm, which selects from possible fixes, values that respect all constraints. To reduce the problem’s complexity, we propose a new variables ordering technique and pruning strategies, allowing to optimize the repair search and find a solution quickly. Our experiments show that CSP-DC provides consistent and accurate repairs in a linear time, while also minimizing user intervention.

Paper Citation

in Harvard Style

Nadjeh N., Abdellaoui S. and Nader F. (2023). CSP-DC: Data Cleaning via Constraint Satisfaction Problem Solving. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-623-1, pages 478-488. DOI: 10.5220/0011897000003393

in Bibtex Style

@conference{icaart23,
author={Nibel Nadjeh and Sabrina Abdellaoui and Fahima Nader},
title={CSP-DC: Data Cleaning via Constraint Satisfaction Problem Solving},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2023},
pages={478-488},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011897000003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - CSP-DC: Data Cleaning via Constraint Satisfaction Problem Solving
SN - 978-989-758-623-1
AU - Nadjeh N.
AU - Abdellaoui S.
AU - Nader F.
PY - 2023
SP - 478
EP - 488
DO - 10.5220/0011897000003393