iXGB: Improving the Interpretability of XGBoost Using Decision Rules and Counterfactuals

Mir Riyanul Islam, Mobyen Uddin Ahmed, Shahina Begum

2024

Abstract

Tree-ensemble models, such as Extreme Gradient Boosting (XGBoost), are renowned Machine Learning models which have higher prediction accuracy compared to traditional tree-based models. This higher accuracy, however, comes at the cost of reduced interpretability. Also, the decision path or prediction rule of XGBoost is not explicit like the tree-based models. This paper proposes the iXGB–interpretable XGBoost, an approach to improve the interpretability of XGBoost. iXGB approximates a set of rules from the internal structure of XGBoost and the characteristics of the data. In addition, iXGB generates a set of counterfactuals from the neighbourhood of the test instances to support the understanding of the end-users on their operational relevance. The performance of iXGB in generating rule sets is evaluated with experiments on real and benchmark datasets, which demonstrated reasonable interpretability. The evaluation result also supports the idea that the interpretability of XGBoost can be improved without using surrogate methods.

Paper Citation

in Harvard Style

Islam M., Ahmed M. and Begum S. (2024). iXGB: Improving the Interpretability of XGBoost Using Decision Rules and Counterfactuals. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 1345-1353. DOI: 10.5220/0012474000003636

in Bibtex Style

@conference{icaart24,
author={Mir Riyanul Islam and Mobyen Uddin Ahmed and Shahina Begum},
title={iXGB: Improving the Interpretability of XGBoost Using Decision Rules and Counterfactuals},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2024},
pages={1345-1353},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012474000003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - iXGB: Improving the Interpretability of XGBoost Using Decision Rules and Counterfactuals
SN - 978-989-758-680-4
AU - Islam M.
AU - Ahmed M.
AU - Begum S.
PY - 2024
SP - 1345
EP - 1353
DO - 10.5220/0012474000003636
PB - SciTePress