Homomorphic Encryption Friendly Multi-GAT for Information Extraction in Business Documents

Djedjiga Belhadj, Yolande Belaïd, Abdel Belaïd

2024

Abstract

This paper presents a homomorphic encryption (HE) system to extract information from business documents. We propose a structured method to replace the nonlinear activation functions of a multi-layer graph attention network (Multi-GAT), including ReLU, LeakyReLU, and the attention mechanism Softmax, with polynomials of different degrees. We also replace the normalization layers with an adapted HE algorithm. To solve the problem of accuracy loss during the approximation, we use a partially HE baseline model to train a fully HE model using techniques such as distillation knowledge and model fine-tuning. The proposed HE-friendly Multi-GAT models the document as a graph of words and uses the multi-head attention mechanism to classify the graph nodes. The first partially HE-Multi-GAT contains polynomial approximations of all ReLU, LeakyReLU and the attention Softmax activation functions. Normalization layers are used to handle values exploding when approximating all the nonlinear activation functions. These layers are approximated as well using an adapted algorithm that doesn’t rely on the training data and minimizes performances loss while avoiding connections between the server and the data owner. Experiments show that our approach minimizes the model accuracy loss. We tested the architecture on three different datasets and obtained competitive results (F1-scores greater than 93%).

Paper Citation

in Harvard Style

Belhadj D., Belaïd Y. and Belaïd A. (2024). Homomorphic Encryption Friendly Multi-GAT for Information Extraction in Business Documents. In Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM; ISBN 978-989-758-684-2, SciTePress, pages 817-825. DOI: 10.5220/0012432400003654

in Bibtex Style

@conference{icpram24,
author={Djedjiga Belhadj and Yolande Belaïd and Abdel Belaïd},
title={Homomorphic Encryption Friendly Multi-GAT for Information Extraction in Business Documents},
booktitle={Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},
year={2024},
pages={817-825},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012432400003654},
isbn={978-989-758-684-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM
TI - Homomorphic Encryption Friendly Multi-GAT for Information Extraction in Business Documents
SN - 978-989-758-684-2
AU - Belhadj D.
AU - Belaïd Y.
AU - Belaïd A.
PY - 2024
SP - 817
EP - 825
DO - 10.5220/0012432400003654
PB - SciTePress