Structural Extensions of Basis Pursuit: Guarantees on Adversarial Robustness

Dávid Szeghy, Dávid Szeghy, Mahmoud Aslan, Áron Fóthi, Balázs Mészáros, Zoltán Milacski, András Lőrincz

2022

Abstract

While deep neural networks are sensitive to adversarial noise, sparse coding using the Basis Pursuit (BP) method is robust against such attacks, including its multi-layer extensions. We prove that the stability theorem of BP holds upon the following generalizations: (i) the regularization procedure can be separated into disjoint groups with different weights, (ii) neurons or full layers may form groups, and (iii) the regularizer takes various generalized forms of the `1 norm. This result provides the proof for the architectural generalizations of (Cazenavette et al., 2021) including (iv) an approximation of the complete architecture as a shallow sparse coding network. Due to this approximation, we settled to experimenting with shallow networks and studied their robustness against the Iterative Fast Gradient Sign Method on a synthetic dataset and MNIST. We introduce classification based on the `2 norms of the groups and show numerically that it can be accurate and offers considerable speedups. In this family, linear transformer shows the best performance. Based on the theoretical results and the numerical simulations, we highlight numerical matters that may improve performance further. The proofs of our theorems can be found in the supplementary material a .

Paper Citation

in Harvard Style

Szeghy D., Aslan M., Fóthi Á., Mészáros B., Milacski Z. and Lőrincz A. (2022). Structural Extensions of Basis Pursuit: Guarantees on Adversarial Robustness. In Proceedings of the 3rd International Conference on Deep Learning Theory and Applications - Volume 1: DeLTA, ISBN 978-989-758-584-5, pages 77-85. DOI: 10.5220/0011138900003277

in Bibtex Style

@conference{delta22,
author={Dávid Szeghy and Mahmoud Aslan and Áron Fóthi and Balázs Mészáros and Zoltán Milacski and András Lőrincz},
title={Structural Extensions of Basis Pursuit: Guarantees on Adversarial Robustness},
booktitle={Proceedings of the 3rd International Conference on Deep Learning Theory and Applications - Volume 1: DeLTA,},
year={2022},
pages={77-85},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011138900003277},
isbn={978-989-758-584-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Conference on Deep Learning Theory and Applications - Volume 1: DeLTA,
TI - Structural Extensions of Basis Pursuit: Guarantees on Adversarial Robustness
SN - 978-989-758-584-5
AU - Szeghy D.
AU - Aslan M.
AU - Fóthi Á.
AU - Mészáros B.
AU - Milacski Z.
AU - Lőrincz A.
PY - 2022
SP - 77
EP - 85
DO - 10.5220/0011138900003277