JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks

N. Erichson, Zhewei Yao, Michael Mahoney

2020

Abstract

It has been demonstrated that very simple attacks can fool highly-sophisticated neural network architectures. In particular, so-called adversarial examples, constructed from perturbations of input data that are small or imperceptible to humans but lead to different predictions, may lead to an enormous risk in certain critical applications. In light of this, there has been a great deal of work on developing adversarial training strategies to improve model robustness. These training strategies are very expensive, in both human and computational time. To complement these approaches, we propose a very simple and inexpensive strategy which can be used to “retrofit” a previously-trained network to improve its resilience to adversarial attacks. More concretely, we propose a new activation function—the JumpReLU—which, when used in place of a ReLU in an already-trained model, leads to a trade-off between predictive accuracy and robustness. This trade-off is controlled by the jump size, a hyper-parameter which can be tuned during the validation stage. Our empirical results demonstrate that this increases model robustness, protecting against adversarial attacks with substantially increased levels of perturbations. This is accomplished simply by retrofitting existing networks with our JumpReLU activation function, without the need for retraining the model. Additionally, we demonstrate that adversarially trained (robust) models can greatly benefit from retrofitting.

Paper Citation

in Harvard Style

Erichson N., Yao Z. and Mahoney M. (2020). JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks. In Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-397-1, pages 103-114. DOI: 10.5220/0009316401030114

in Bibtex Style

@conference{icpram20,
author={N. Erichson and Zhewei Yao and Michael Mahoney},
title={JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks},
booktitle={Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2020},
pages={103-114},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009316401030114},
isbn={978-989-758-397-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks
SN - 978-989-758-397-1
AU - Erichson N.
AU - Yao Z.
AU - Mahoney M.
PY - 2020
SP - 103
EP - 114
DO - 10.5220/0009316401030114