iRNN: Integer-only Recurrent Neural Network

Eyyüb Sari, Vanessa Courville, Vahid Partovi Nia

2022

Abstract

Recurrent neural networks (RNN) are used in many real-world text and speech applications. They include complex modules such as recurrence, exponential-based activation, gate interaction, unfoldable normalization, bi-directional dependence, and attention. The interaction between these elements prevents running them on integer-only operations without a significant performance drop. Deploying RNNs that include layer normalization and attention on integer-only arithmetic is still an open problem. We present a quantization-aware training method for obtaining a highly accurate integer-only recurrent neural network (iRNN). Our approach supports layer normalization, attention, and an adaptive piecewise linear approximation of activations (PWL), to serve a wide range of RNNs on various applications. The proposed method is proven to work on RNNbased language models and challenging automatic speech recognition, enabling AI applications on the edge. Our iRNN maintains similar performance as its full-precision counterpart, their deployment on smartphones improves the runtime performance by 2×, and reduces the model size by 4×.

Paper Citation

in Harvard Style

Sari E., Courville V. and Partovi Nia V. (2022). iRNN: Integer-only Recurrent Neural Network. In Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-549-4, pages 110-121. DOI: 10.5220/0010975700003122

in Bibtex Style

@conference{icpram22,
author={Eyyüb Sari and Vanessa Courville and Vahid Partovi Nia},
title={iRNN: Integer-only Recurrent Neural Network},
booktitle={Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2022},
pages={110-121},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010975700003122},
isbn={978-989-758-549-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - iRNN: Integer-only Recurrent Neural Network
SN - 978-989-758-549-4
AU - Sari E.
AU - Courville V.
AU - Partovi Nia V.
PY - 2022
SP - 110
EP - 121
DO - 10.5220/0010975700003122