Invertibility of ReLU-Layers: A Practical Approach
Hannah Eckert
1 a
, Daniel Haider
1 b
, Martin Ehler
2 c
and Peter Balazs
1 d
1
Acoustics Research Institute, Austrian Academy of Sciences, Dominikanerbastei 16, Vienna, Austria
2
Faculty of Mathematics, University of Vienna, Oskar-Morgenstern-Platz 1, Vienna, Austria
Keywords:
Invertible Neural Networks, Reconstruction, Frame Theory, Stability, Interpretability.
Abstract:
Invertibility in machine learning models is a pivotal feature that bridges the gap between model complexity and
interpretability. For ReLU-layers, the practical verification of invertibility has been shown to be a difficult task
that still remains unsolved. Recently, a frame theoretic condition has been proposed to verify invertibility on
an open or convex set, however, the computations for this condition are computationally infeasible in high di-
mensions. As an alternative, we propose an algorithm that stochastically samples the dataset to approximately
verify the above condition for invertibility and can be efficiently implemented even in high dimensions. We
use the algorithm to monitor invertibility and to enforce it during training in standard classification tasks.
1 INTRODUCTION
Keeping models powerful while designing them in a
transparent and interpretable way is one of the most
significant challenges in modern machine learning
(Fan et al., 2020). In this context, understanding
the underlying mechanisms by which a model makes
its predictions has become a focal point of research
(Samek et al., 2019). One of the fundamental proper-
ties associated with the interpretability of a model is
invertibility. When a model is invertible, each predic-
tion can be traced back to its source inputs, providing
the ability to decipher the decision-making process
(Kothari et al., 2021). This traceability is crucial for
diagnosing model behavior, identifying biases, and
ensuring accountability (Lipton, 2016). Invertibility
offers a multitude of avenues for interpretability. One
such avenue is the potential for the systematic pertur-
bation of single features of the intermediate represen-
tations, mapping them back to the data space, and the
subsequent interpretation of the reconstructed input
with perturbed features. However, unless the model
is designed to be invertible, the practical verification
is generally an ill-posed problem.
Given the prevalence of ReLU-layers as build-
ing blocks in neural networks, understanding how in-
a
https://orcid.org/0009-0006-2987-650X
b
https://orcid.org/0000-0001-8012-5521
c
https://orcid.org/0000-0002-3247-6279
d
https://orcid.org/0000-0003-4939-0831
formation flows through can be essential to under-
standing the whole model architecture and enhanc-
ing its interpretability. So, in this paper we focus on
putting existing theoretical results on the invertibility
of ReLU-layers (Haider et al., 2024; Maillard et al.,
2023; Puthawala et al., 2022) into practice. Our ap-
proach to this can be described as follows: For any
given weight matrix W and bias vector α we want to
assess if the associated ReLU-layer is invertible on
some set K, that contains data points of interest. By
appropriate sampling from (a part of) this dataset, in-
stead of K, we compute an approximation of a bias
vector α
∗
which is maximal with the property that the
associated ReLU-layer is invertible on K. Hence, if
α ≤ α
∗
(entry-wise) we can deduce that the original
ReLU-layer is invertible, too. We use this to promote
the invertibility of ReLU-layers by enforcing α ≤ α
∗
during training via regularization. With these exper-
iments, we explore the trade-off between model ex-
pressivity versus information preservation of ReLU-
layers.
This manuscript is organized as follows. In Sec-
tion 2 we present frame theory as theoretical back-
bone of our approach and derive a sufficient condi-
tion for the invertibility of ReLU-layers on a given
set. Section 3 introduces an algorithm that satisfies
the condition, which is demonstrated in practical ap-
plications in Section 4.
Eckert, H., Haider, D., Ehler, M. and Balazs, P.
Invertibility of ReLU-Layers: A Practical Approach.
DOI: 10.5220/0012951300003837
Paper published under CC license (CC BY-NC-ND 4.0)
In Proceedings of the 16th International Joint Conference on Computational Intelligence (IJCCI 2024), pages 423-429
ISBN: 978-989-758-721-4; ISSN: 2184-3236
Proceedings Copyright © 2024 by SCITEPRESS – Science and Technology Publications, Lda.
423

2 THEORETICAL
INVERTIBILITY OF
ReLU-LAYERS
Frame theory is a mathematical paradigm that deals
with stable and invertible representations (Chris-
tensen, 2003). We adopt the basic notions to study the
invertibility of ReLU-layers via the injectivity, i.e., in-
vertibility “from the left” of the mappings that provide
these representations.
2.1 Introduction to Frame Theory
Definition 2.1 (Frames). A matrix W ∈ R
m×n
with
row vectors w
i
∈ R
n
for i = 1, . . . , m induces a frame
for R
n
if there are constants A, B > 0 such that
A∥x∥
2
≤
m
∑
i=1
|⟨x, w
i
⟩|
2
≤ B∥x∥
2
(1)
holds for all x ∈ R
n
. The vectors w
i
are called the
frame elements and the collection (w
i
)
i=1,...,m
is called
a frame.
A frame is equivalent to a spanning set (Casazza
and Kutyniok, 2012), hence m ≥ n has to hold, and
the constants A, B determine the numerical stability of
the representation of x in terms of the inner products
(⟨x, w
i
⟩)
i=1,...,m
.
Definition 2.2 (Analysis Operator). The analysis op-
erator associated with the matrix W ∈ R
m×n
is given
by the mapping C : R
n
→ R
m
defined by
x 7→ (⟨x, w
i
⟩)
i=1,...,m
. (2)
We call (⟨x, w
i
⟩)
i=1,...,m
the representation of x, or
frame coefficients. The analysis operator is equiva-
lent to matrix multiplication, i.e. C(x) = W x.
Lemma 2.3. The analysis operator C : R
n
→ R
m
for
the matrix W is injective if and only if the matrix W ∈
R
m×n
induces a frame.
In the context of neural network layers, we may
interpret the above lemma as that the frame-inducing
property of a weight matrix W ∈ R
m×n
is character-
izing for the injectivity (i.e., invertibility) of the cor-
responding linear layer. Note that for linear layers
with i.i.d. randomly initialized weights that map into
a higher dimensional space (m ≥ n) the frame prop-
erty holds true with probability one (Blumensath and
Davies, 2009).
Note that the smallest possible bound B in (1)
coincides with the upper, and the largest possible A
with the lower Lipschitz bound of the analysis oper-
ator (i.e., the upper Lipschitz bound of the pseudo-
inverse).
2.2 Frame Theory for ReLU-Layers
It turns out that it is possible to derive a character-
ization of the invertibility of ReLU-layers on a set
K ⊆ R
n
using Lemma 2.3 (Haider et al., 2024). Let us
first define a ReLU-layer in this spirit of the analysis
operator as follows.
Definition 2.4 (ReLU-Layer). Let W ∈ R
m×n
, α ∈
R
m
, and C(x)
i
= ⟨x, w
i
⟩ denote the i-th component of
the analysis operator for W . The mapping ReLU
W,α
:
R
n
→ R
m
defined by
x 7→ (max(0,C(x)
i
− α
i
))
i=1,...,m
, (3)
is the ReLU-layer with weight matrix W and bias vec-
tor α.
More explicitly, the image of x under a ReLU-
layers can be written as
ReLU
W,α
(x)
i
=
(
C(x)
i
− α
i
if C(x)
i
≥ α
i
0 if C(x)
i
< α
i
.
(4)
Since C is injective if and only if W induces a frame it
is clear that the invertibility of ReLU
W,α
depends on
how the ReLU function affects the frame coefficients
of x. This motivates the following definition.
Definition 2.5 (Active Frame Elements). A frame el-
ement w
i
is called active for x and α if C(x)
i
≥ α
i
.
The index set of all active frame elements for given x
and α is denoted by
I
α
x
:= {i ∈ I : C(x)
i
≥ α
i
}. (5)
Using this notion, we consider frames which sat-
isfy that for all x ∈ K the active frame elements form
a frame:
Definition 2.6 (α-rectifying). A matrix W ∈ R
m×n
is
called α-rectifying on K ⊆ R
n
if for all x ∈ K the ac-
tive frame elements for x and α form a frame. In
particular, we call W α-rectifying for x if it is α-
rectifying on the set {x}.
This definition is naturally connected to the frame
theoretic condition for the invertibility of a ReLU-
layer on some open or convex set K:
Theorem 2.7 ((Haider et al., 2024)). If W ∈ R
m×n
is
α-rectifying on an open or convex set K ⊆ R
n
then
ReLU
W,α
is invertible on K.
This allows us to approach the invertibility of
ReLU-layers by verifying that the weight matrix W is
α-rectifying on some open or convex set K. We will
make an additional assumption that further simplifies
the problem.
Definition 2.8 (Full Spark (Alexeev et al., 2012)).
A matrix W ∈ R
m×n
is called full spark if each sub-
collection of n row vectors forms a frame.
NCTA 2024 - 16th International Conference on Neural Computation Theory and Applications
424

With this assumption, the verification of the α-
rectifying property can be reduced to a counting ar-
gument.
Lemma 2.9. Let W ∈ R
m×n
be full spark and K ⊆ R
n
.
If the cardinality of the set of active frame elements
satisfies
|
I
α
x
|
≥ n for all x ∈ K then W is α-rectifying
on K.
Similarly as for the frame property, matrices with
entries that are i.i.d. samples from an absolutely con-
tinuous probability distribution are known to be full
spark with probability one (Blumensath and Davies,
2009). Although not proven, it is reasonable to as-
sume that the full spark property is also preserved
during gradient steps in training. As a consequence,
a full spark assumption is very natural in the context
of neural network training. Moreover, it is also one
of the fundamental ingredients for the algorithm that
we propose to approximately check the α-rectifying
property in practice.
3 DATA DRIVEN MONTE CARLO
BIAS ESTIMATION
We now introduce a method called Monte Carlo Bias
Estimation (MCBE) to approximately check the α-
rectifying property of a weight matrix W ∈ R
m×n
on
finitely many data points. (Approximate) invertibility
of the associated ReLU-layer can be concluded from
Theorem 2.7 only if all the points lie within an open
or convex set K where W is α-rectifying. MCBE
circumvents this by addressing the problem from a
stochastic point of view: By drawing data points ac-
cording to an approximation of the distribution of a
given data set we conclude the α-rectifying property
of W on all data that follow the same distribution.
This shall provide a sufficient condition for the ap-
proximate invertibility of ReLU
W,α
for all data points
that are close to the initial ones with high probability.
3.1 Motivation and Overview
For a set of points X
N
= {x
1
, . . . , x
N
} in R
n
the out-
put of MCBE is a bias vector α
∗
such that W is α-
rectifying on X
N
for all α ≤ α
∗
with high probabil-
ity. Here, “≤” is meant in a entry-wise sense, i.e.,
α
i
≤ α
∗
i
for all i = 1, . . . , m. To obtain such a maxi-
mal bias we iteratively compute biases α(x
i
) such that
W is α(x
i
)-rectifying on {x
i
} and update them in a
suitable manner. According to this idea, the follow-
ing theorem provides the theoretical foundation for
MCBE.
Theorem 3.1 (Maximal Bias). Let W ∈ R
m×n
be full
spark, and let α
∗
(x) for x ∈ R
n
be given as
α(x)
j
:=
(
C(x)
j
if j ∈ J
x
∞ else ,
(6)
where J
x
denotes the index set corresponding to the
n largest entries of the analysis operator. Then W is
α-rectifying on {x} if α ≤ α(x).
Moreover, let X
N
= {x
1
, . . . , x
N
} ⊂ R
n
and α
∗
(X
N
)
be given as
α
∗
(X
N
)
j
= min
i=1,...,N
α(x
i
)
j
. (7)
Then W is α-rectifying on X
N
if α ≤ α
∗
(X
N
).
Proof. Let α ≤ α(x), then in particular,
α
j
≤ α(x) = C(x)
j
for all j ∈ J
x
. (8)
Since J
x
contains n elements, the frame has n active
frame elements and, therefore, is α-rectifying on {x}
by Definitions 2.6 and 2.8. This shows the first part.
For the moreover part, let α ≤ α
∗
(X
N
) then
α ≤ α
∗
(X
N
) =⇒ α ≤ α(x
i
)
for all i = 1, . . . , N. Consequently, W is α-rectifying
on {x
i
} for all i and therefore also on X
N
.
3.2 Implementation
The basic functionality of the algorithm for MCBE
can be summarized as follows. Let k denote the k-th
iteration.
k = 0 : (α
∗
j
)
0
= ∞ for all j
0 < k < N : (α
∗
j
)
k
= min{(α
∗
j
)
k−1
, α(x
k
)
j
}
for j ∈ J
x
k
and x
k
∈ X
N
k = N : (α
∗
)
N
=: α
∗
(X
N
)
For a more detailed pseudo-code of the algorithm we
refer to the appendix. As by Theorem 3.1, W is
α
∗
(X
N
)-rectifying on X
N
. To conclude invertibility
of ReLU
W,α
on a specific data set of interest we de-
rive the sampling set X
N
from a given training data
set using randomized smoothing (Cohen et al., 2019).
It is important to note that this approach circumvents
the problem that the number of necessary samples, to
cover a set K sufficiently dense, explodes in high di-
mensions (Reznikov and Saff, 2016).
Definition 3.2 (Randomized Smoothing (Cohen et al.,
2019)). Let X
(train)
⊂ K be a collection of M data
points x
(train)
1
, . . . , x
(train)
M
that all follow the same dis-
tribution on K. The procedure of uniformly draw-
ing points from X
(train)
and adding Gaussian noise
Invertibility of ReLU-Layers: A Practical Approach
425

Figure 1: Illustration of randomized smoothing on Iris data.
Dark red dots represent the original data points light gray
dots sampled data. Left: Uniformly sampled data on the
ball. Right: Samples obtained from randomized smoothing,
respecting the distribution of data set well.
is called randomized smoothing of X
(train)
. Hence, a
point x in the resulting sampling set is given by
x = x
(train)
j
+ ε with ε ∼ N (0, σI
n
)
for some σ ∈ R, and j chosen uniformly.
The sampling set obtained from randomized
smoothing is an augmented data set X
N
that follows
approximately the same distribution as the training
data. This makes it a suitable choice for the procedure
described in Theorem 3.1. Figure 1 shows how this
sampling methods looks like on a three-dimensional
version of the Iris dataset.
4 PRACTICAL INVERTIBILITY
OF ReLU-LAYERS
With the possibility to check the approximate invert-
ibility of ReLU-layers we can directly study the cir-
cumstances under which ReLU-layers are invertible.
In the following we demonstrate how MCBE can be
used to study the correlation of the performance of a
neural network model and the invertibility of a ReLU-
layer as part of the model architecture. On the one
hand, we aim to understand if invertibility is natu-
rally occurring in learning a ReLU-layer, and one step
further, whether enforcing its invertibility comes at
the cost of the performance of the whole model. For
all experiments, we solve a classification task on two
standard benchmark datasets: Iris (Fisher, 1988), and
MNIST (LeCun et al., 2010). For detailed descrip-
tions of the model architectures and training proce-
dures we refer to the appendix.
4.1 Monitor Invertibility
Our first experimental goal is to monitor the invert-
ibility of the first ReLU-layer in the respective mod-
els during training. We define the redundancy of a
Figure 2: Invertibility of the first ReLU-layer of a neural
net quantified in two ways. Left: The percentage of points
of the test set, for which perfect reconstruction is possi-
ble. Right: The percentage of bias values α
i
such that
α
i
≤ α
∗
i
(X
N
). Different color and line style indicate dif-
ferent redundancy settings.
layer by its output dimension m divided by its input
dimension n. We always compare three different re-
dundancy configurations:
m
n
= 2, 3, 9.
We can quantify the invertibility of a ReLU-layer
with bias vector α in two different ways. Let X
N
=
{x
1
, . . . , x
N
}.
1. Naively: Measure the percentage of points x ∈ X
N
such that
|
I
α
x
|
≥ n holds. This can be interpreted
as the percentage of points x ∈ X
N
for which the
ReLU-layer is invertible, and therefore, recon-
struction without loss is possible.
2. MCBE: Measure the percentage of the bias val-
ues α
i
such that α
i
≤ α
∗
i
(X
N
), which is the per-
centage of bias values that do not have to be
changed to make the layer invertible.
We plot the results for both ways in Figure 2. It is cru-
cial to acknowledge that the two measures represent
distinct approaches to invertibility. Consequently, it
is anticipated that there will be some discrepancy be-
tween the measures. We observe, that when using a
ReLU-layer with redundancy two on MNIST (upper
left in Fig. 2), only about 30% of the test data points
can be reconstructed perfectly, and 10% of the bias
values fulfill the proposed condition. Note that the
jump from redundancy two to three has a significant
effect in terms of invertibility, observed in both mea-
sures.
Moreover, across datasets and measures, we ob-
serve that training does not influence the invertibility
of the layer significantly, seconding the comment in
Sec. 2.2.
Backed by this experiment we assume that enforc-
ing invertibility will not influence the performance of
the model remarkably. We will test this assumption in
Section 4.2.
NCTA 2024 - 16th International Conference on Neural Computation Theory and Applications
426

Figure 3: Classification accuracy of a neural net with an
approximately invertible ReLU-layer (red dashed line) as
the first layer, compared to non-regularized baseline during
training. Accuracy is measured on the test set and the re-
dundancy of the layers is
m
n
= 2.
4.2 Enforce Invertibility During
Training
Our second experimental goal is to use the bias
α
∗
(X
N
) from MCBE in a regularization procedure
where the ReLU-layer is gradually promoted to be in-
vertible during training. We implement this by adding
ρ
inj
(α) =










max{0, α
1
− α
∗
(X
N
)
1
}
.
.
.
max{0, α
m
− α
∗
(X
N
)
m
}










2
to the learning objective. This term affects those bias
values where α
i
> α
∗
(X
N
)
i
and has no effect, other-
wise. In this sense, it penalizes non-invertibility.
Across the datasets, we found that enforcing in-
vertibility can come with a trade-off with the perfor-
mance of the model during training (Figure 3). For
MNIST, this trade-off is even negligible. At the end
of the training, both models achieved the same accu-
racy while the reconstruction error could be signifi-
cantly reduced (Table 1). In Section 4.3 we go into
more detail about reconstruction.
4.3 Reconstruction
The input x ∈ R
n
of a ReLU-layer can be recon-
structed from its output z = ReLU
W,α
(x) ∈ R
m
by cal-
culating the least-squares solution x
∗
of
W
+
x = z
+
+ α
+
, (9)
where W
+
is the matrix, whose row vectors are the
row vectors w
i
of W that satisfy (ReLU
W,α
(x))
i
> 0.
The bias α
+
and z
+
are defined analogously (Haider
et al., 2023). If the ReLU-layer is invertible, the re-
construction is exact, i.e., x = x
∗
. In Table 1, we
show the mean reconstruction errors on the test set
with and without enforcing injectivity during train-
ing. The mean reconstruction error is defined as the
Euclidean distance between the input x and the recon-
structed input x
∗
. Although we do not obtain perfect
Figure 4: Original (left) and reconstructed MNIST images
from the output of trained ReLU-layers. Mid: Approxi-
mately invertible. Right: Not invertible. The input dimen-
sion is n = 784 and the output dimension m = 1568.
Table 1: Mean reconstruction error measures on the test set
with and without enforcing injectivity during training.
Data set approx. invertible unregularized
IRIS 0.84 2.02
MNIST 0.79 1.63
reconstruction in our examples, the approximately in-
vertible layers provide an error reduction by more
than a factor of two. In Figure 4 we show a visual
comparison of a reconstructed MNIST image using
an approximately invertible ReLU-layer and a non-
regularized one.
4.4 Stability Analysis
A standard metric for evaluating the stability of a neu-
ral network layer is via its upper Lipschitz bound,
which represents the maximal factor by which dis-
tances of input vectors can be amplified by applying
the layer. In the context of invertibility, the lower Lip-
schitz bound of the layer is of central interest since
it corresponds to the upper Lipschitz bound of the
inverse - provided that it exists. In this sense, a
ReLU-layer ReLU
W,α
is bi-Lipschitz stable if there
are A, B > 0 such that
A||x−x
′
||
2
≤ || ReLU
W,α
(x)−ReLU
W,α
(x
′
)||
2
≤ B||x−x
′
||
2
(10)
holds for all x, x
′
∈ R
n
. The following has been shown
in (Bruna et al., 2014).
Theorem 4.1 (ReLU Bi-Lipschitz Bounds). Let W ∈
R
m×n
and α ∈ R
m
. For any x ∈ R
n
let W
I
α
x
(x) be de-
fined as the matrix whose row vectors are the active
frame elements of W . Let us further denote the small-
est singular value of some matrix V as λ
−
(V ) and
the largest singular value as λ
+
(V ). Then upper and
lower Lipschitz bounds of ReLU
W,α
are given as
A
0
= min
x
λ
−
(W
I
α
x
(x)),
B
0
= max
x
λ
+
(W
I
α
x
(x)).
Moreover, ReLU
W,α
is invertible if and only if A > 0.
Note that λ
−
(W
I
α
x
(x)) and λ
+
(W
I
α
x
(x)) correspond
to the lower and upper frame bounds (1) of the sub-
frame that is active for x and α.
Invertibility of ReLU-Layers: A Practical Approach
427

Figure 5: Sets A, B, defined such that high values in the set
A indicate stability of the inverse mapping and low values
in the set B indicate stability of the ReLU-layer.
To visualize the effect on the stability of enforcing
injectivity with our described method in Figure 5 we
plot the sets
A = {λ
−
(W
I
α
x
(x)) for x ∈ X
(test)
}
and
B = {λ
+
(W
I
α
x
(x)) for x ∈ X
(test)
}.
These sets contain the true Lipschitz bounds A
0
and
B
0
as minimum and maximum, respectively. Low val-
ues in the set B indicate good stability behavior of the
corresponding ReLU-layer in the classic sense, while
high values in the set A indicate good stability be-
havior of the inverse mapping. Zero values in the set
A correspond to points, where the ReLU-layer is lo-
cally not invertible. Across the datasets, we found,
that enforcing injectivity reduces the stability of the
ReLU-layer but increases the stability of the inverse.
4.5 Discussion
Our experiments demonstrate that approximate in-
vertibility can be enforced by adding a term to the
loss function that penalizes bias values that are larger
than the estimated maximum. With this the recon-
struction loss from the layer output can be decreased
significantly without loss of accuracy at the end of
training. However, there is a natural trade-off between
the upper Lipschitz bound of the layer and the Lips-
chitz bound of its inverse mapping. Further research
is needed to identify if the observed behavior is spe-
cific to the considered examples or more general.
5 CONCLUSION
This paper introduces a Monte Carlo-type method to
numerically verify a sufficient condition for the in-
vertibility of a ReLU-layer in a neural network. For
any given weight matrix and dataset the algorithm cal-
culates a bias α
∗
which is maximal with the prop-
erty that any ReLU-layer with bias α is invertible
if α ≤ α
∗
. This can be used to study the informa-
tion flow in ReLU-layers in an alternative manner by
comparing α with α
∗
during training. Moreover, it
can be used to enforce invertibility by penalizing bias
values that exceed the maximal bias during training.
We demonstrate these applications in two basic ex-
amples. Our findings are that the performance of the
model is only slightly hindered by enforcing invert-
ibility of their ReLU-layers, suggesting that invertible
models can achieve comparable performance to their
non-invertible counterparts.
ACKNOWLEDGMENT
The research presented in this paper is derived from
the first author’s Master’s thesis submitted to the Uni-
versity of Vienna which contains a more comprehen-
sive treatment of the subject matter. Financial sup-
port comes from the Austrian Science Fund (FWF)
projects LoFT (P 34624) and NoMASP (P 34922). D.
Haider is recipient of a DOC Fellowship of the Aus-
trian Academy of Sciences at the Acoustics Research
Institute (A 26355). We thank the reviewers for their
time reviewing the paper and their feedback.
REFERENCES
Alexeev, B., Cahill, J., and Mixon, D. G. (2012). Full spark
frames. Journal of Fourier Analysis and Applications,
18:1167–1194.
Blumensath, T. and Davies, M. E. (2009). Sampling theo-
rems for signals from the union of finite-dimensional
linear subspaces. IEEE Transactions on Information
Theory, 55(4):1872–1882.
Bruna, J., Szlam, A., and Lecun, Y. (2014). Signal recovery
from pooling representations. In Proceedings of In-
ternational Conference on Machine Learning (ICML),
pages 1585–1598.
Casazza, P. G. and Kutyniok, G. (2012). Finite frames: The-
ory and applications. Springer.
Christensen, O. (2003). An Introduction to Frames and
Riezs Bases. Birkh
¨
auser.
Cohen, J., Rosenfeld, E., and Kolter, Z. (2019). Certified
adversarial robustness via randomized smoothing. In
Proceedings of International Conference on Machine
Learning (ICML), pages 1310–1320. PMLR.
Fan, F., Xiong, J., Li, M., and Wang, G. (2020). On in-
terpretability of artificial neural networks: A survey.
IEEE Transactions on Radiation and Plasma Medical
Sciences, 5:741–760.
Fisher, R. A. (1988). Iris. UCI Machine Learning Reposi-
tory.
Haider, D., Balazs, P., and Ehler, M. (2023). Convex geom-
etry of ReLU-layers, injectivity on the ball and local
NCTA 2024 - 16th International Conference on Neural Computation Theory and Applications
428

reconstruction. In Proceedings of International Con-
ference on Machine Learning (ICML), pages 12339–
12350.
Haider, D., Ehler, M., and Balazs, P. (2024). Injectivity of
ReLU-layers: Perspectives from frame theory. arXiv,
abs/2406.15856.
Kingma, D. P. and Ba, J. (2017). Adam: A method for
stochastic optimization.
Kothari, K., Khorashadizadeh, A., de Hoop, M., and Dok-
mani
´
c, I. (2021). Trumpets: Injective flows for in-
ference and inverse problems. In de Campos, C. and
Maathuis, M. H., editors, Proceedings of the 37th
Conference on Uncertainty in Artificial Intelligence,
volume 161 of Proceedings of Machine Learning Re-
search, pages 1269–1278.
LeCun, Y., Cortes, C., and Burges, C. (2010). MNIST hand-
written digit database. ATT Labs [Online]. Available:
http://yann.lecun.com/exdb/mnist, 2.
Lipton, Z. (2016). The mythos of model interpretability.
Communications of the ACM, 61.
Maillard, A., Bandeira, A. S., Belius, D., Dokmani
´
c, I.,
and Nakajima, S. (2023). Injectivity of relu net-
works: perspectives from statistical physics. arXiv,
abs/2302.14112.
Puthawala, M., Kothari, K., Lassas, M., Dokmani
´
c, I.,
and de Hoop, M. (2022). Globally injective ReLU
networks. Journal of Machine Learning Research,
23(105):1–55.
Reznikov, A. and Saff, E. B. (2016). The covering
radius of randomly distributed points on a mani-
fold. International Mathematics Research Notices,
2016(19):6065–6094.
Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., and
M
¨
uller, K.-R., editors (2019). Explainable AI: Inter-
preting, Explaining and Visualizing Deep Learning.
Springer Cham.
APPENDIX
The MCBE Algorithm
This algorithm represents an extension of the one ini-
tially proposed by Haider et al. in (Haider et al.,
2024).
Neural Network Architectures and
Training Details
Both datasets were normalized before further process-
ing. In both cases an Adam optimizer (Kingma and
Ba, 2017) with a learning rate of 0.01 was used.
Data: weight matrix W ∈ R
m×n
, train data
X
(train)
∈ R
M×n
, number of iterations s,
for which α has to stay the same for
convergence
Initialize (α
∗
(x
1
, . . . , x
N
)
j
)
0
= −∞ for
j = 1, . . . , m;
while
||(α
∗
(x
1
, . . . , x
N
))
i−s
− (α
∗
(x
1
, . . . , x
N
))
i
|| do
sample x
(train)
i
from X
(train)
;
sample ε
i
from N (0, σI
n
);
set x
i
= x
(train)
i
+ ε
i
;
calculate J
x
i
;
for j ∈ J
x
i
do
(α
∗
(x
1
, . . . , x
N
)
j
)
i
=
min{(α
∗
(x
1
, . . . , x
N
)
j
)
i−1
, ⟨x
i
, w
j
⟩}
end
i = i + 1
end
Algorithm 1: Monte Carlo Bias Estimation using ran-
domized smoothing.
Neural Network Architecture for Iris and
Redundancy
m
n
∈ {2, 3} of the First Layer
For the Iris data and redundancy
m
n
∈ {2, 3}, we
used two fully connected layers with ReLU activation
function followed by an output layer with Softmax ac-
tivation function.
1. FC1: m units, ReLU activation
2. FC2: 27 units, ReLU activation
3. Output Layer: 3 units, Softmax activation
Neural Network Architecture for Iris and
Redundancy
m
n
∈ {2, 3} of the First Layer
For the Iris data and redundancy
m
n
∈ {2, 3}, we used
one fully connected layer with ReLU activation func-
tion followed by an output layer with Softmax activa-
tion function.
1. FC1: 27 units, ReLU activation
2. Output Layer: 3 units, Softmax activation
Neural Network Architecture for MNIST
For the MNIST data, we used one fully connected lay-
ers with ReLU activation function followed by an out-
put layer with Softmax activation function.
1. FC1: m units, ReLU activation
2. Output Layer: 10 units, Softmax activation
Invertibility of ReLU-Layers: A Practical Approach
429