MAXIMUM TOLERANCE AND
MAXIMUM GREATEST TOLERANCE
Weights and Threshold of Strict Separating Systems
J. Freixas and X. Molinero
∗
Escola Polit
`
ecnica Superior d’Enginyeria de Manresa, DMA3 / LSI, and UPC, E-08240 Manresa, Spain
Keywords:
Neural nets, Circuit-switching networks, Deterministic and structural pattern recognition.
Abstract:
An important consideration when applying neural networks is the sensitivity to weights and threshold in strict
separating systems representing a linearly separable function. Two parameters have been introduced to mea-
sure the relative errors in weights and threshold of strict separating systems: the tolerance and the greatest
tolerance. Given an arbitrary separating system we study which is the equivalent separating system that pro-
vides maximum tolerance or/and maximum greatest tolerance.
1 INTRODUCTION
Recent research in Capacitive Threshold Logic (Sang-
Hoon and Lee, 1995), (Ozdemir et al., 1996) and
(Beiu et al., 2003) has revived interest in this area,
and it has re–introduced some of the problems that
have yet to be solved. One of the main issues of
threshold logic is the application of neural networks
to the problem of realizing Boolean functions, the lin-
ear separability problem has been dealt with, among
others, in (Yao and Ostapko, 1968), (Roychowdhury
et al., 1994), (Siu et al., 1995), (Picton, 1991), (Eli-
zondo, 2004) and (Elizondo, 2006). Neural networks
are usually designed to have the ability to learn and
generalize.
A neural network is capable of setting and
input–output mapping by adjusting its threshold and
weights. How can network designers predict the ef-
fect of threshold and weight perturbations on neu-
ral networks’ output, which are unavoidable because
of limited precision of digital an analog hardware?
What is then a tight bound for weights and threshold
to prevent output deviation? Which is the sharpest
bound one may consider for weights and threshold
to maintain the linearly separable function unchange-
∗
Research partially supported by Grants SGR 2009-
1029 and ALBCOM-SGR 2009-1137 of “Generalitat
de Catalunya”, programme TIN2006-11345 (ALINEX)
from the Spanish “Ministerio de Educaci
´
on y Ciencia”,
MTM2009-08037 from the Spanish Science and Inno-
vation Ministry, and 9-INCREC-11 of “(PRE/756/2008)
Ajuts a la iniciaci
´
o/reincorporaci
´
o a la recerca (Universitat
Polit
`
ecnica de Catalunya)”
able? Is there any guideline for designing a more ro-
bust and safer neural network? These kind of prob-
lems were deeply studied in the sixties (Hu, 1960),
(Elgot, 1961), (Myhill and Kautz, 1961) and (Winder,
1962), until (Hu, 1965) proposed as a solution to the
main problem, a number (defined for each strict sep-
arating system) which he called, the tolerance. Re-
cently (Freixas and Molinero, 2008a) proposed a new
bound which improves the tolerance, the greatest tol-
erance (G-tolerance), and proved that the G-tolerance
is the greatest bound one may consider.
In this work we consider two parameters for an
arbitrary linearly separable switching function: the
number of variables n and the number of types
2
of
distinguished variables k. For each pair (n,k) we
are interested in determining the maximum achiev-
able value for the tolerance and for the G-tolerance.
Moreover, we demonstrate that taking strict separat-
ing systems with positive integer (natural) weights are
enough to our purpose.
2 PRELIMINARIES
Let Q be the set {0,1}. For any given positive in-
teger n, consider the cartesian power product Q
n
=
Q ×···×Q. Thus, the elements of Q
n
are the 2
n
or-
dered n-tuples (x
1
,...,x
n
), with variables x
i
∈ {0,1}
for all i = 1,...,n. By a switching function of n vari-
ables, we mean a function f : Q
n
→ Q from the n-
2
A type is a set of variables which are equivalent among
them.
511
Freixas J. and Molinero X. (2010).
MAXIMUM TOLERANCE AND MAXIMUM GREATEST TOLERANCE - Weights and Threshold of Strict Separating Systems.
In Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Artificial Intelligence, pages 511-514
DOI: 10.5220/0002706905110514
Copyright
c
 SciTePress

cube Q
n
into Q. If f is not surjective then either
{x ∈ Q
n
: f (x) = 0} =
/
0 or {x ∈ Q
n
: f (x) = 1} =
/
0
that is, f is a constant function. The two constant
functions are called the two trivial switching func-
tions. A switching function is monotonic if: (i) f is
not decreasing in each argument; and (ii) f (0) = 0
and f (1) = 1. The two constant Boolean functions
are usually considered to be monotonic, but here the
restriction (ii) relegates them to be non-monotonic. In
a monotonic switching function a variable i is relevant
if there exists at least x, y ∈ Q
n
, x ≥ y with f (x) = 1,
f (y) = 0, x
i
6= y
i
and y
j
= x
j
for j 6= i. Note that
irrelevant (that is, not relevant) variables do not add
any value to the outcome f , i.e., if i is irrelevant then
f (x) = f (y) whenever x
i
6= y
i
and y
j
= x
j
for j 6= i.
A switching function f : Q
n
→Q is a linearly sep-
arable function or threshold function if it admits a
system of n + 1 real numbers T , w
1
,...,w
n
, denoted
by [T ;w
1
,...,w
n
] such that for each arbitrary point
x = (x
1
,...,x
n
) in the n-cube Q
n
we have
w(x) ≥ T, if f (x) = 1,
w(x) < T, if f (x) = 0;
being w(x) =
∑
n
i=1
w
i
x
i
.
The n real numbers w
1
,...,w
n
in this system are
called the weights, and the first real number T is re-
ferred to as the threshold. By the finiteness of Q
n
it is
always possible to modify the threshold in such a way
that the previous definition could be rewritten using
strict inequalities. In this case, the system is called a
strict separating system for the linear separable func-
tion f . Thus, from now on we will just consider strict
separating systems.
Note that a switching function has only two pos-
sible values, true or false. True or false can also be
referred, as we do here, to as 1 or 0 often used in com-
puter programming, on or off as seen with computer
hardware circuits, firing or resting used to describe the
state of an artificial neuron, functioning state or fail-
ing state as seen with reliability, and voting in favor
(“yea”) or against (“nay”) in binary decision–making
mechanisms. If one draws the graph of a linearly sep-
arable switching function of two variables in a square
with the elements of Q
2
as vertices, it takes only one
straight line to separate the true outputs from the false
outputs. In general, for n variables it is required an
(n −1)-hyperplane to separate the true outputs from
the false outputs.
It is important to point out that both, the tolerance
and the G-tolerance, we are going to introduce next,
are defined for each strict separating system and, thus
they depend on the threshold and the set of weights
chosen to implement the linearly separable switching
function.
2.1 The Tolerance
In this part we are going to define the tolerance in-
troduced by (Hu, 1960). Let A denote the maximum
of the function w(x) for all x such that f (x) = 0,
A = max
x : f (x)=0
w(x), and let B denote the minimum of
the function w(x) for all x such that f (x) = 1, B =
min
x : f (x)=1
w(x).
Figure 1: Separating hyperplane for a threshold function
defined in Q
3
.
The two trivial switching functions are linearly
separable; if f ≡ 0, we set A = −∞; if f ≡ 1, we set
B = ∞. Then we have A < T < B. Let m denote the
smallest of the two positive numbers T −A and B−T .
On the other hand, let M =
|
T
|
+
∑
n
i=1
|
w
i
|
. Then,
for each point x ∈ Q
n
we have
|
T
|
+
∑
n
i=1
|
w
i
|
x
i
≤ M.
Let λ
1
,...,λ
n
and Λ be n + 1 arbitrary real num-
bers and let w
0
i
= (1 + λ
i
)w
i
for all i = 1,...,n, and
T
0
= (1 + Λ)T . Then, the real numbers λ
1
,...,λ
n
and Λ represent the relative errors if we use the num-
bers w
0
1
,...,w
0
n
and T
0
instead of the original non–null
numbers w
1
,...,w
n
and T as weights and threshold.
In other words, if we initially start with [T ; w
1
,...,w
n
]
which is transformed into [T
0
;w
0
1
,...,w
0
n
] then λ
i
=
w
0
i
−w
i
w
i
for all i = 1, . . . , n and Λ =
T
0
−T
T
where it is re-
quired w
i
and T to be different from zero. That is to
say, these numbers are the relative errors in weights
and threshold, that is why we use the term “error” in
the title of this paper.
Theorem 2.1 (Hu, 1965) Let f : Q
n
→ Q be an arbi-
trary linearly separable switching function of n vari-
ables, let [T ;w
1
,...,w
n
] be a given strict separat-
ing system
3
for f and let τ[T ; w
1
,...,w
n
] :=
m
M
be
the tolerance for [T ; w
1
,...,w
n
]. If
|
λ
i
|
< τ for each
i = 1, . . . ,n and if
|
Λ
|
< τ then [T
0
;w
0
1
,...,w
0
n
] is a
strict separating system for the given linearly separa-
ble switching function f .
Hu called this positive number the tolerance of the
separating system and denoted it τ[T ; w
1
,...,w
n
].
Note that the tolerance is well–defined, in fact, as
w(0) = 0 for any strict separating system, it occurs
that T 6= 0 and so M 6= 0.
3
τ[T ; w
1
,. .., w
n
] is denoted by τ if there is no confusion
about the strict linearly separating system
ICAART 2010 - 2nd International Conference on Agents and Artificial Intelligence
512

2.2 The Greatest Tolerance
(Freixas and Molinero, 2008a) improves the Hu’s tol-
erance. They find the greatest positive real number δ
such that if
|
Λ
|
< δ and
|
λ
i
|
< δ, for all i = 1,...,n;
then [T
0
;w
0
1
,...,w
0
n
] is equivalent to [T ;w
1
,...,w
n
],
i.e., both strict separating systems still represent the
same linearly separable switching function.
Definition 2.2 (Freixas and Molinero, 2008a) Given
a strict separating system [T ;w
1
,...,w
n
], for each
x ∈ Q
n
let a(x) =
|
w(x) −T
|
, b(x) =
|
T
|
+
∑
n
i=1
|
w
i
|
x
i
and χ[T ; w
1
,...,w
n
] = min
x∈Q
n
a(x)
b(x)
. This number
4
is
called the greatest tolerance (briefly, G-tolerance) of
[T ; w
1
,...,w
n
]. Note that the G-tolerance of the strict
separating system [T ; w
1
,...,w
n
] depends on the cho-
sen threshold and weights. Because of the finiteness
of Q
n
, χ is attained for, at least one x ∈Q
n
, let x
0
∈Q
n
be one of the points attaining the G-tolerance.
Theorem 2.3 (Freixas and Molinero, 2008a) Let f :
Q
n
→ Q be an arbitrarily linearly separable switch-
ing function of n variables and let [T ;w
1
,...,w
n
] be
a given strict separating system for f . If
|
λ
i
|
< χ for
each i = 1,...,n and if
|
Λ
|
< χ then:
(i) [T
0
;w
0
1
,...,w
0
n
] is a strict separating system for
the given linearly separable switching function f .
(ii) χ is the greatest upper bound for the constants
λ
1
,...,λ
n
,Λ.
3 MAXIMUM TOLERANCE AND
MAXIMUM G-TOLERANCE
Let f be an arbitrary monotonic
5
linearly separable
switching function of n variables, now we are looking
for a strict linearly
6
separating system [T ;w
1
,...,w
n
]
representing f with maximum achievable tolerance
and maximum achievable G-tolerance.
We start by relating the tolerance (the G-tolerance)
of an arbitrary strict separating system with the toler-
ance (G-tolerance) of a strict separating system with
natural weights.
Theorem 3.1 If [T ; w
1
,...,w
n
] ∈ R × R
n
is a real
strict separating system, then there exists an equiv-
alent strict separating natural system, i.e., a system
with natural weights w
0
1
,...,w
0
n
, with the same toler-
ance and the same G-tolerance.
4
χ[T ; w
1
,. .., w
n
] is denoted by χ if there is no confusion
about the strict separating system
5
From now on we will omit the word monotonic.
6
From now on we will omits the word linearly.
Note that, even though the weights are the same
natural numbers for both systems (the system applied
to the tolerance and the system applied to the G-
tolerance), the threshold can be a different real num-
ber for each system.
An obvious corollary for maximums arises:
Corollary 3.2 If [T ;w
1
,...,w
n
] ∈ R × R
n
is a real
strict separating system with maximum tolerance
(maximum G-tolerance), then there exists an equiv-
alent strict separating natural system with the same
maximum tolerance (maximum G-tolerance).
Second, we propose a way to get maximum toler-
ance and maximum G-tolerance.
Theorem 3.3 Maximum tolerance and maximum G-
tolerance among all equivalent strict separating sys-
tems are attained when the weights are natural num-
bers and their sum is the minimum achievable one.
Given a strict separating system with fixed
weights, [T ; w
1
,...,w
n
], it is known (Freixas and
Molinero, 2008a) that adjusting the corresponding
threshold,
A+B
2
for the tolerance and
√
AB for the
G-tolerance, the system achieves the maximum tol-
erance and the maximum G-tolerance, respectively.
Thus, given a strict separating system of an arbitrary
separable switching function, we have a procedure to
compute a strict separating system with maximum tol-
erance or/and maximum G-tolerance, respectively.
For instance, let [5; 8, 8, 6, 3, 3] be a strict sepa-
rating system, then τ =
1
5+28
= 0.0303...; but the
equivalent strict separating natural system with natu-
ral weights having minimum sum and threshold equal
to
A+B
2
, i.e., [
3
2
;2,2,2,1,1], achieves the maximum
available tolerance τ
0
=
1/2
3/2+8
= 0.0526.... In the
same vein, the strict separating system [5; 8,8,6,3,3]
has χ =
1
11
= 0.0909..., but the equivalent strict sep-
arating natural system with natural weights having
minimum sum and threshold equal to
√
AB, i.e.,
[
√
2;2,2,2,1,1], achieves the maximum available G-
tolerance χ
0
=
√
2−1
√
2+1
= 0.1715....
Another important new result is how we get the
maximum tolerance with n variables:
Proposition 3.4 The maximum tolerance for n vari-
ables is given by the following strict separating (nat-
ural) system:

1
2
; 1, ..., 1

, and such tolerance is
τ = 1/(1 + 2n).
Any other strict (natural or not) separating system
(of non null weights) with tolerance τ
0
fulfills τ
0
≤ τ.
Finally, we extend this result for systems with n
variables and k distinguished types of variables, i.e.,
with exactly k non-equivalent variables:
Conjecture 3.5 The maximum achievable tolerance
for n variables and k types of distinguished variables
MAXIMUM TOLERANCE AND MAXIMUM GREATEST TOLERANCE - Weights and Threshold of Strict Separating
Systems
513

is obtained by the following strict separable system:
[k −1/2; k,k −1,k −2,...,1,1,...,1
| {z }
n−k
]
and such a tolerance is τ = 1/(2n + k
2
+ k −1).
Again, any other strict linearly (natural or not)
separating system (of non null weights) with tolerance
τ
0
fulfills τ
0
≤ τ.
In the same vein, we establish the corresponding
conjecture for the G-tolerance:
Conjecture 3.6 The maximum G-tolerance for n
variables and k types (k > 1) of distinguished vari-
ables is given by the following strict separable sys-
tem:
[
p
k(k −1);k,k −1,k −2,..., 1, 1, . . . , 1
| {z }
n−k
].
and such a G-tolerance is χ =
√
k(k−1)−(k−1)
√
k(k−1)+(k−1)
.
In general, given a strict separating system for an
arbitrary separable switching function, we are able to
find an equivalent strict separating system (with nat-
ural weights having minimum integer sum) achiev-
ing the maximum tolerance and the maximum G-
tolerance. Moreover, it is known, see (Freixas and
Molinero, 2008b), that for less than 8 variables all
linearly separable switching functions have, up to iso-
morphism, a unique strict separating system with nat-
ural weights having minimum sum. So, we only
need to adjust the threshold,
A+B
2
for the tolerance
and
√
AB for the G-tolerance, for these systems to
achieve the maximum tolerance and the maximum
G-tolerance, respectively. Note that for 8 variables
the situation changes: There are 154 linearly separa-
ble switching functions, up to isomorphism, with two
strict separating systems having natural weights and
minimum sum.
4 FUTURE WORK
Generating a huge number of random strict separating
natural systems and then to conjecturate what distri-
bution follows the tolerance, the maximum tolerance,
the G-tolerance and the maximum G-tolerance.
It is also interesting to provide tables of all, up
to isomorphism, linearly separable switching func-
tions of a reasonable high number of variables with:
a strict separating system achieving its tolerance; its
tolerance; a strict separating system achieving its G-
tolerance; and its G-tolerance.
Although the computational limitation (complex-
ity) of finding the tolerance and the G-tolerance of a
given strict separating system (both are NP-hard), it
will be of interest to develop an efficient algorithm
able to calculate the tolerance and the greatest toler-
ance for strict separating systems with a reasonable
high number of variables.
REFERENCES
Beiu, V., Quintana, J. M., and Avedillo, M. J. (2003).
Vlsi implementation of threshold logic-a comprehen-
sive survey. IEEE Transactions on Neural Networks,
14(5):1217–1243.
Elgot, C. C. (1961). Truth functions realizable by single
threshold organs. In AIEE Conference Paper 60-1311
(October), revised November 1960; paper presented
at IEEE Symposium on Switching Circuit Theory and
Logical Design.
Elizondo, D. (2004). Searching for linearly separable sub-
sets using the class of linear separability method. In
Proceedings of the 2004 International Joint Confer-
ence on Neural Networks (IJCNN 04), volume 2 of
IEEE, pages 955–959.
Elizondo, D. (2006). The linear separability problem: some
testing methods. IEEE Transactions on Neural Net-
works, 17(2):330–344.
Freixas, J. and Molinero, X. (2008a). The greatest allowed
relative error in weights and threshold of strict separat-
ing systems. IEEE Transactions on Neural Networks,
19(5).
Freixas, J. and Molinero, X. (2008b). On the existence of
a minimum integer representation for weighted voting
systems. Annals of Operations Research, 166(1):243–
260.
Hu, S. T. (1960). Linearly separable switching functions.
Technical report, Technical report, LMSC, Technical
Document, LMSD-703024.
Hu, S. T. (1965). Threshold Logic. Univ. of California
Press. xiv + 338 pp. Let xx, x2.
Myhill, J. and Kautz, W. H. (1961). On the size of weights
required for linear–input switching functions. IRE-
Transactions, EC-10.
Ozdemir, H., Kepkep, A., Pamir, B., Leblebici, Y., and
Cilingiroglu, U. (1996). A capacitive threshold logic
gate. IEEE J. Solid–State Circuits, 31.
Picton, P. D. (1991). Neural Networks. Palgrave, second
edition.
Roychowdhury, V., Siu, K., and (Eds.), A. O. (1994). The-
oretical Advances in Neural Computation and Learn-
ing. Kluwer Academic Publishers, Stanford, USA.
Sang-Hoon, O. and Lee, Y. (1995). Sensitivity analysis
of single hidden-layer neural networks with threshold
functions. IEEE Trans. Computers, 6(4):1005–1007.
Siu, K., Roychowdhury, V., and Kailath, T. (1995). Dis-
crete Neural Computation: A Theoretical Foundation.
Prentice Hall, New Jersey, USA.
Winder, R. O. (1962). Threshold logic. PhD thesis, Depart-
ment of Mathematics, Princeton University.
Yao, S. S. and Ostapko, D. L. (1968). Realization of a
class switching functions by threshold-logic networks.
IEEE Transactions on Computers, C-17(4):391–399.
ICAART 2010 - 2nd International Conference on Agents and Artificial Intelligence
514