From Targets to Rewards: Continuous Target Sets in the Algorithmic

Search Framework

Milo Knell

1,∗ a

, Sahil Rane

1,∗ b

, Forrest Bicker

1,∗ c

, Tiger Che

1,∗ d

, Alan Wu

2,∗ e

and George D. Monta

nez

1 f

AMISTAD Lab, Department of Computer Science, Harvey Mudd College, Claremont, CA 91711, U.S.A.

Department of Computer Science, California Institute of Technology, Pasadena, CA 91125, U.S.A.

Keywords:

Algorithmic Search Framework, Satisfaction, Fuzzy Membership.

Abstract:

Many machine learning tasks have a measure of success that is naturally continuous, such as error under a

loss function. We generalize the Algorithmic Search Framework (ASF), used for modeling machine learning

domains as discrete search problems, to the continuous space. Moving from discrete target sets to a continuous

measure of success extends the applicability of the ASF by allowing us to model fundamentally continuous

notions like fuzzy membership. We generalize many results from the discrete ASF to the continuous space and

prove novel results for a continuous measure of success. Additionally, we derive an upper bound for the expected

performance of a search algorithm under arbitrary levels of quantization in the success measure, demonstrating

a negative relationship between quantization and the performance upper bound. These results improve the

ﬁdelity of the ASF as a framework for modeling a range of machine learning and artiﬁcial intelligence tasks.

1 INTRODUCTION

The Algorithmic Search Framework (ASF) is a theo-

retical model that has been used to rigorously study

properties of machine learning (ML), artiﬁcial intel-

ligence (AI), and search problems (Monta

nez, 2017;

Monta

nez et al., 2019; Monta

nez et al., 2021). This

framework has been used to bound the performance

of learning models, prove trade-offs between bias and

expressivity (Lauw et al., 2020), derive generaliza-

tion bounds for supervised classiﬁcation (Ramalingam

et al., 2022), and quantify performance bounds on

transfer learning (Williams et al., 2020). However, one

fundamental limitation of the ASF is that it measures

the performance of a machine learning algorithm with

a binary target set to which elements in the search

space (also referred to as hypotheses) either belong or

do not belong, rendering them indistinguishable from

one another. This limitation makes it impossible to

https://orcid.org/0009-0002-2951-8324

https://orcid.org/0009-0001-3986-1129

https://orcid.org/0009-0000-9872-7619

https://orcid.org/0009-0000-3586-4288

https://orcid.org/0009-0006-2454-4354

https://orcid.org/0000-0002-1333-4611

∗

Denotes equal authorship.

account for the fuzzy membership of hypotheses over

a search space where each hypothesis may have a vary-

ing degree of ﬁdelity. As a result, the strongest and

weakest satisfactory hypotheses are treated equally. By

using a continuous metric instead we can incorporate

meaningful information about the relative certainty

of our hypothesises, allowing us to both strengthen

existing results and prove novel theorems.

Many modern applications of machine learning

could beneﬁt from a continuous success measure,

which we term the

satisfaction

of a hypothesis. Hence,

we propose a degree of satisfaction as a continuous-

scale measure of the quality of a hypothesis function,

instead of having the notion of binary membership

in a target set. Examples of continuous measures of

success include cross-entropy loss, mean squared er-

ror, hinge loss, accuracy, and F

score. To accurately

model such problems in the ASF, we must account for

continuous membership measures. Prior work avoided

this limitation by implicitly deﬁning some threshold

of acceptability, where the target set was deﬁned as

the set of all elements with acceptable satisfaction val-

ues (Monta

nez, 2017). By deﬁning such a threshold

we lose information about the underlying satisfaction

structure between different hypotheses in the target set.

However, employing a framework that directly inter-

faces with the underlying satisfaction structure enables

558

Knell, M., Rane, S., Bicker, F., Che, T., Wu, A. and Montañez, G.

From Targets to Rewards: Continuous Target Sets in the Algorithmic Search Framework.

DOI: 10.5220/0012370600003636

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2024) - Volume 3, pages 558-567

ISBN: 978-989-758-680-4; ISSN: 2184-433X

us to generalize the results of the ASF, and measure

the performance of machine learning algorithms more

effectively, accounting for the fuzzy membership of

hypotheses functions.

We examine related work to machine learning as

search and fuzzy membership, rigorously and math-

ematically deﬁne the continuous ASF, present novel

results, and show a real-world example applying these

novel bounds.

2 RELATED WORK

Machine learning can be modeled as search (Mitchell,

1982; Monta

nez, 2017). The conversion of machine

learning problems to search problems enables us to

perform a variety of analyses on their performance,

using a mathematical and information-theoretic per-

spective. This approach helps us prove bounds on

the performance of machine learning algorithms and

gain an improved understanding of ‘big picture’ con-

cepts in machine learning such as the bias-expressivity

trade-off (Lauw et al., 2020).

Since its introduction, many results have been

proven within the context of the ASF. For instance,

researchers have demonstrated that a well-performing

machine learning algorithm cannot exist without a

predisposition to a certain group of outcomes (bias)

(Monta

nez et al., 2019). Deﬁning expressivity as the

variability of outputs a machine learning algorithm

can generate with different training data (Lauw et al.,

2020), the ASF has been employed to prove the ex-

istence of fundamental trade-offs between bias and

expressivity in machine learning (Lauw et al., 2020).

Therefore, the ASF has proven effective in establishing

fundamental properties of machine learning and ma-

chine learning algorithm performance. The framework

has been used to prove ‘famine’ results, such as the

fact that favorable algorithms for a speciﬁc task are

scarce (Monta

nez, 2017).

In the ASF, researchers make several simplifying

assumptions, including the use of binary target sets

to measure the satisfaction of hypotheses (Monta

nez,

2017). Our work extends the ASF and generalizes

the previously mentioned results. Moreover, we relax

some simplifying assumptions within the ASF and gen-

eralize existing results proven within the framework

to a continuous measure of satisfaction of hypotheses

(Monta

nez, 2017; Monta

nez et al., 2019; Lauw et al.,

2020). By extending the framework to account for con-

tinuous measures of satisfaction, we pave the way for

future progress within the ASF, offering a more gen-

eral framework applicable to a larger set of machine

learning problems.

This generalization is especially valuable in the

context of recent machine learning advances that in-

corporate fuzzy membership functions in a variety of

capacities. This includes within models to enhance

their accuracy, and in performing tasks such as im-

age classiﬁcation and various engineering applications

ullermeier, 2005; Gottwald, 2005; Resti et al., 2022;

Ghofrani et al., 2014). Moreover, expanding the frame-

work to encompass continuous target sets broadens its

applicability, making it relevant to a broader range of

machine learning challenges, thereby increasing its

practicality.

3 THE ALGORITHMIC SEARCH

FRAMEWORK (ASF)

3.1 The Search Problem

The ASF recasts machine learning problems as search

problems, simplifying proofs for results on their per-

formance. Following Monta

nez (Monta

nez, 2017), we

model the search problem as a modular system of three

parts,

(Ω, T, F)

, where

Ω

represents the discrete, ﬁnite

search space comprising hypotheses. We search within

this space to ﬁnd an element in the non-empty subset

, known as the target set. The external information

resource

guides this search, providing initialization

information, and offering evaluations on sampled ele-

ments from the search space to further steer our search.

For instance, in a machine learning context, the exter-

nal information resource

could be a training dataset

with an accompanying loss function. Therefore, evalu-

ating the external information resource on a particular

element of the search space yields the loss function

value for a speciﬁc hypothesis.

The target set

corresponds to the set of hypothe-

ses that attain sufﬁciently high levels of satisfaction

on a dataset for some desired threshold value of sat-

isfaction. In the context of machine learning, we can

interpret the satisfaction level of hypothesis as a notion

of accuracy or performance on a test dataset. The loss

function included in

directs the algorithm in search-

ing through

Ω

for a hypothesis in

. This implies that

we use our training data in our external information

resource F to ﬁnd a hypothesis in the target set T .

3.2 The Search Algorithm

The search algorithm A iteratively assigns a probabil-

ity distribution over the search space, drawing from

its search history and the evaluation of the external

information resource on each element, as shown in

From Targets to Rewards: Continuous Target Sets in the Algorithmic Search Framework

559

Figure 1. The search history comprises a query trace

and a resource evaluation trace. The query trace holds

the history of the elements that have been sampled by

the search algorithm, and the resource evaluation trace

records the history of the evaluations of the external

information resource on these elements. A search algo-

rithm within this framework is considered successful

if it samples an element of the target set during its

search. Importantly, only the external information re-

source, not the target set, guides the algorithm during

the search process.

Figure 1: A black-box search algorithm. Reproduced from

(Monta

nez et al., 2019).

4 CONTINUOUS ASF

4.1 Deﬁnitions

The satisfaction measure serves as the continuous case

analog of the target set. It provides an indication of the

quality of a hypothesis in the search space, signifying

how good or bad a particular hypothesis is.

Deﬁnition 4.1

(Satisfaction)

The satisfaction func-

tion

s(ω) : Ω → [0, c]

maps from the search space

Ω

to real-valued quantities denoted satisfactions. We as-

sume that these satisfactions exist in the range

[0, c]

where

is a ﬁnite, positive real number. It is possible

to assume that total satisfaction sums to 1 over the

search space, which is achievable without loss of gen-

erality since we can linearly transform any satisfaction

space to satisfy this property

Deﬁnition 4.2

(Continuous Search Problem)

Let the

tuple

(Ω, s, F)

deﬁne a search problem. The search

space

Ω

contains the elements (hypotheses) to be

queried/explored. For each

ω ∈ Ω

s(ω)

denotes the

level of satisfaction corresponding to the hypothesis

. The function

s(ω)

can be represented by a vec-

tor

s ∈ R

|Ω|

where

= s(ω)

. This deviation from

binary membership target sets allows us to account for

a continuous measure of satisfaction for hypotheses.

One such transformation is s

(ω) =

s(ω)−min

Ω

s(ω)

∑

Ω

s(ω)−min

Ω

s(ω)

denotes the external information resource available

to the learning algorithm, and for each element

ω ∈ Ω

let

F(ω)

be the evaluation of the external information

resource corresponding to the element of the search

space

. Thus, the only departure from the classic ASF

lies in replacing binary

with continuous satisfaction

measure s.

Deﬁnition 4.3

(Expected Per-Query Satisfaction)

Monta

nez’s ASF (Monta

nez, 2017), success is mea-

sured using an expected per-query probability of suc-

cess metric. In the continuous case, this generalizes

to an expected per-query satisfaction metric. We do

so by weighting the probability that each element is

sampled by a search algorithm with its corresponding

satisfaction level. Let

be the history of the search

algorithm,

the external information resource,

sequence of probability distributions over the search

space assigned by the search algorithm, and

be the

probability distribution assigned by the search algo-

rithm over the search space at a time step

in the search

history

. Formally, we deﬁne the expected per-query

satisfaction as

q(s, F) = E

P,H

∑

i=1



Deﬁnition 4.4

(Decomposability)

We observe that

each

q(s, F)

can be decomposed into the inner product

of s and P

q(s, F) = E

P,H

∑

i=1



= s

P,H

∑

i=1



= s

(1)

where we have deﬁned

:= E

P,H

∑

i=1

| F

the expected average conditional distribution on the

search space given

. Intuitively, this is equivalent to

weighting each satisfaction value with its correspond-

ing probability mass in P

Our notation is summarized in Table 1. We include

a real-world example in Section 6, anchoring the ASF

to a practical machine-learning problem. Note that

there are many learning processes, and some may dif-

fer from the examples below (for example, an unsuper-

vised learning problem must have a different measure

of success from a supervised classiﬁcation problem).

The ASF is general enough to encompass any algorith-

mic search problem.

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

560

Table 1: Notation.

SYMBOL DEFINITION

Ω, ω

Search space, element of search space,

e.g., an element could be a set of param-

eters, like weights.

s, s(ω)

Satisfaction vector, satisfaction of ele-

ment

, e.g., how good a particular hy-

pothesis is, such as performance on the

test set.

T, t

Target set in the binary ASF (the equiv-

alent of

in the continuous ASF), e.g.,

a set of hypothesis that are sufﬁciently

satisfactory, possibly performing above

some threshold on the test set. The bi-

nary vector representation of this set is

given by t.

F, F(ω)

External information resource, evalua-

tion of external information resource on

an element

, e.g., our training measure

of how good a hypothesis is, such as

training data and some loss function.

q(s, F) Expected per-query satisfaction.

q(t, F)

Expected per-query probability of suc-

cess in ASF.

Decomposable satisfaction metric, e.g,

A closed-under-permutation set of

such

that all s

1 = k.

Distribution over a set of satisfaction vec-

tors in τ

An abstract search algorithm which itera-

tively explores the search space.

Distribution assigned by

over

Ω

at step

during the iterative search process. This

could be conditioned on F.

Sequence of probability distributions as-

signed by algorithm

, each element is a

. This could be conditioned on F.

Expected averaged conditional distribu-

tion assigned by the search algorithm

given the external information resource

5 RESULTS

5.1 Famine of Favorable Satisfactions

Theorem 5.1.

For ﬁxed

k ∈ R

≥0

, ﬁxed information

resource

, decomposable, non-negative satisfaction

metric

such as

, and minimum acceptable per-query

satisfaction q

min

, we deﬁne

= {s ∈ R

|Ω|

∑

i=1

= k}, and

min

= {s ∈ τ

| φ(s, F) ≥ q

min

Then

µ(τ

min

)

µ(τ

)

≤

min

where

is the expected per-query satisfaction under

uniform random sampling and

is Lebesgue measure.

This theorem shows that the proportion of satis-

faction functions for which our algorithm performs

extremely well (with more than

min

expected satis-

faction) is small. In most practical applications,

extremely small, as

will typically be extremely small

in comparison to the size of the search space. The

upper bound for the probability of successful search

decreases linearly with the increase of threshold value

min

5.2 Success Under Dependence

Theorem 5.2.

Let

be a ﬁnite positive constant, and

restrict

to an arbitrary quantization

Q = {i · c |

i ∈ {1, ..., m}}

. Let

be the set of satisfaction vec-

tors such that

= {s | s ∈ Q

|Ω|

, s

1 = k}

, and let

H(U

;|Ω|

)

denote the information-theoretic entropy

of the uniform distribution over

for a search space

of cardinality |Ω|.

Let the satisfaction vector

S ∼ D

be a vector-

valued random variable over the set

. Let

be the

random variable such that

X ∼ P

over the elements

Ω

S(X)

is similar to

s(ω)

, except we are dealing

with random variables

and

rather than speciﬁc re-

alizations

and

. Then for any non-negative constant

Pr(S(X) ≥ u) ≤

I(S;F) + D

k U

) + H(S(X) | X)

H(U

;|Ω|−1

) − H(U

k−u

;|Ω|−1

)

Theorem 5.2 provides a bound on the probability

of sampling an element with a sufﬁciently large satis-

faction deﬁned by threshold

. This expression tells

us that the upper bound of the probability of success

monotonically improves as dependence between the

satisfaction vector values and information resource

values increases. The term

k U

)

represents

the Kullback-Leibler (KL) divergence between the ac-

tual distribution over the set

and the uniform

distribution over the same set

. This can be in-

terpreted as the predictability of the distribution of sat-

isfaction vectors, where large values of KL-divergence

From Targets to Rewards: Continuous Target Sets in the Algorithmic Search Framework

561

represent more probability mass concentrated on a few

elements. The

H(S(X ) | X)

term indicates the condi-

tional entropy or surprisal associated with the possible

satisfaction values for an element

sampled from the

search space. This term is large when there are large

variations in the satisfaction values thus resulting in

an increase in the upper bound for the probability of

sampling an element with a sufﬁciently large satisfac-

tion value. The denominator in the bound essentially

serves as a normalizing factor appropriately scaling

the value of the upper bound.

We see that our upper bound increases with an in-

crease in the predictability of the satisfaction vector,

the dependence between the satisfaction vector and

the external information resource, and the conditional

entropy in the satisfaction values associated with an el-

ement in the search space. Thus, this theorem gives us

an interpretable upper bound on the probability of sam-

pling an element with a sufﬁciently large satisfaction

value. Moreover, this theorem is particularly useful

to allow us to determine situations where we cannot

expect to perform well.

5.3 Expected Satisfaction Under

Dependence

Theorem 5.3.

We will continue using all the deﬁni-

tions from Theorem 5.2. Let

q = E[S(X )] =

∑

c∈Q

Pr(S(X) = c)· c.

Then,

q ≤

I(S; F) + D

k U

) + H(S(X) | X)

(H(U

;|Ω|−1

) − H(U

k−c

;|Ω|−1

))

where c

= inf Q and c

= sup Q.

Extending from the bound on

Pr(S(X) ≥ u)

pre-

sented in Theorem 5.2, we present a similar bound for

the expected satisfaction, i.e.,

q = E[S(X)]

, without the

need to specify a target satisfaction value deﬁned by a

constant threshold

. Compared to Theorem 5.2, this

bound gives more context of the search problem, and

can serve as a more robust metric since it’s not suscep-

tible to the skewness and kurtosis of the distribution

of satisfaction values over the search space.

The interpretation of this bound is similar to The-

orem 5.2 with a small change in the scaling factor

in the denominator. Here, the bounded quantity is

the expected satisfaction instead of the probability of

exceeding a certain satisfaction. Comparing the two

bounds in 5.2 and 5.3, we see that the bound in 5.3

is useful when sub-optimal elements contribute to the

success of the search algorithm, whereas the bound

in 5.2 is useful when sub-optimal elements do not

contribute to the success of the search algorithm.

5.4 Difference in Satisfaction

We next quantify and bound the difference between

expected per-query satisfaction (i.e., for continuous

targets) and expected per-query probability of suc-

cess (i.e., for binary targets), beginning with a helpful

lemma.

Lemma 5.4.

Let

be the threshold value for convert-

ing a continuous target set into a discrete (binary)

target set where all elements with satisfaction greater

than or equal to the threshold

are included in the

target set and the rest are excluded. Given a probabil-

ity vector

, satisfaction vector

, target vector

, and

vector v = s − t,

w| ≤ max(1 − g,g).

Theorem 5.5.

Let

F,s

be the average conditional dis-

tribution assigned by the search algorithm under a

continuous satisfaction measure, and let

F,t

be the av-

eraged conditional distribution assigned by the search

algorithm under a discrete target set. Let

be the

maximum rounding amount deﬁned as

max(1 − g, g)

Then,

|q(s, F) − q(t, F)| ≤ |T |



F,s

k P

F,t



+ r.

Theorem 5.5 bounds the difference in the success

measure in the discrete and continuous case using the

KL-divergence between the distributions learned in

the continuous and discrete cases. It indicates that the

potential for improved performance obtained by tran-

sitioning from a discrete target set to continuous target

sets relies on the chosen threshold value

and the size

of the target set. The degree of divergence between the

outputs of the search algorithm in the two scenarios is

measured by KL-divergence. This relationship is logi-

cal since the potential for performance improvement

between the case with discrete and continuous target

sets should be proportional to the amount of rounding

required, which is related to both the threshold and

the size of the target set. By transitioning from using

discrete to continuous target sets, we also have the

potential to gain from the divergence between the av-

erage conditional probability mass functions produced

by the search algorithm in both cases. This is because

the external information resource has the potential to

be more useful in the case of a continuous satisfaction

measure.

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

562

6 EXAMPLE

6.1 Setup

To anchor this theoretical framework we provide an

example of how it can be applied to a simple machine

learning regression problem. We ﬁrst create an inde-

pendent variable

then use some stochastic data gen-

erating process to obtain our dependent variable

. For

the purposes of this example we use

Y = 2X + 5 + ε

with ε ∼ N (0, 100).

Suppose we try to model the data from this learn-

ing problem with a linear regression of the form

Y = aX + b

. We would then be able to model the

training process within the ASF as a search over the

space of possible learned parameters. We consider a

ﬁnite set of values from which to take

and

. Let

and

be sets of evenly spaced numbers in the ﬁnite

interval

min

, a

max

]

and

min

, b

max

]

with a step size

That is,

A = {a

min

, a

min

+ x, ..., a

max

}

and likewise for

. Then,

Ω = A × B

or the Cartesian product of

and

. For this example, we selected

A = [0, 4]

with a step

size of

0.01

and

B = [1, 7]

, also with a step size of

0.01

In general, this is done by inspecting the distribution

of X and Y .

We must also determine an external information

resource

that will guide our search through the pa-

rameter space. For this regression problem, we use the

mean squared error (MSE) of our hypothesized model

calculated on the training set. That is, for a particular

hypothesis

corresponding to a pair of parameters

(a, b)

where

a ∈ A

and

b ∈ B

, the evaluation of the ex-

ternal information resource is the mean squared error

on the training set, or, F(ω) =

∑

− (aX

+ b))

The overarching goal of the search algorithm is

to ﬁnd elements in the search space that have large

satisfaction values associated with them. Our search

algorithm determines the distribution

to attempt

to maximize

, but it only has the information

provided by

. In this example, the satisfaction values

can be interpreted as the mean squared error on the

test set. While we perform our search we don’t use

these values of satisfaction to guide our search, only

the evaluation of

. In machine learning terms, the

algorithm does not have access to the test data during

the training step.

6.2 Example Result

Let us evaluate the bound presented in Theorem

5.2

both on differing levels of quantization and with dif-

ferent evaluation scenarios to gain more insight into

the theorem. If our training data was generated by the

same process as our testing data and success measured

similarly, we would expect our mean squared error on

the training data to be reﬂective of the mean squared

error of the test data, thus giving us a high satisfac-

tion for a trained model. However, if the test data was

generated via a different process or the measure of suc-

cess between training and testing data were different,

we would not expect our trained model to have high

satisfaction.

Consider a case where mean squared error is used

to evaluate a hypothesized model on the training data

in the information resource, but the satisfaction mea-

sure is the mean absolute error. In this case, the infor-

mation a learning algorithm is guided by during search

is systematically different from the information it is

supposed to learn. This would be reﬂected by a lower

value in the mutual information term I(S; F).

By making reasonable assumptions about the struc-

ture of our example problem, we can compute the

value of

I(S; F)

and the bound for Theorem

5.2

. First,

we set

k = 1

, and

|Ω| = |A| × |B| = 400 × 600

(the

bounds assigned in the previous section). We compute

the bounds for levels of quantization

m = 2

(binary)

and

m = 3

(ternary). For binary we set our

c = 0.5

∈ {0, 0.5}

. For ternary, we set

c =

∈ {0,

}

We assume that

= U

;|Ω|

, that is

S ∼ U

;|Ω|

While this assumption is not necessary it simpliﬁes

our calculations.

We produce these results in Table 2. The column

Match means that MSE is used for evaluating in both

the train and test phase (i.e

and

), while the column

Not Match means that MSE was used in the train phase

and MAE was used in the test phase. The rows

m = 2

and

m = 3

mean a binary level and ternary level of

quantization, respectively.

Table 2: Example Results.

Match Not Match

m=2

I(S;F) = 0.60

Pr(S(X) ≥

) ≤ 0.31

I(S;F) = 0.27

Pr(S(X) ≥

) ≤ 0.14

m=3

I(S;F) = 0.97

Pr(S(X) ≥

) ≤ 0.34

I(S;F) = 0.55

Pr(S(X) ≥

) ≤ 0.19

From the table, we can observe that having the

same evaluation metric for our train and test set raises

our potential for performing well. This change is

largely driven by the decreased mutual information

term

I(S; F)

displayed for each. Comparing across

levels of quantization does not necessarily make sense,

especially since the selection of

(i.e.,

and

) differ

in the two cases based on the level of quantization.

This demonstrates how these bounds can be ap-

plied to real-world problems, and show how changes

I(S; F)

inﬂuence our ability to perform well. Our

From Targets to Rewards: Continuous Target Sets in the Algorithmic Search Framework

563

ability to do well on a problem is limited by the quality

of our information with respect to what we are trying

to learn. In other words: garbage in, garbage out.

7 CONCLUSION

We extend the Algorithmic Search Framework from

discrete target sets to a continuous measure of suc-

cess, addressing one the framework’s core limitations

and increasing its versatility. We generalize theorems

previously proven using the discrete ASF to the con-

tinuous and quantized cases, and derive novel results.

Speciﬁcally, we prove an upper bound on performance

under an arbitrary level of quantization, demonstrating

that increasing the granularity of our success metric

reduces our maximum theoretical performance. We

bound the absolute difference in performance between

the binary and continuous cases. We provide an ex-

ample how the ASF can be applied to a regression

problem and show how different processes for generat-

ing data or measuring success change key terms, like

I(S; F), thus varying our bound on performance.

These results improve the ability of the ASF to

model machine learning problems that naturally have

continuous measures of success, unlocking the poten-

tial to further the body of existing ASF research. There

remain many opportunities for extension. One possible

application of this framework is that it can be used for

an information theory-based analysis of auto-ML algo-

rithms by giving us a framework to better understand

the performance of this domain of machine learning

algorithms. Strengthening this theoretical framework

will give researchers the tools to analyze learning algo-

rithms with a naturally continuous measure of success.

REFERENCES

Ghofrani, F., Helfroush, M. S., Danyali, H., and Kazemi, K.

(2014). Improving the performance of machine learn-

ing algorithms using fuzzy-based features for medical

x-ray image classiﬁcation. In Journal of Intelligent &

Fuzzy Systems, volume 6, pages 3169–3180.

Gottwald, S. (2005). Mathematical aspects of fuzzy sets

and fuzzy logic: Some reﬂections after 40 years. In

Mathematical aspects of fuzzy sets and fuzzy logic:

Some reﬂections after 40 years, volume 156, pages

357–364. 40th Anniversary of Fuzzy Sets.

ullermeier, E. (2005). Fuzzy methods in machine learn-

ing and data mining: Status and prospects. In Fuzzy

methods in machine learning and data mining: Sta-

tus and prospects, volume 156, pages 387–406. 40th

Anniversary of Fuzzy Sets.

Lauw, J., Macias, D., Trikha, A., Vendemiatti, J., and

Monta

nez, G. D. (2020). The Bias-Expressivity Trade-

off. In Rocha, A. P., Steels, L., and van den Herik, H. J.,

editors, Proceedings of the 12th International Confer-

ence on Agents and Artiﬁcial Intelligence, ICAART

2020, Volume 2, Valletta, Malta, February 22-24, 2020,

pages 141–150. SCITEPRESS.

Mitchell, T. M. (1982). Generalization as search. Artiﬁcial

Intelligence, 18(2):203–226.

Monta

nez, G. D. (2017). The Famine of Forte: Few Search

Problems Greatly Favor Your Algorithm. In 2017

IEEE International Conference on Systems, Man, and

Cybernetics (SMC), pages 477–482. IEEE.

Monta

nez, G. D., Bashir, D., and Lauw, J. (2021). Trad-

ing Bias for Expressivity in Artiﬁcial Learning. In

Rocha, A. P., Steels, L., and van den Herik, J., edi-

tors, Agents and Artiﬁcial Intelligence, pages 332–353,

Cham. Springer International Publishing.

Monta

nez, G. D., Hayase, J., Lauw, J., Macias, D., Trikha,

A., and Vendemiatti, J. (2019). The Futility of Bias-

Free Learning and Search. In 32nd Australasian Joint

Conference on Artiﬁcial Intelligence, pages 277–288.

Springer.

Ramalingam, R., Dice, N. E., Kaye, M. L., and Monta

nez,

G. D. (2022). Bounding Generalization Error Through

Bias and Capacity. In 2022 International Joint Confer-

ence on Neural Networks (IJCNN), pages 1–8.

Resti, Y., Irsan, C., Amini, M., Yani, I., Passarella, R., and

Zayantii, D. A. (2022). Performance improvement

of decision tree model using fuzzy membership func-

tion for classiﬁcation of corn plant diseases and pests.

In Performance Improvement of Decision Tree Model

using Fuzzy Membership Function for Classiﬁcation

of Corn Plant Diseases and Pests, volume 7, page

284–290.

Williams, J., Tadesse, A., Sam, T., Sun, H., and Monta

nez,

G. D. (2020). Limits of Transfer Learning. The Sixth

International Conference on Machine Learning, Opti-

mization, and Data Science (LOD 2020).

APPENDIX

Theorem 5.1.

For ﬁxed

k ∈ R

≥0

, ﬁxed information

resource

, decomposable, non-negative satisfaction

metric

, and minimum acceptable per-query satisfac-

tion q

min

, we deﬁne

= {s ∈ R

|Ω|

∑

i=1

= k}, and

min

= {s ∈ τ

| φ(s, F) ≥ q

min

Then

µ(τ

min

)

µ(τ

)

≤

min

where

is the per-query expected

satisfaction under uniform random sampling and

Lebesgue measure.

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

564

Proof. Under uniform sampling on τ

, we have

µ(τ

min

)

µ(τ

)

= Pr(φ(s, F) ≥ q

min

)

≤

U[τ

]

[φ(s, F)]

min

where

U[τ

]

is the uniform distribution over

s ∼

U[τ

]

, and the second step follows from Markov’s

inequality. By the decomposability of

and linearity

of expectation, we have:

µ(τ

min

)

µ(τ

)

≤

U[τ

]



φ,F



min

U[τ

]





φ,F

min

U[τ

]

is uniform,

U[τ

]





= p · 1

. It follows

that

µ(τ

min

)

µ(τ

)

≤

p · 1

φ,F

min

Furthermore, as

φ,F

is a probability distribution,

∑

φ,F

(ω) = 1

and

φ,F

. Hence, we conclude

µ(τ

min

)

µ(τ

)

≤

min

Theorem 5.2.

Let

be the random variable such that

X ∼ P

. For any non-negative u ∈ Q:

Pr(S(X) ≥ u) ≤

I(S;F) + D

k U

) + H(S(X) | X)

H(U

;|Ω|−1

) − H(U

k−u

;|Ω|−1

)

Proof. Q

is a set of integer multiples of a constant

spacing

(an arbitrary quantization) and can be ex-

pressed as

Q = {0, c

, c

, ..., c

}

, where

corre-

sponds to the minimum positive value and

corre-

sponds to the maximum value. Now, observe that

H(U

;|Ω|

) = log(|τ

|) = log



|Ω| +

− 1



Note that

H(U

;|Ω|

)

is monotonically increasing

|Ω|

and

. For notational simplicity, let

Pr(S(X) ≥ u). We see that:

H(S | S(X), X ) = (1 − P

)H(S | S(X) < u, X )

+ P

H(S | S(X) ≥ u, X )

≤ (1 −P

)H(U

;|Ω|−1

)

+ P

H(U

k−u

;|Ω|−1

)

= H(U

;|Ω|−1

)

− P

(H(U

;|Ω|−1

) − H(U

k−u

;|Ω|−1

)).

The inequality follows from the fact that the entropy

of a distribution of

is not larger than the entropy of

uniform distribution of s.

Also, by the chain rule of conditional entropy,

H(S, S(X) | X) = H(S | S(X), X) + H(S(X) | X)

H(S, S(X) | X ) = H(S(X ) | S, X) + H(S | X ) = H(S | X) + 0

By the data processing inequality (with the Markov

chain S → F → X),

H(S | F) ≤ H(S | X)

= H(S, S(X)|X )

= H(S | S(X), X ) + H(S(X) | X)

Then, we have:

H(S | F) = H(S | S(X ), X) + H(S(X) | X)

≤ H(U

;|Ω|−1

)

| {z }

≤H(U

)

− P

(H(U

;|Ω|−1

) − H(U

k−u

;|Ω|−1

))

+ H(S(X) | X)

and by the deﬁnition of conditional entropy,

H(S) − I(S; F) ≤ H(U

) − P

(H(U

;|Ω|−1

)

− H(U

k−u

;|Ω|−1

))

+ H(S(X) | X).

Thus,

Pr(S(X) ≥ u) ≤

I(S;F) + D

k U

) + H(S(X) | X)

H(U

;|Ω|−1

) − H(U

k−u

;|Ω|−1

)

Theorem 5.3.

Let

be a set of integer multiples of

a constant spacing

and can be expressed as

Q =

{0, c

, c

, ..., c

}

, where

corresponds to the mini-

mum positive value and

corresponds to the maxi-

mum value. Let

= Pr(S(X) = c)

q =

∑

c∈Q,c≥0

then we have:

q ≤

I(S; F) + D

k U

) + H(S(X) | X)

(H(U

;|Ω|−1

) − H(U

k−c

;|Ω|−1

)).

From Targets to Rewards: Continuous Target Sets in the Algorithmic Search Framework

565

Proof.

H(S | S(X), X ) = (1 −

∑

)H(S | S(X) = 0,X)

∑

H(S | S(X) = c,X)

≤ (1 −

∑

)H(U

;|Ω|−1

)

∑

· c ·

H(U

k−c

;|Ω|−1

)

= H(U

;|Ω|−1

)

−

∑

c ·

(H(U

;|Ω|−1

)

− H(U

k−c

;|Ω|−1

))

≤ H(U

;|Ω|−1

)

− (

∑

c) ·

(H(U

;|Ω|−1

)

− H(U

k−c

;|Ω|−1

)).

The last inequality is due to the monotonicity of

H(U

;|Ω|

)

. Then, following the same steps as used in

Theorem 5.2,

H(S) − I(S; F) ≤ H(U

;|Ω|−1

)

| {z }

≤H(U

)

− (

∑

|{z}

)

(H(U

;|Ω|−1

)

− H(U

k−c

;|Ω|−1

)).

After simpliﬁcation,

q ≤

I(S; F) + D

k U

) + H(S(X) | X)

(H(U

;|Ω|−1

) − H(U

k−c

;|Ω|−1

))

Lemma 5.4.

Given a probability vector

and a vector

v = s − t

w| ≤ max(1 − g,g)

Proof.

First, note that

g − 1 ≤ v

< g

. This is because

is the value that must be subtracted from the

el-

ement of the search space to attain the value

, since

t = s − v

and

is the cutoff threshold for either round-

ing

up to 1 or down to 0. Therefore, in the case that

we are rounding up, the most extreme value that can be

subtracted is

g −1

(which is equivalent to adding

1 −g

to arrive at 1). In the case that we are rounding down,

the largest value we could subtract is strictly less than

. Now, we see that

w| ≤ kvkkwk

, and since

is a

probability vector,

kwk ≤ 1

. Therefore,

w| ≤ kvk

and

kvk ≤ max(g, 1 − g)

, given that

g − 1

is negative.

Thus, |v

w| ≤ max(g, 1 − g).

Theorem 5.5.

Given that

F,s

is the averaged condi-

tional distribution assigned by the search algorithm

where our target set has a continuous satisfaction mea-

sure,

F,t

refers to the averaged conditional distribu-

tion assigned by the search algorithm when we use

a discrete target set, and

is the threshold value for

converting a continuous target set into a discrete target

set (all elements with satisfaction greater than or equal

to the threshold

are included in the target set and the

rest are excluded):

|q(s, F) − q(t, F)|

≤ |T |

F,s

k P

F,t

) + max(1 − g, g).

This theorem bounds the difference in the success mea-

sure in the discrete and continuous case using the KL-

divergence between the distributions learned in the

continuous and discrete cases.

Proof.

Consider

|q(s, F) − q(t, F)|

. Using the decom-

posable probability of success metrics we get:

|q(s, F) − q(t, F)| = |s

F,s

− t

F,t

Now, we deﬁne a vector

such that

v = s − t

. There-

fore,

F,s

− t

F,t

| = |(t + v)

F,s

− t

F,t

= |t

F,s

− t

F,t

+ v

F,s

≤ |t

F,s

− t

F,t

| + |v

F,s

Using Lemma 5.4,

F,s

| ≤ max(1 − g, g)

. There-

fore,

F,s

− t

F,t

| + |v

F,s

≤ |t

F,s

− P

F,t

)| + max(g, 1 − g).

Deﬁning r := max(g, 1 − g), we note that

F,t

− P

F,s

)| + max(g, 1 − g)

≤ |T |sup

F,s

(ω) − P

F,t

(ω)) +r

≤ |T |

F,s

k P

F,t

) + r,

where the last step follows from Pinsker’s inequality.

Hence,

|q(s, F) − q(t, F)| ≤ |T |

F,s

k P

F,t

) + r.

Example.

We set

k = 1

. For all levels of quanti-

zation, note that

I(S; F)

can be found computation-

ally by directly computing

s(ω)

and

F(ω)

for all

ω ∈ Ω

. We know that the

term will equal

ICAART 2024 - 16th International Conference on Agents and Artiﬁcial Intelligence

566

since the KL divergence between two identical dis-

tributions is

. Since

S(X)

is independent from

H(S(X )|X) = H(S(X)). We also know that

H(U

;|Ω|

) = −

∑

s∈τ

|τ

log



|τ



So then we simply need to ﬁnd

|τ

for a given

|Ω|

This can be done by simple combinatorics.

First, we set

m = 2

and

u = 0.5

with

c = 0.5

∈ {0, 0.5}

. We know that

P(S(X) = 0) =

n−2

since

there are exactly two non-zero elements (both

0.5

sum up to

k = 1

), and then

P(S(X) = 0.5) =

so the

distribution is known. To ﬁnd

|τ

, we know that it

is formed from all sets

that sum up to

, so that is

any set with exactly two elements with value

0.5

, so

then there are





such sets. We also need to compute

|τ

k−u

for the subtracted term in the bound, this is

0.5

We know this happens when exactly one element is

0.5, so there are





= n such sets.

Second, we set

m = 3

and

u =

with

c =

∈ {0,

}

. There are two possible cases for

satisfactory vectors in

: ones that have

elements

with

and those with one

and one

. There are





sets satisfying the ﬁrst case, and





· 2!

sets satisfying

the second so

|τ

| =









· 2!

. We must also ﬁnd

|τ

k−u

| = |τ

, this happens when exactly one element

is a

so there are





= n

such elements. Next, we

must ﬁnd the distribution over

S(X)

. Let

be the

probability that we have the case of three

(there are





such sets) and

be the complementary probability

of having one

and one

(there are





· 2!

such

sets). Then,

P(S(X) = 0) = a ·

n−3

+ b ·

n−2

since

there are 3 non-zero elements in a and 2 in b.

P(S(X) =

) = a ·

+ b ·

since there are 3

in the a

case and 1 in the b case. Finally,

P(S(X) =

) = b ·

since there is only a

in the b case and only 1.

Now that every term has been determined either

mathematically or computationally, we can combine

them to compute the bounds as given in Section 6.

From Targets to Rewards: Continuous Target Sets in the Algorithmic Search Framework

567