The Bias-Expressivity Trade-off

Julius Lauw, Dominique Macias, Akshay Trikha, Julia Vendemiatti, George Montañez

2020

Abstract

Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can adapt to changing training data, altering its outcome based on changes in its input. We measure expressivity by using an information-theoretic notion of entropy on algorithm outcome distributions, demonstrating a trade-off between bias and expressivity. To the degree an algorithm is biased is the degree to which it can outperform uniform random sampling, but is also the degree to which is becomes inflexible. We derive bounds relating bias to expressivity, proving the necessary trade-offs inherent in trying to create strongly performing yet flexible algorithms.

Paper Citation

in Harvard Style

Lauw J., Macias D., Trikha A., Vendemiatti J. and Montañez G. (2020). The Bias-Expressivity Trade-off. In Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-395-7, pages 141-150. DOI: 10.5220/0008959201410150

in Bibtex Style

@conference{icaart20,
author={Julius Lauw and Dominique Macias and Akshay Trikha and Julia Vendemiatti and George Montañez},
title={The Bias-Expressivity Trade-off},
booktitle={Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2020},
pages={141-150},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008959201410150},
isbn={978-989-758-395-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - The Bias-Expressivity Trade-off
SN - 978-989-758-395-7
AU - Lauw J.
AU - Macias D.
AU - Trikha A.
AU - Vendemiatti J.
AU - Montañez G.
PY - 2020
SP - 141
EP - 150
DO - 10.5220/0008959201410150