Fitness Landscape Analysis of a Cell-Based Neural Architecture Search Space

Devon Tao, Lucas Bang

2024

Abstract

Neural Architecture Search (NAS) research has historically faced issues of reproducibility and comparability of algorithms. To address these problems, researchers have created NAS benchmarks for NAS algorithm evaluation. However, NAS search spaces themselves are not yet well understood. To contribute to an understanding of NAS search spaces, we use the framework of fitness landscape analysis to analyze the topology search space of NATS-Bench, a popular cell-based NAS benchmark. We examine features of density of states, local optima, fitness distance correlation (FDC), fitness distance rank correlations, basins of attraction, neutral networks, and autocorrelation in order to characterize the difficulty and describe the shape of the NATS-Bench topology search space on CIFAR-10, CIFAR-100, and ImageNet16-120 image classification problems. Our analyses show that the difficulties associated with each fitness landscape could correspond to the difficulties of the image classification problems themselves. Furthermore, we demonstrate the importance of using multiple metrics for a nuanced understanding of an NAS fitness landscape.

Paper Citation

in Harvard Style

Tao D. and Bang L. (2024). Fitness Landscape Analysis of a Cell-Based Neural Architecture Search Space. In Proceedings of the 1st International Conference on Explainable AI for Neural and Symbolic Methods - Volume 1: EXPLAINS; ISBN 978-989-758-720-7, SciTePress, pages 77-86. DOI: 10.5220/0012892400003886

in Bibtex Style

@conference{explains24,
author={Devon Tao and Lucas Bang},
title={Fitness Landscape Analysis of a Cell-Based Neural Architecture Search Space},
booktitle={Proceedings of the 1st International Conference on Explainable AI for Neural and Symbolic Methods - Volume 1: EXPLAINS},
year={2024},
pages={77-86},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012892400003886},
isbn={978-989-758-720-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Explainable AI for Neural and Symbolic Methods - Volume 1: EXPLAINS
TI - Fitness Landscape Analysis of a Cell-Based Neural Architecture Search Space
SN - 978-989-758-720-7
AU - Tao D.
AU - Bang L.
PY - 2024
SP - 77
EP - 86
DO - 10.5220/0012892400003886
PB - SciTePress