Neural Architecture Search: Tradeoff Between Performance and Efficiency

Tien Dung Nguyen, Nassim Mokhtari, Alexis Nédélec

2025

Abstract

Many Neural Architecture Search (NAS) methods have designed models that outperform manually configured networks on various tasks. Due to computational cost of model’s training, recent trend includes performing NAS without training candidate networks in the process. Many such methods have proven that training-free metrics are an effective way to assess model’s performance, especially if they are combined together. Multi-training-free-objectives NAS methods usually construct a Pareto front that gives a wide range of solutions. However, only one solution is chosen in the end. We introduce the Rank-based Improved Firefly Algorithm (RB-IFA), which focuses the search in a single direction by converting multiple objective ranks into one weighted sum. Weights are derived from a performance-efficiency tradeoff. Our search algorithm is based on an Improved Firefly Algorithm (IFA). IFA effectively explores the NAS landscape by combining the Firefly Algorithm, which has fast convergence, with a genetic algorithm, which improves the ability to overcome local optima. RB-IFA NAS identifies highly efficient architectures with competitive performance within 8 minutes. These results highlight the potential of multi-training-free metrics and a rank-based approach in finding efficient neural networks.

Paper Citation

in Harvard Style

Nguyen T., Mokhtari N. and Nédélec A. (2025). Neural Architecture Search: Tradeoff Between Performance and Efficiency. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 1154-1163. DOI: 10.5220/0013296900003890

in Bibtex Style

@conference{icaart25,
author={Tien Nguyen and Nassim Mokhtari and Alexis Nédélec},
title={Neural Architecture Search: Tradeoff Between Performance and Efficiency},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2025},
pages={1154-1163},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013296900003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - Neural Architecture Search: Tradeoff Between Performance and Efficiency
SN - 978-989-758-737-5
AU - Nguyen T.
AU - Mokhtari N.
AU - Nédélec A.
PY - 2025
SP - 1154
EP - 1163
DO - 10.5220/0013296900003890
PB - SciTePress