Model-Free Surrogate-Assisted Neural Architecture Search for Evolving Variable-Length Dense Blocks

Abstract

Neural Architecture Search (NAS) has emerged as a powerful paradigm for automatically designing deep neural networks; however, its practical adoption is often limited by substantial computational cost. To alleviate expensive full-training evaluations, surrogate-based methods have been introduced to estimate network performance efficiently. Nevertheless, existing approaches-particularly model-based surrogates-require training many candidate architectures and involve additional optimization overhead. In this work, we propose a Model-Free Surrogate PSO Network (MFSPNet) for evolving convolutional neural network architectures. The proposed method integrates a lightweight model-free surrogate predictor within a particle swarm optimization (PSO) framework, eliminating the need for pre-trained surrogate models. Specifically, MFSPNet introduces two key contributions: (1) a validation-loss-driven exponential moving average estimator (VLE-EMA) that captures early generalization behavior for reliable architecture ranking; and (2) a block-based dense connection strategy that enables effective stacking of evolved blocks while mitigating vanishing-gradient issues. This design also facilitates transferability of learned blocks across datasets. Extensive experiments demonstrate that MFSPNet achieves competitive performance with reduced computational cost. Under a consistent training protocol with ten independent runs, the proposed method attains error rates of 3.91%, 17.68%, and 1.91% on CIFAR-10, CIFAR-100, and SVHN, respectively, along with top-1/top-5 error rates of 28.29%/12.82% on ImageNet, while requiring less than three GPU days for architecture search. Due to computational constraints, the ImageNet result is based on a single run and should be interpreted as indicative of scalability. Overall, MFSPNet provides an efficient and reliable framework for cost-aware neural architecture search.