FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking

--Strong priors are imposed on the search space of Differentiable Architecture Search (DARTS), such that cells of the same type share the same topological structure and each intermediate node retains two operators from distinct nodes. While these priors reduce optimization difficulties and improve the applicability of searched architectures, they hinder the subsequent development of automated machine learning (Auto-ML) and prevent the optimization algorithm from exploring more powerful neural networks through improved architectural flexibility. This paper aims to reduce these prior constraints by eliminating restrictions on cell topology and modifying the dis-cretization mechanism for super-networks. Specifically, the Flexible DARTS (FX-DARTS) method, which leverages an Entropy-based Super-Network Shrinking (ESS) framework, is presented to address the challenges arising from the elimination of prior constraints. Notably, FX-DARTS enables the derivation of neural architectures without strict prior rules while maintaining the stability in the enlarged search space. Experimental results on image classification benchmarks demonstrate that FX-DARTS is capable of exploring a set of neural architectures with competitive trade-offs between performance and computational complexity within a single search procedure. Derong Liu is with the School of System Design and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen 518000, China (email: liudr@sustech.edu.cn), and also with the Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL 60607, USA (e-mail: derong@uic.edu). This manuscript is submitted to IEEE Transaction on Neural Network and Learning Systems and is under reviewed. Personal use of this manuscript is permitted. Over the past decade, the powerful representation ability of deep neural networks (DNNs) has contributed to significant progress in various machine learning tasks, including computer vision [1], [2], natural language processing [3], [4], system identification and control [5], [6], time series prediction [7], and autonomous vehicles [8]-[10], among others. Undoubtedly, the architecture design of neural networks plays a pivotal role in these breakthroughs [11]-[15]. To address the labor-intensive and time-consuming trial-and-error process of DNN architecture design, neural architecture search (NAS) [16] has emerged as a promising approach. NAS automates the exploration of a vast space of potential architectures, traditionally through three key steps: defining a search space, selecting a search algorithm, and identifying an optimal architecture within the search space. The effectiveness of NAS heavily relies on the careful design of both the search space and the search strategy, as a well-constructed search space can significantly enhance the search algorithm's ability to discover optimal neural architectures [17].