A.1 NASSearchSpaces NASBench-1011 introduces a large and expressive search space with 423k unique convolutional neural architectures and training statistics on CIFAR-10. NASBench-2012 contains the training statistics of15,625 architectures across three different datasets, including CIFAR-10, CIFAR-100, and Tiny-ImageNet-16. NASBench-NLP5 [10] is an NLP neural architecture search space, including 14k recurrent cells trained on the Penn Treebank (PTB)[11]dataset. Thegenerated noise maps aredirectly multiplied bythelevelwhich canbeselected from 0 to 1 with a step of 0.1. Other settings are already described in Section 3.1.2.

Most existing neural architecture search (NAS) algorithms are dedicated to and evaluated bythedownstream tasks, e.g.,image classification incomputer vision.

Most existing neural architecture search (NAS) algorithms are dedicated to and evaluated by the downstream tasks, e.g., image classification in computer vision. However, extensive experiments have shown that, prominent neural architectures, such as ResNet in computer vision and LSTM in natural language processing, are generally good at extracting patterns from the input data and perform well on different downstream tasks. In this paper, we attempt to answer two fundamental questions related to NAS. (1) Is it necessary to use the performance of specific downstream tasks to evaluate and search for good neural architectures?

The ResNeXt-A and ResNeXt-B have different channel-number and group-convolution settings. Initialize an empty population queue, Q_pop // The maximum population is P Initialize an empty set, history // Will contain all visited individuals for i = 1, 2,, P do new _individual RandomInit() new _individual.fitness Eval(new _individual) Enqueue(Q _pop, new _individual) // Add individual to the right of Q _pop Add new _ individual to history end // Evolve for T _iter for i = 1, 2,, T_iter do Initialize an empty set, sample _ set for i = 1, 2,, S do Add an individual to sample _ set from Q_ pop without replacement. A.2.2 Proxy T ask Search Figure 1: The configuration of a task in JSON style and the illustration of task mutation. The configuration of a task is shown in Figure 1.

Extensive experiments across 13 CNN search spaces and one NLP space demonstrate the remarkable efficiency of GenNAS using regression, in terms of both evaluating the neural architectures (quantified by the ranking correlation Spearman's

Most existing neural architecture search (NAS) algorithms are dedicated to and evaluated by the downstream tasks, e.g., image classification in computer vision. However, extensive experiments have shown that, prominent neural architectures, such as ResNet in computer vision and LSTM in natural language processing, are generally good at extracting patterns from the input data and perform well on different downstream tasks. In this paper, we attempt to answer two fundamental questions related to NAS. (1) Is it necessary to use the performance of specific downstream tasks to evaluate and search for good neural architectures? To answer these questions, we propose a novel and generic NAS framework, termed Generic NAS (GenNAS). GenNAS does not use task-specific labels but instead adopts regression on a set of manually designed synthetic signal bases for architecture evaluation.

Most existing neural architecture search (NAS) algorithms are dedicated to the downstream tasks, e.g., image classification in computer vision. However, extensive experiments have shown that, prominent neural architectures, such as ResNet in computer vision and LSTM in natural language processing, are generally good at extracting patterns from the input data and perform well on different downstream tasks. These observations inspire us to ask: Is it necessary to use the performance of specific downstream tasks to evaluate and search for good neural architectures? Can we perform NAS effectively and efficiently while being agnostic to the downstream task? In this work, we attempt to affirmatively answer the above two questions and improve the state-of-the-art NAS solution by proposing a novel and generic NAS framework, termed Generic NAS (GenNAS). GenNAS does not use task-specific labels but instead adopts \textit{regression} on a set of manually designed synthetic signal bases for architecture evaluation. Such a self-supervised regression task can effectively evaluate the intrinsic power of an architecture to capture and transform the input signal patterns, and allow more sufficient usage of training samples. We then propose an automatic task search to optimize the combination of synthetic signals using limited downstream-task-specific labels, further improving the performance of GenNAS. We also thoroughly evaluate GenNAS's generality and end-to-end NAS performance on all search spaces, which outperforms almost all existing works with significant speedup.