A Appendix A.1 NAS Search Spaces NASBench-101

The ResNeXt-A and ResNeXt-B have different channel-number and group-convolution settings. Initialize an empty population queue, Q_pop // The maximum population is P Initialize an empty set, history // Will contain all visited individuals for i = 1, 2,, P do new _individual RandomInit() new _individual.fitness Eval(new _individual) Enqueue(Q _pop, new _individual) // Add individual to the right of Q _pop Add new _ individual to history end // Evolve for T _iter for i = 1, 2,, T_iter do Initialize an empty set, sample _ set for i = 1, 2,, S do Add an individual to sample _ set from Q_ pop without replacement. A.2.2 Proxy T ask Search Figure 1: The configuration of a task in JSON style and the illustration of task mutation. The configuration of a task is shown in Figure 1.