Supplementary Materials for "ZARTS: On Zero-order Optimization for Neural Architecture Search " 1 Appendix 1.1 Estimation for Second-order Partial Derivative in DARTS

DARTS utilizes difference method, which is also a zero-order optimization algorithm.1.2 To draw loss landscapes w.r.t. In (b), we illustrate the landscape with second-order approximation. We fix iteration number M = 10 for all settings. Therefore, we remove zero operation from the search space. We apply Alg. 1 to train architecture parameters Models are trained for 600 epochs by SGD with a batch size of 96.