Spectral Architecture Search for Neural Networks

Neural networks are very effective machine learning tools that prove extremely valuable in unwinding the best representation of the data at hand. To improve the ability of neural networks to automatically perform the tasks assigned, innovative architectures have been proposed and thoroughly tested. Employed architectures have been customarily developed by human experts, with manual, time-consuming, and error-prone processes. To go beyond manual design, novel algorithmic strategies for automated discovery of optimal neural architectures have been developed. Consequently, architecture engineering has become a relevant field of active research [1,2]. Neural Architecture Search (NAS), the process that seeks to optimize network architecture, has been successfully applied on tasks as image classification [3,4], object detection [3], or semantic segmentation [5], yielding remarkable performance, as compared to manually designed benchmarks. According to [1], NAS is a subfield of Automated Machine Learning (AutoML) [6], the process that aims at automating the steps propaedeutic to applying machine learning to real-world problems. It also shows a notable overlap with hyperparameter optimization, a critical process in machine learning that involves selecting the optimal set of hyperparameters for a learning algorithm.