Neural Architecture Search (NAS): basic principles and different approaches

Neural Architecture Search (NAS) is the process of automating the design of neural networks' topology in order to achieve the best performance on a specific task. The goal is to design the architecture using limited resources and with minimal human intervention. At its core, NAS is a search algorithm. It operates on the search space of possible network topologies, which consists of a list of predefined operations (e.g.convolutional layers, recurrent, pooling, fully connected etc.) and their connections. A controller then chooses a list of possible candidate architectures from the search space. The candidate architectures are trained and ranked based on their performance on the validation test. The ranking is used to readjust the search and obtain new candidates.