Design by Evolution: How to evolve your neural network with AutoML

For most machine learning practitioners designing a neural network is an artform. Usually, it begins with a common architecture and then parameters are tweaked until a good combination of layers, activation functions, regularisers, and optimisation parameters are found. Guided by popular architectures -- like VGG, Inception, ResNets, DenseNets and others -- one will iterate through variations of the network until it achieves the desired balance of speed and accuracy. But as the available processing power increases, it makes sense to begin automating this network optimisation process. In shallow models like Random Forests and SVMs we are already able to automate the process of tweaking through hyper-parameter optimisation.