Neural Network Architectures – Towards Data Science – Medium

Reporting top-1 one-crop accuracy versus amount of operations required for a single forward pass in multiple popular neural network architectures. At the time GPU offered a much larger number of cores than CPUs, and allowed 10x faster training time, which in turn allowed to use larger datasets and also bigger images. Christian thought a lot about ways to reduce the computational burden of deep neural nets while obtaining state-of-art performance (on ImageNet, for example). Inspired by NiN, the bottleneck layer of Inception was reducing the number of features, and thus operations, at each layer, so the inference time could be kept low.