Neural network: what is a neural network?

The amount of data needed to train deep neural networks can be truly immense, and in many commercial systems today it can take weeks or months of training to obtain adequate performance. This is even considering that networks are often trained in parallel across many highly optimized machines with specialized hardware such as ASICs and GPUs. This is a significant limitation of deep neural networks – one could imagine the frustration of a deep learning engineer who learns only after weeks of expensive training that there was an error or bug in the design of the neural network that inhibited performance.