Artificial Intelligence and Deep Learning For the Extremely Confused

These complex and abstract representations can then be identified anywhere in the image. One drawback to CNN's is that increasing model power requires increased model depth. This increases the number of parameters in the model, lengthening training time and predisposing to the vanishing gradient problem, where gradients disappear and the model stalls in stochastic gradient descent, failing to converge. The introduction of Residual Networks in 2015 (ResNets) solved some of the problems with increasing network depth, as residual connections (seen above in a DenseNet) allow backpropagation to take a gradient from the last layer and follow it through all the way to the first layer. Recognition that CNN's are agnostic to position, but not orientation is important to note.