DropBlock: A New Regularization Technique

Regularization is a strategy implemented in a deep neural network that will reduce the generalization error but not the training error to perform well on not just the training data but also on new unseen inputs. An effective regularizer reduces the variance significantly while not overly increasing the bias, thus preventing overfitting. We use regularization techniques like L1 and L2 to reduce overfitting, penalizing the loss function, or regularization techniques like Dropouts and Spatial Dropouts, which discourage model complexity. The principle behind regularization methods in a neural network is to inject noise into neural networks to avoid overfitting the training data. L2 regularization is commonly known as weight decay or ridge regression, or Tikhonov regularization.