Ensemble Methods for Deep Learning Neural Networks to Reduce Variance and Improve Performance

An equivalent approach might be to use a smaller subset of the training dataset without regularization to allow faster training and some overfitting. The desire for slightly under-optimized models applies to the selection of ensemble members more generally.