r/deeplearning - Deep Double Descent

This video explores the recent study on Deep Double Descent! This is an interesting phenomenon of increasing and decreasing test error with respect to scaling up model size, training data, and epochs. The lens of bias-variance tradeoff may lead you to interpret increasing test error as overfitting, however the double descent phenomenon shows remarkable cases of a second descent when continuing to increase model size / epoch count. Awareness of this phenomenon may help you interpret your training curves for hyperparameter optimization and architectures search!