N-Student Learning

Overfitting is a fundamental problem in the field of machine learning, and is especially important in the context of training with noisy data. As we scale our datasets, the amount of noise naturally increases due to the infeasibility of careful human labeling. In the following article, we will be introducing the main ideas behind N-Student Learning, a multi-network architecture that can be applied to any network in order to reduce the impact of overfitting. The setup also allows for precise control over how a network learns to model any noise or uncertainty in the data. All images unless otherwise noted are by the author.