Impact of Dataset Size on Deep Learning Model Skill And Performance Estimates

Supervised learning is challenging, although the depths of this challenge are often learned then forgotten or willfully ignored. This must be the case, because dwelling too long on this challenge may result in a pessimistic outlook. In spite of the challenge, we continue to wield supervised learning algorithms and they perform well in practice. Generally, it is common knowledge that too little training data results in a poor approximation. Too little test data will result in an optimistic and high variance estimation of model performance. It is critical to make this "common knowledge" concrete with worked examples. In this post, we will work through a detailed case study for developing a Multilayer Perceptron neural network on a simple two-class classification problem. You will discover that, in practice, we don't have enough data to learn the mapping function or to evaluate models, yet supervised learning algorithms like neural networks remain remarkably effective. Impact of Dataset Size on Deep Learning Model Skill And Performance Estimates Photo by Eneas De Troya, some rights reserved.