Machine learning limitations marked by data demands

Data is the core of machine learning. The very nature of machine learning is to train an algorithm on clean and prepared sample data. Through this repeated process, it can learn from the data set and create and apply generalizations to data it has never seen before. One of the more impressive feats of machine learning to date is represented by the remarkable performance and capability of OpenAI's GPT-3 model, which can generate surprisingly humanlike text output from just a small amount of starter text. While the results are noteworthy, the reality is that petabytes of data, millions of dollars of CPU and GPU power and many hours of training time went into creating the resulting model. This quantity of data and computing is not available to the average machine learning model developer and highlights one of the major challenges with the current state of the art for machine learning: an extreme dependency on data.