Chimera: Large-Scale Classification Using Machine Learning, Rules, and Crowdsourcing

Large-scale classification, where we need to classify hundreds of thousands or millions of items into thousands of classes, is becoming increasingly common in this age of Big Data… So far, however, very little has been published on how large-scale classification has been carried out in practice, even though there are many interesting questions about such cases. Today's paper is a case study on large-scale classification of products at Walmart. The requirement is to classify 10M products into 5000 categories based on fairly minimal product descriptions. Oh, and new products turn up all the time, and the set of categories is continuously evolving. Many learning solutions assume that we can take a random sample from the universe of items, manually label the sample to create training data, then train a classifier.