Building NLP Classifiers Cheaply With Transfer Learning and Weak Supervision

There is a catch to training state-of-the-art NLP models: their reliance on massive hand-labeled training sets. That's why data labeling is usually the bottleneck in developing NLP applications and keeping them up-to-date. For example, imagine how much it would cost to pay medical specialists to label thousands of electronic health records. In general, having domain experts label thousands of examples is too expensive. On top of the initial labeling cost, there is another huge cost in keeping models up-to-date with changing contexts in the real-world.