new deep learning language model
Microsoft Unveils Record-Breaking New Deep Learning Language Model
New model has "real business impact" Microsoft has unveiled the world's largest deep learning language model to-date: a 17 billion-parameter "Turing Natural Language Generation (T-NLG)" model that the company believes will pave the way for more fluent chatbots and digital assistants. The T-NLG "outperforms the state of the art" on a several benchmarks, including summarisation and question answering, Microsoft claimed in a new research blog, as the company stakes its claim to a potentially dominant position in one of the most closely watched new technologies, natural language processing. Deep learning language models like BERT, developed by Google, have hugely improved the powers of natural language processing, by training on colossal data sets with billions of parameters to learn the contextual relations between words. Bigger is not always better, those working on language models may recognise, but Microsoft scientist Corby Rosset said his team "have observed that the bigger the model and the more diverse and comprehensive the pretraining data, the better it performs at generalizing to multiple downstream tasks even with fewer training examples." He emphasised: "Therefore, we believe it is more efficient to train a large centralized multi-task model and share its capabilities across numerous tasks."