Microsoft and Nvidia team up to train one of the world's largest language models

The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. Microsoft and Nvidia today announced that they trained what they claim is the largest and most capable AI-powered language model to date: Megatron-Turing Natural Language Generation (MT-NLP). The successor to the companies' Turing NLG 17B and Megatron-LM models, MT-NLP contains 530 billion parameters and achieves "unmatched" accuracy in a broad set of natural language tasks, Microsoft and Nvidia say -- including reading comprehension, commonsense reasoning, and natural language inferences. "The quality and results that we have obtained today are a big step forward in the journey towards unlocking the full promise of AI in natural language. The innovations of DeepSpeed and Megatron-LM will benefit existing and future AI model development and make large AI models cheaper and faster to train," Nvidia's senior director of product management and marketing for accelerated computing, Paresh Kharya, and group program manager for the Microsoft Turing team, Ali Alvi wrote in a blog post.