Amazon Releases 51-Language Dataset for Language Understanding

Imagine that all people around the world could use voice AI systems such as Alexa in their native tongues. One promising approach to realizing this vision is massively multilingual natural-language understanding (MMNLU), a paradigm in which a single machine learning model can parse and understand inputs from many typologically diverse languages. By learning a shared data representation that spans languages, the model can transfer knowledge from languages with abundant training data to those in which training data is scarce. Today we are pleased to make three announcements related to MMNLU. First, we are releasing a new dataset called MASSIVE, which is composed of one million labeled utterances spanning 51 languages, along with open-source code that provides examples of how to perform massively multilingual NLU modeling and allows practitioners to re-create the baseline results presented in our paper.. Second, we are launching a new competition using the MASSIVE dataset called Massively Multilingual NLU 2022 (MMNLU-22).