One hyperparameter in our speech translation evaluation is the threshold on the alignment scores. Common V oice in Figure 5, and find the optimal value for the threshold. As we decrease the threshold, more mined data is added to the train set improving model performance. Similar to our observation on the dev set, the optimal threshold is t = 1 .07 on the test set.

Our approach can also be used to directly perform speech-to-speech mining, without the need to first transcribe or translate the data.

We study speech-to-speech translation (S2ST) that translates speech from one language into another language and focuses on building systems to support languages without standard text writing systems. We use English-Taiwanese Hokkien as a case study, and present an end-to-end solution from training data collection, modeling choices to benchmark dataset release. First, we present efforts on creating human annotated data, automatically mining data from large unlabeled speech datasets, and adopting pseudo-labeling to produce weakly supervised data. On the modeling, we take advantage of recent advances in applying self-supervised discrete representations as target for prediction in S2ST and show the effectiveness of leveraging additional text supervision from Mandarin, a language similar to Hokkien, in model training. Finally, we release an S2ST benchmark set to facilitate future research in this field. The demo can be found at https://huggingface.co/spaces/facebook/Hokkien_Translation .

We present SpeechMatrix, a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. It contains speech alignments in 136 language pairs with a total of 418 thousand hours of speech. To evaluate the quality of this parallel speech, we train bilingual speech-to-speech translation models on mined data only and establish extensive baseline results on EuroParl-ST, VoxPopuli and FLEURS test sets. Enabled by the multilinguality of SpeechMatrix, we also explore multilingual speech-to-speech translation, a topic which was addressed by few other works. We also demonstrate that model pre-training and sparse scaling using Mixture-of-Experts bring large gains to translation performance. The mined data and models are freely available.

We present a textless speech-to-speech translation (S2ST) system that can translate speech from one language into another language and can be built without the need of any text data. Different from existing work in the literature, we tackle the challenge in modeling multi-speaker target speech and train the systems with real-world S2ST data. The key to our approach is a self-supervised unit-based speech normalization technique, which finetunes a pre-trained speech encoder with paired audios from multiple speakers and a single reference speaker to reduce the variations due to accents, while preserving the lexical content. With only 10 minutes of paired data for speech normalization, we obtain on average 3.2 BLEU gain when training the S2ST model on the \vp~S2ST dataset, compared to a baseline trained on un-normalized speech target. We also incorporate automatically mined S2ST data and show an additional 2.0 BLEU gain. To our knowledge, we are the first to establish a textless S2ST technique that can be trained with real-world data and works for multiple language pairs.

There's a lot of misinformation and misunderstanding around what computers can and can't do. Unfortunately, while artificial intelligence might not be as sensational as a summer blockbuster, it's just as exciting in the market research industry. A quick education on the difference between data mining, artificial intelligence, and machine learning (and how they play together) can give you a basic understanding of why they're the real stars of market research, and, if used together, can present a formidable tactic that one can use to conquer any data question or conundrum. Data mining is actually one of the newer methods that market research companies are employing, but it serves as a foundation for both artificial intelligence and machine learning. Data mining, as a practice, is more than just culling supersets of information from various sources. Data mining can cull and then aggregate information to alert you to patterns and correlations that you hadn't even thought of.