Computations for the softmax function are significantly expensive when the number of output classes is large. In this paper, we present a novel softmax inference speedup method, Doubly Sparse Softmax (DS-Softmax), that leverages sparse mixture of sparse experts to efficiently retrieve top-k classes. Different from most existing methods that require and approximate a fixed softmax, our method is learning-based and can adapt softmax weights for a better approximation. In particular, our method learns a two-level hierarchy which divides entire output class space into several partially overlapping experts. Each expert is sparse and only contains a subset of output classes. To find top-k classes, a sparse mixture enables us to find the most probable expert quickly, and the sparse expert enables us to search within a small-scale softmax. We empirically conduct evaluation on several real-world tasks (including neural machine translation, language modeling and image classification) and demonstrate that significant computation reductions can be achieved without loss of performance.

While there have been several key developments in machine translation (MT) in recent years, MT has not yet reached the level where businesses might be confident to allow it to proceed unchecked by humans. There is a paradox insofar that we want to allow artificial intelligence (AI) and automation to take on more and more tasks to relieve pressure on the human workforce, but, in turn, this creates more work for humans in terms of supervising their digital colleagues. We need look no further than the restaurant in China called "Translate Server Error" or Hillary Clinton's gift to the Russian foreign minister that was inscribed with a message that was supposed to say "reset" in Russian but actually showed the word "overcharge." AI still commits fundamental errors that are embarrassing at best, and at worst, they can convey offensive and/or completely unintended meanings. This is where the importance of quality estimation comes to the fore. A good definition of quality estimation comes from eBay, an enthusiastic user of QE: "A method used to automatically provide a quality indication for machine translation output without depending on human reference translations.

Late last year we hosted SDL's Japan Customer Summit, a one-day event attended by close to 50 of Japan's leading companies from all industries including retail, life sciences, automotive and finance. While the event itself has passed, we wanted to reflect on some of the key highlights that came out of the event and what we heard from a number of industry specialists. The event itself played host to a raft of experts from across SDL, exploring the latest developments in AI and Machine Learning (click here to read part 1 of this blog). This blog looks at the second half of the day, which covered practical use cases and scenarios where we showed how the latest technological developments can offer the greatest impact on a business. Mihai Vlad, VP of Machine Learning, explored the world of Machine Learning, more specifically how the accuracy of Machine Translation affects the ROI of content.

Artificial intelligence technology has run into a potentially delicate issue: gender bias. In November 2018, mainstream news media reported that Google's automatic suggestion tool for Google Mail will not suggest gender-based pronouns to avoid autocompleting a sentence with the wrong gender. The feature (called Smart Compose) will avoid suggesting genders because, as Gmail Product Manager Paul Lambert put it, "not all'screw-ups' are equal…[gender is] a big, big thing." Google Translate, which now largely runs on neural machine translation (NMT), had also recently addressed the question of gender bias. On December 6, 2018, Google published a first blog post about its efforts to reduce gender bias in Google Translate.

Maxim Khalilov is a director of applied artificial intelligence at Unbabel leading a team of AI engineers to apply AI technologies to meet the needs of the Unbabel business. Prior to his current role, he was a product owner in data science at Booking.com responsible for exploitation, collection and exploitation of digital content for hospitality market, a CTO at an innovative language service provider bmmt GmbH and an R&D manager at TAUS, a resource center for the global language industries. Maxim is also a co-founder of an Natural Language Processing company NLPPeople.com,

Let's say I am given an Excel sheet with data about various fruits and I have to tell which look like Apples. What I will do is ask a question "Which fruits are red and round?" and divide all fruits which answer yes and no to the question. Now, All Red and Round fruits might not be apples and all apples won't be red and round. So I will ask a question "Which fruits have red or yellow color hints on them? " on red and round fruits and will ask "Which fruits are green and round?" on not red and round fruits. Based on these questions I can tell with considerable accuracy which are apples. This cascade of questions is what a decision tree is. However, this is a decision tree based on my intuition.

In a previous article, I discussed the current pace of AI innovation. The shortage of available AI training data is a huge blocker in AI innovation today, leaving some businesses frustrated. In recent years, some media channels hyped up that AI technology will advance exponentially at lightning speed, but so far that has not happened. We don't have enough AI training data because companies often underestimate the amount of data they need, and the time to collect that data. The few companies invested in data collection often refuse to make their data public, usually due to privacy concerns or fear of losing to their competitors.

One question I get fairly often from folks who are just getting into NLP is how to evaluate systems when the output of that system is text, rather than some sort of classification of the input text. These types of problems, where you put some text into your model and get some other text out of it, are known as sequence to sequence or string transduction problems. This sort of technology is right out of science fiction. With such a wide range of exciting applications, it's easy to see why sequence to sequence modeling is more popular than ever. What's not easy is actually evaluating these systems. Unfortunately for folks who are just getting started, there's no simple answer about what metric you should use to evaluate your model. Even worse, one of the most popular metrics for evaluating sequence to sequence tasks, BLEU, has major drawbacks, especially when applied to tasks that it was never intended to evaluate.

Artificial intelligence (AI) has infiltrated numerous aspects of our lives in recent years, thanks to improvements in the field of machine learning, where computers ostensibly program themselves. This drive towards digital self-learning has led to major breakthroughs in our day-to-day interactions with machines, most notably the rise of digital home assistants such as Amazon Echo, and the recently launched Google Lens, which identifies objects based on visual cues from your phone's camera. One of the most widely-discussed advances has been the use of AI in translation. Not unlike the Babel Fish from The Hitchhiker's Guide to the Galaxy, with AI translation, "you can instantly understand anything said to you in any form of language." The technology works by recognizing words individually and then, as MIT Technology Review puts it, "takes advantage of the fact that relationships between certain words…are similar across languages" to create its translations. It has already found its way into a number of our most commonly used websites and platforms, with even grander plans in the pipeline – but just how reliable is the technology?

Recent GAN-based architectures have been able to deliver impressive performance on the general task of image-to-image translation. In particular, it was shown that a wide variety of image translation operators may be learned from two image sets, containing images from two different domains, without establishing an explicit pairing between the images. This was made possible by introducing clever regularizers to overcome the under-constrained nature of the unpaired translation problem. In this work, we introduce a novel architecture for unpaired image translation, and explore several new regularizers enabled by it. Specifically, our architecture comprises a pair of GANs, as well as a pair of translators between their respective latent spaces. These cross-translators enable us to impose several regularizing constraints on the learnt image translation operator, collectively referred to as latent cross-consistency. Our results show that our proposed architecture and latent cross-consistency constraints are able to outperform the existing state-of-the-art on a wide variety of image translation tasks.