As deep neural networks have come to dominate AI, the Conference on Neural Information Processing Systems (NeurIPS) has become the most popular conference in the field. And at the most popular conference in the field, one of the most popular topics is reinforcement learning: at this year's NeurIPS, 95 accepted papers use the term in their titles. "Reinforcement learning is very, very powerful, because you can kind of learn anything, adaptively from the feedback, and by exploring the decision space," says Shipra Agrawal, an Amazon Scholar, an assistant professor in Columbia University's Industrial Engineering and Operations Research Department, and an area chair at NeurIPS, who studies reinforcement learning. "In concept, it's very akin to how humans learn, by trial and error, and how they adapt to what they see -- without requiring a loss function and so on, just by some kind of rewards or positive feedback." In reinforcement learning, an agent explores its environment, trying out different responses to different states of affairs, gradually learning a set of policies that will enable it to maximize some reward.

Amazon operates in 14 countries around the world, nine of which are eligible for its Prime yearly subscription service. It goes without saying that the company has a real desire to make available its shopping experience in any number of languages, particularly where customers who speak different dialects are searching for the same products. In pursuit of an efficient means of translating multiple languages, Amazon researchers devised a shopping model called a multitask model, in which the functions overlap across tasks and tend to reinforce each other. They say that their AI, which was trained on data from several different languages at once, delivered better results using any of those languages. As Amazon applied scientist Nikhil Rao explained in a blog post, the reason for the improvement is that a corpus in one language is able to fill gaps in that of another language.

Being an early adopter of artificial intelligence and automation, Amazon always had an edge in using AI to improve its business efficiencies. Not only has it been using AI to enhance its customer experience but has been heavily focused internally. From using AI to predict the number of customers willing to buy a new product to running a cashier-less grocery store, Amazon's AI capabilities are designed to provide customised recommendations to its customers. According to a report, Amazon's recommendation engine is driving 35% of its total sales. One of the main areas where Amazon is applying continuous AI is to better understand their customer search queries and what is the reason they are looking for a particular product.

Amazon operates in 14 countries around the world, nine of which are eligible for its Prime yearly subscription service. It goes without saying that the company has a real desire to make available its shopping experience in any number of languages, particularly where customers who speak different dialects are searching for the same products. In pursuit of an efficient means of translating multiple languages, Amazon researchers devised a shopping model called a multitask model, in which the functions overlap across tasks and tend to reinforce each other. They say that their AI, which was trained on data from several different languages at once, delivered better results using any of those languages. As Amazon applied scientist Nikhil Rao explained in a blog post, the reason for the improvement is that a corpus in one language is able to fill gaps in that of another language.

Amazon operates in 14 countries around the world, nine of which are eligible for its Prime yearly subscription service. It goes without saying that the company has a real desire to make available its shopping experience in any number of languages, particularly where customers who speak different dialects are searching for the same products. In pursuit of an efficient means of translating multiple languages, Amazon researchers devised a shopping model called a multitask model, in which the functions overlap across tasks and tend to reinforce each other. They say that their AI, which was trained on data from several different languages at once, delivered better results using any of those languages. As Amazon applied scientist Nikhil Rao explained in a blog post, the reason for the improvement is that a corpus in one language is able to fill gaps in that of another language.

Over the past 10 years, commercial AI has enjoyed what we at Amazon call the flywheel effect: customer interactions with AI systems generate data; with more data, machine learning algorithms perform better, which leads to better customer experiences; better customer experiences drive more usage and engagement, which in turn generate more data. Those data are used to train machine learning systems in three chief ways. The first is supervised learning, in which the training data are hand-labeled (with, say, words' parts of speech or the names of objects in an image) and the system learns to apply labels to unlabeled data. A variation of this is weakly supervised learning, which uses easily acquired but imprecise labels to enable machine learning at scale. If a website visitor performs a search, for instance, the links she clicks indicate which search results should have been at the top of the list; that kind of implicit information can be used to automatically label data. Training with entirely unlabeled data is called unsupervised learning.

Classifiers are a staple of modern-day machine learning. Simply put, they categorize input data -- photos, videos, objects, and recordings -- by type, and do it very efficiently. However, problems arise when a classifier needs a new class -- that is, a new category. Adding even one new class is traditionally arduous and involves lots of data collection and model retraining. But scientists at Amazon's Alexa research division say it doesn't have to be that way.