Beating humans at board games is passé in the AI world. Now, top academics and tech companies want to challenge us at video games instead. Today, OpenAI, a research lab founded by Elon Musk and Sam Altman, announced its latest milestone: a team of AI agents that can beat the top 1 percent of amateurs at popular battle arena game Dota 2. You may remember that OpenAI first strode into the world of Dota 2 last August, unveiling a system that could beat the top players at 1v1 matches. However, this game type greatly reduces the challenge of Dota 2. OpenAI has now upgraded its bots to play humans in 5v5 match-ups, which require more coordination and long-term planning. And while OpenAI has yet to challenge the game's very best players, it will do so later this year at The International, a Dota 2 tournament that's the biggest annual event on the e-sports calendar.

OpenAI's video-game-playing bots are getting much better at mastering sci-fi strategy war game Dota 2, seeing off semi pro players with ease in team matchups. However, they can't quite master the whole game to beat top professional teams – yet. Last August, machine-learning software built by the OpenAI lab headquartered in San Francisco managed to best Dendi, a pro Dota 2 player, winning two matches out of three. But the victories were only in one-on-one games – a single bot against a single human – and under very limited circumstances that are not applicable in real competitions. Fast forward about a year, and now OpenAI's bots can play in the more traditional five-versus-five settings, beating amateurs and semi-pro gamers.

The Dota 2 world championship, The Invitational, is fast approaching, and a top team will have a different-looking squad to contend with: a group of artificial intelligence bots. OpenAI, which Elon Musk co-founded, has been taking on top Dota 2 players with the bots since last year, and now it's gunning for a team of top professionals in an exhibition match at one of the biggest events in eSports. OpenAI took on individual players at last year's The Invitational in a one-on-one minigame, and pros said that by watching the matches back, they were able to learn from the bots. But playing as a team introduces different types of intricacies, and OpenAI had to teach the AI how to coordinate the five bots. At any time, a hero (or character) can make one of around 1,000 actions; the bots have to make effective decisions while processing around 20,000 values representing what's going on in the game at a given time.

Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2. While today we play with restrictions, we aim to beat a team of top professionals at The International in August subject only to a limited set of heroes. We may not succeed: Dota 2 is one of the most popular and complex esports games in the world, with creative and motivated professionals who train year-round to earn part of Dota's annual $40M prize pool (the largest of any esports game). OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of Proximal Policy Optimization running on 256 GPUs and 128,000 CPU cores -- a larger-scale version of the system we built to play the much-simpler solo variant of the game last year. Using a separate LSTM for each hero and no human data, it learns recognizable strategies.

This August, some of the world's best professional gamers will travel to Vancouver to fight for millions of dollars in the world's most valuable esports competition. They'll be joined by a team of five artificial intelligence bots backed by Elon Musk, trying to set a new marker for the power of machine learning. The bots were developed by OpenAI, an independent research institute the Tesla CEO cofounded in 2015 to advance AI and prevent the technology from turning dangerous. Vancouver is hosting the annual world championship of Dota 2, one of the internet's most-watched videogames. The prize purse is more than $15 million and growing, exceeding the $11 million at stake at golf's Masters.

When humans and artificial intelligence face off in a game, like chess or Go, it's typically a one-against-one affair. Each player, human or AI, just has to outsmart a single opponent on a board that only changes when the players make a move. OpenAI is announcing today (June 25) that its newest AI bots can hold their own as a team of five against human gamers at Dota 2, a multiplayer game popular in e-sports for its complexity and necessity for teamwork. The AI research lab is looking to take the bots to Dota 2 championship matches in August to compete against the pros. Dota 2 is a challenging game for AI to master simply because of the amount of decisions that the players have to juggle. While chess can end in fewer than 40 moves, and Go fewer than 150, OpenAI's Dota 2 bots make 20,000 moves over the course of a 45 minute game.

The broad set of deep generative models (DGMs) has achieved remarkable advances. However, it is often difficult to incorporate rich structured domain knowledge with the end-to-end DGMs. Posterior regularization (PR) offers a principled framework to impose structured constraints on probabilistic models, but has limited applicability to the diverse DGMs that can lack a Bayesian formulation or even explicit density evaluation. PR also requires constraints to be fully specified {\it a priori}, which is impractical or suboptimal for complex knowledge with learnable uncertain parts. In this paper, we establish mathematical correspondence between PR and reinforcement learning (RL), and, based on the connection, expand PR to learn constraints as the extrinsic reward in RL. The resulting algorithm is model-agnostic to apply to any DGMs, and is flexible to adapt arbitrary constraints with the model jointly. Experiments on human image generation and templated sentence generation show models with learned knowledge constraints by our algorithm greatly improve over base generative models.

We investigate deep generative models, which allow us to use training data from one domain to build a model for another domain. We consider domains to have similar structure (texts, images). We propose the Variational Bi-domain Triplet Autoencoder (VBTA) that learns a joint distribution of objects from different domains. There are many cases when obtaining any supervision (e.g. paired data) is difficult or ambiguous. For such cases we can seek a method that is able to the information about data relation and structure from the latent space. We extend the VBTAs objective function by the relative constraints or triplets that sampled from the shared latent space across domains. In other words, we combine the deep generative model with a metric learning ideas in order to improve the final objective with the triplets information. We demonstrate the performance of the VBTA model on different tasks: bi-directional image generation, image-to-image translation, even on unpaired data. We also provide the qualitative analysis. We show that VBTA model is comparable and outperforms some of the existing generative models.

Essentially, Q-learning lets the agent use the environment's rewards to learn, over time, the best action to take in a given state. In our Taxi environment, we have the reward table, P, that the agent will learn from. It does thing by looking receiving a reward for taking an action in the current state, then updating a Q-value to remember if that action was beneficial. The values store in the Q-table are called a Q-values, and they map to a (state, action) combination. A Q-value for a particular state-action combination is representative of the "quality" of an action taken from that state.

Our system works in two stages; first we train a transformer model on a very large amount of data in an unsupervised manner -- using language modeling as a training signal -- then we fine-tune this model on much smaller supervised datasets to help it solve specific tasks. We developed this approach following our sentiment neuron work, in which we noted that unsupervised learning techniques can yield surprisingly discriminative features when trained on enough data. Here, we wanted to further explore this idea: can we develop one model, train it in an unsupervised way on a large amount of data, and then fine-tune the model to achieve good performance on many different tasks? Our results indicate that this approach works surprisingly well; the same core model can be fine-tuned for very different tasks with minimal adaptation. This work builds on the approach introduced in Semi-supervised Sequence Learning, which showed how to improve document classification performance by using unsupervised pre-training of an LSTM followed by supervised fine-tuning.