Artificial intelligence has swept the board with humans in games like chess and Go, but taking on e-sports might be a step too far -- for now. At The International tournament last night in Vancouver, a team of human pro gamers defeated a team of AI bots at the battle arena game Dota 2. The victory for team human was decisive but by no means inevitable, with the AI players putting up a valiant fight. And with two more games to play this week, machine might yet triumph over humanity. The bots were the creation of OpenAI, a non-profit research lab founded by tech luminaries such as SpaceX CEO Elon Musk. The lab's main goal is to develop artificial intelligence that "benefits all of humanity," but teaching bots to play Dota has been an important research task for some time now.

One way to measure progress in artificial intelligence is to chart victories by algorithms over champions of increasingly challenging games--checkers, chess, and, in 2016, Go. On Wednesday, five bots sought to extend AI's mastery to e-sports, in the fantasy battle game Dota 2. They failed, as a team of pro gamers from Brazil called paiN defended humanity's honor--for now. A crowd of thousands in Vancouver's hockey arena watched the bots battle paiN over 52 tense minutes packed with spells and firebolts. The human-machine contest was a side event to The International, a Dota 2 tournament that boasts the biggest purse in e-sports, at $25 million. The five bots that lost Wednesday were created by OpenAI, a research institute cofounded by Tesla CEO Elon Musk to work towards human-level artificial intelligence, and make the technology safe.

The International In the past hour, OpenAI's artificially intelligent bots lost their first match against professional players at smash-hit computer game Dota 2 at The International – the video game's annual championship tournament. It's the first bout in a best-of-three competition between human professional players versus OpenAI's code, the other two rounds will take place over the next two days, each day a different human team. Thousands of hardcore Dota fans lit up by glowing bracelets sat down at the Rogers Arena in Vancouver, Canada, to watch pros battle against a machine running OpenAI's software in this first round. The humans – dubbed Team paiN – were five players from Brazil, while OpenAI Five is made up five long-short-term memory neural-network-based agents. Dota 2 is a popular battle strategy game played online.

How did OpenAI's team of 5 neural networks manage to beat some of the world's best DOTA 2 players? In this video, I'll explain in detail the cutting edge research techniques OpenAI used to create such an incredible AI algorithm, and how it could be used in the real world. These techniques include Long Short Term Memory Recurrent Neural Networks, Proximal Policy Optimization, and a custom rollout system they've dubbed'Rapid'. That's what keeps me going. Sign up for the next course at The School of AI: https://www.theschool.ai

It aims to promote and develop friendly AI in a way to benefit humanity as a whole. It aims to "freely collaborate" by making its patents and research open to the public.

To put that into perspective, experts at OpenAI recently developed Dactyl, a robotic hand that could handle objects. This is a task that any human child learns to perform subconsciously at an early age. But it took Dactyl 6,144 CPUs and 8 GPUs and about one hundred years' worth of experience to develop the same skills. While it is a fascinating achievement, it also highlights the stark differences between narrow AI and the way the human brain works.

For engineers at the Elon Musk-founded nonprofit OpenAI, this presented both a challenge and an opportunity. How could their researchers use artificial intelligence to teach a robot to manipulate objects as artfully as a human? Usually, when teaching an AI to control a physical robot, scientists tend to come up against the same problems. Training is often done using reinforcement learning; a method where the AI learns through a process of trial and error. But this requires a lot of time, usually amounting to years of experience.

"OpenAI Five plays 180 years worth of games against itself every day." Artificial Intelligence vs Humans is a hotly debated topic these days and the gaming arena holds one of the prime witnesses of that. DOTA 2 players have taken on an AI algorithm before in a head-to-head match and have lost. The story seems to continue to date, a recent match between a team of 5 humans vs a mix of AI programs, being a proof of that. The AI programs, developed by OpenAI - an AI research lab founded by Elon Musk and Y Combinator president Sam Altman, won 2 out of 3 matches against the 5 semi-professional humans working together as a team.

Summarizing, there are four desiderata that one would like to see in A.I. systems, if they have to compare well with human (or just animal) brains: meta-learning, learning by demonstration ("few-shot learning"), transfer learning and multi-task learning. Meta-learning is particularly relevant in the case of reinforcement learning. It is obvious that reinforcement learning is highly unnatural. DeepMind's AlphaGo and OpenAi Five need to learn from scratch via a huge number of trials. Animals, instead, use built-in or acquired "meta-skills" to learn new tasks in just a few trials. Modern computational theory of meta-learning (learning how to learn) dates back at least to the 1990s, when Schmidhuber published the manifesto "Simple Principles of Metalearning" (1996), followed by his student Sepp Hochreiter ("Learning to Learn Using Gradient Descent", 2001), and by Nicolas Schweighofer and Kenji Doya at Japan's ATR ("Meta-learning in Reinforcement Learning", 2001). Examples of "deep" meta-learning systems of the new generation are: RL Square by Pieter Abbeel's student Yan Duan at UC Berkeley, based on Schulman's TRPO ("RL Square: Fast Reinforcement Learning via Slow Reinforcement Learning", 2016); the "model-agnostic meta-learning" (MAML) of Sergey Levine's student Chelsea Finn at UC Berkeley ("Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks", 2017); Marcel Binz's thesis at KTH Royal Institute of Technology ("Learning Goal-Directed Behaviour", 2017); Jane Wang's "deep meta-reinforcement learning" at DeepMind ("Learning to Reinforcement Learn", 2017); and OpenAI's Reptile, developed by Alex Nichol and John Schulman, a generalization of Finn's MAML ("On First-Order Meta-Learning Algorithms", 2018). DeepMind's neuroscientist Matthew Botvinick believes that the latter could be a model for how our brain learns: the dopamine system trains another part of the brain, the prefrontal cortex, to operate as its own free-standing learning system ("Prefrontal Cortex as a Meta-reinforcement Learning System", 2018).

This document aims to provide a review on learning with deep generative models (DGMs), which is an highly-active area in machine learning and more generally, artificial intelligence. This review is not meant to be a tutorial, but when necessary, we provide self-contained derivations for completeness. This review has two features. First, though there are different perspectives to classify DGMs, we choose to organize this review from the perspective of graphical modeling, because the learning methods for directed DGMs and undirected DGMs are fundamentally different. Second, we differentiate model definitions from model learning algorithms, since different learning algorithms can be applied to solve the learning problem on the same model, and an algorithm can be applied to learn different models. We thus separate model definition and model learning, with more emphasis on reviewing, differentiating and connecting different learning algorithms. We also discuss promising future research directions. This review is by no means comprehensive as the field is evolving rapidly. The authors apologize in advance for any missed papers and inaccuracies in descriptions. Corrections and comments are highly welcome.