High sample complexity has long been a challenge for RL. On the other hand, humans learn to perform tasks not only from interaction or demonstrations, but also by reading unstructured text documents, e.g., instruction manuals. Instruction manuals and wiki pages are among the most abundant data that could inform agents of valuable features and policies or task-specific environmental dynamics and reward structures. Therefore, we hypothesize that the ability to utilize human-written instruction manuals to assist learning policies for specific tasks should lead to a more efficient and better-performing agent. We propose the Read and Reward framework. Read and Reward speeds up RL algorithms on Atari games by reading manuals released by the Atari game developers. Our framework consists of a QA Extraction module that extracts and summarizes relevant information from the manual and a Reasoning module that evaluates object-agent interactions based on information from the manual. An auxiliary reward is then provided to a standard A2C RL agent, when interaction is detected. Experimentally, various RL algorithms obtain significant improvement in performance and training speed when assisted by our design.

High sample complexity has long been a challenge for RL. On the other hand, humans learn to perform tasks not only from interaction or demonstrations, but also by reading unstructured text documents, e.g., instruction manuals. Instruction manuals and wiki pages are among the most abundant data that could inform agents of valuable features and policies or task-specific environmental dynamics and reward structures. Therefore, we hypothesize that the ability to utilize human-written instruction manuals to assist learning policies for specific tasks should lead to a more efficient and better-performing agent. We propose the Read and Reward framework.

Model-based reinforcement learning agents utilizing transformers have shown improved sample efficiency due to their ability to model extended context, resulting in more accurate world models. However, for complex reasoning and planning tasks, these methods primarily rely on continuous representations. This complicates modeling of discrete properties of the real world such as disjoint object classes between which interpolation is not plausible. In this work, we introduce discrete abstract representations for transformer-based learning (DART), a sample-efficient method utilizing discrete representations for modeling both the world and learning behavior. We incorporate a transformer-decoder for auto-regressive world modeling and a transformer-encoder for learning behavior by attending to task-relevant cues in the discrete representation of the world model. For handling partial observability, we aggregate information from past time steps as memory tokens. DART outperforms previous state-of-the-art methods that do not use look-ahead search on the Atari 100k sample efficiency benchmark with a median human-normalized score of 0.790 and beats humans in 9 out of 26 games. We release our code at https://pranaval.github.io/DART/.

Google is teaching machines to play Atari games like Space Invaders, Video Pinball, and Breakout. At DeepMind, a Google subsidiary based in Cambridge, England, researchers have built artificial intelligence software that's so adept at these classic games, it can sometimes beat a human player--and a professional, at that. This may seem like a frivolous, if intriguing, pursuit. If a machine can learn to navigate the digital world of a video game, Google says, it eventually could learn to navigate the real world, too. Today, this AI can play Space Invaders.

Deep Learning is becoming the next big area for companies and universities to explore. Deep Learning libraries are growing and their adoption is expanding. With Google's open sourcing of TensorFlow, there is a massive rise in deep learning adoption. I have started using it for it's very interesting Image Recognition capabilities which can be used out of the box with their ImageRecognition example. Google has released a new TensorFlow library - Image Recognition, Slim. TF-Slim is a lightweight library for defining, training and evaluating complex models in TensorFlow, which should speed up adoption and ease of use.

Google is teaching machines to play Atari games like Space Invaders, Video Pinball, and Breakout. At DeepMind, a Google subsidiary based in Cambridge, England, researchers have built artificial intelligence software that's so adept at these classic games, it can sometimes beat a human player--and a professional, at that. This may seem like a frivolous, if intriguing, pursuit. If a machine can learn to navigate the digital world of a video game, Google says, it eventually could learn to navigate the real world, too. Today, this AI can play Space Invaders.

We show that setting a reasonable frame skip can be critical to the performance of agents learning to play Atari 2600 games. In all of the six games in our experiments, frame skip is a strong determinant of success. For two of these games, setting a large frame skip leads to state-of-the-art performance.