The technological standards of photography have dramatically increased over the last few years. While cell phones used to not even have photo capture capabilities, nowadays, it is becoming more and more expected that modern smartphones can take pictures of a quality close to that of a dedicated camera. The computer vision community has recently focused on assessing image aesthetics for image cropping and learning systems for capturing cinematographic videos. In other words, researchers are asking themselves if robots can understand and capture aesthetically pleasing imagery. However, these systems often focus on composing specific objects of interest into the photo by using composition heuristics, like the rule of thirds, and do not translate directly to a system that can capture well-composed photographs in general.

Machine Learning is a subset of Artificial Intelligence that focuses on the use of data and algorithms to allow the machine to learn automatically and improve from experiences without being explicitly coded to do so -- mimicking the human learning process. There are three ways a machine can learn: supervised learning, unsupervised learning, and reinforcement learning. Let's link this back to humans, specifically 6-year-old Anika who is just learning how to add. Supervised Learning is when we teach a machine with labeled data. For example, we would give Anika 30 solved problem sets and tell her which one was added correctly and which ones were added incorrectly.

Reinforcement learning (RL) techniques have shown great success in quantitative investment tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market that reflect billions of rapidly fluctuating values. However, it is hard to apply existing RL methods to intraday trading due to the following three limitations: 1) overlooking micro-level market information (e.g., limit order book); 2) only focusing on local price fluctuation and failing to capture the overall trend of the whole trading day; 3) neglecting the impact of market risk. To tackle these limitations, we propose DeepScalper, a deep reinforcement learning framework for intraday trading. Specifically, we adopt an encoder-decoder architecture to learn robust market embedding incorporating both macro-level and micro-level market information. Moreover, a novel hindsight reward function is designed to provide the agent a long-term horizon for capturing the overall price trend. In addition, we propose a risk-aware auxiliary task by predicting future volatility, which helps the agent take market risk into consideration while maximizing profit. Finally, extensive experiments on two stock index futures and four treasury bond futures demonstrate that DeepScalper achieves significant improvement against many state-of-the-art approaches.

The advent of large pre-trained generative language models has provided a common framework for AI story generation via sampling the model to create sequences that continue the story. However, sampling alone is insufficient for story generation. In particular, it is hard to direct a language model to create stories to reach a specific goal event. We present two automated techniques grounded in deep reinforcement learning and reward shaping to control the plot of computer-generated stories. The first utilizes proximal policy optimization to fine-tune an existing transformer-based language model to generate text continuations but also be goal-seeking. The second extracts a knowledge graph from the unfolding story, which is used by a policy network with graph attention to select a candidate continuation generated by a language model. We report on automated metrics pertaining to how often stories achieve a given goal event as well as human participant rankings of coherence and overall story quality compared to baselines and ablations.

Deep reinforcement learning (Deep RL) has recently seen significant progress in developing algorithms for generalization. However, most algorithms target a single type of generalization setting. In this work, we study generalization across three disparate task structures: (a) tasks composed of spatial and temporal compositions of regularly occurring object motions; (b) tasks composed of active perception of and navigation towards regularly occurring 3D objects; and (c) tasks composed of remembering goal-information over sequences of regularly occurring object-configurations. These diverse task structures all share an underlying idea of compositionality: task completion always involves combining recurring segments of task-oriented perception and behavior. We hypothesize that an agent can generalize within a task structure if it can discover representations that capture these recurring task-segments. For our tasks, this corresponds to representations for recognizing individual object motions, for navigation towards 3D objects, and for navigating through object-configurations. Taking inspiration from cognitive science, we term representations for recurring segments of an agent's experience, "perceptual schemas". We propose Feature Attending Recurrent Modules (FARM), which learns a state representation where perceptual schemas are distributed across multiple, relatively small recurrent modules. We compare FARM to recurrent architectures that leverage spatial attention, which reduces observation features to a weighted average over spatial positions. Our experiments indicate that our feature-attention mechanism better enables FARM to generalize across the diverse object-centric domains we study.

The technologies used in smart homes have recently improved to learn the user preferences from feedback in order to enhance the user convenience and quality of experience. Most smart homes learn a uniform model to represent the thermal preferences of users, which generally fails when the pool of occupants includes people with different sensitivities to temperature, for instance due to age and physiological factors. Thus, a smart home with a single optimal policy may fail to provide comfort when a new user with a different preference is integrated into the home. In this paper, we propose a Bayesian Reinforcement learning framework that can approximate the current occupant state in a partially observable smart home environment using its thermal preference, and then identify the occupant as a new user or someone is already known to the system. Our proposed framework can be used to identify users based on the temperature and humidity preferences of the occupant when performing different activities to enable personalization and improve comfort. We then compare the proposed framework with a baseline long short-term memory learner that learns the thermal preference of the user from the sequence of actions which it takes. We perform these experiments with up to 5 simulated human models each based on hierarchical reinforcement learning. The results show that our framework can approximate the belief state of the current user just by its temperature and humidity preferences across different activities with a high degree of accuracy.

Yifeng Jiang, Research Intern and Jie Tan, Research Scientist, Robotics at Google Simulation empowers various engineering disciplines to ...

The following article is about my RL-Lab idea to make Reinforcement Learning an easier topic to learn. It is the culmination of several years of experience in the field. After several years of involvement in Reinforcement Learning, I have come to the conclusion that no matter how much you study and research this field, you still have this awkward feeling that you don't master it yet, even when it comes to the fundamentals. This nasty feeling stems from the fact that RL is not a simple subject, it is hard and frustrating, even when you have done the same project several times. It is not uncommon that you look at the results in a shock at the end of hours of waiting.

One of the most exciting areas in machine learning right now is reinforcement learning. Its application is found in a diverse set of sectors like data processing, robotics, manufacturing, recommender systems, energy, and games, among others. What makes reinforcement learning (RL) different from other kinds of algorithms is that it does not depend on historical data sets. It learns through trial and error like human beings. Understanding its importance, the last few years have seen an accelerated pace in understanding and improving RL.

In the last post, we learn how to train a simple MOBA game using Deep Reinforcement Learning. In this post, I am going to explain what we need to know before applying the same method to the Dota2. You just need to run the Dotaservice and that code together at same PC. Unlike Derk training, each headless environment of Dota2 requires more than 1GB of RAM memory. Therefore, it is better to use a separate PC for running only environment because DRL training is usually better when there are many environments.