In this paper, we derive a generalization of the Speedy Q-learning (SQL) algorithm that was proposed in the Reinforcement Learning (RL) literature to handle slow convergence of Watkins' Q-learning. In most RL algorithms such as Q-learning, the Bellman equation and the Bellman operator play an important role. It is possible to generalize the Bellman operator using the technique of successive relaxation. We use the generalized Bellman operator to derive a simple and efficient family of algorithms called Generalized Speedy Q-learning (GSQL-w) and analyze its finite time performance. We show that GSQL-w has an improved finite time performance bound compared to SQL for the case when the relaxation parameter w is greater than 1. This improvement is a consequence of the contraction factor of the generalized Bellman operator being less than that of the standard Bellman operator. Numerical experiments are provided to demonstrate the empirical performance of the GSQL-w algorithm.

We consider Monte-Carlo Tree Search (MCTS) applied to Markov Decision Processes (MDPs) and Partially Observable MDPs (POMDPs), and the well-known Upper Confidence bound for Trees (UCT) algorithm. In UCT, a tree with nodes (states) and edges (actions) is incrementally built by the expansion of nodes, and the values of nodes are updated through a backup strategy based on the average value of child nodes. However, it has been shown that with enough samples the maximum operator yields more accurate node value estimates than averaging. Instead of settling for one of these value estimates, we go a step further proposing a novel backup strategy which uses the power mean operator, which computes a value between the average and maximum value. We call our new approach Power-UCT and argue how the use of the power mean operator helps to speed up the learning in MCTS. We theoretically analyze our method providing guarantees of convergence to the optimum. Moreover, we discuss a heuristic approach to balance the greediness of backups by tuning the power mean operator according to the number of visits to each node. Finally, we empirically demonstrate the effectiveness of our method in well-known MDP and POMDP benchmarks, showing significant improvement in performance and convergence speed w.r.t. UCT.

DD-PPO is distributed (uses multiple machines), decentralized (lacks a centralized server), and synchronous (no computation is ever'stale'), making it conceptually simple and easy to implement. In our experiments on training virtual robots to navigate in Habitat-Sim (Savva et al., 2019), DD-PPO exhibits near-linear scaling - achieving a speedup of 107x on 128 GPUs over a serial implementation. We leverage this scaling to train an agent for 2.5 Billion steps of experience (the equivalent of 80 years of human experience) - over 6 months of GPU-time training in under 3 days of wall-clock time with 64 GPUs. This massive-scale training not only sets the state of art on Habitat Autonomous Navigation Challenge 2019, but essentially'solves' the task - near-perfect autonomous navigation in an unseen environment without access to a map, directly from an RGB-D camera and a GPS Compass sensor. Fortuitously, error vs computation exhibits a power-law-like distribution; thus, 90% of peak performance is obtained relatively early (at 100 million steps) and relatively cheaply (under 1 day with 8 GPUs). Finally, we show that the scene understanding and navigation policies learned can be transferred to other navigation tasks - the analog of'ImageNet pre-training task-specific fine-tuning' for embodied AI. Our model outperforms ImageNet pre-trained CNNs on these transfer tasks and can serve as a universal resource (all models code will be publicly available). 1 I NTRODUCTION Recent advances in deep reinforcement learning (RL) have given rise to systems that can outperform human experts at variety of games (Silver et al., 2017; Tian et al., 2019; OpenAI, 2018). These advances, even more-so than those from supervised learning, rely on significant numbers of training samples, making them impractical without large-scale, distributed parallelization. Thus, scaling RL via multi-node distribution is of importance to AI - that is the focus of this work. Several works have proposed systems for distributed RL (Heess et al., 2017; Liang et al., 2018; Tian et al., 2019; Silver et al., 2016; OpenAI, 2018; Espeholt et al., 2018). These works utilize two core components: 1) workers that collect experience ('rollout workers'), and 2) a parameter server that optimizes the model. The rollout workers are then distributed across, potentially, thousands of CPUs 1 .

Generative adversarial imitation learning (GAIL) has attracted increasing attention in the field of robot learning. It enables robots to learn a policy to achieve a task demonstrated by an expert while simultaneously estimating the reward function behind the expert's behaviors. However, this framework is limited to learning a single task with a single reward function. This study proposes an extended framework called situated GAIL (S-GAIL), in which a task variable is introduced to both the discriminator and generator of the GAIL framework. The task variable has the roles of discriminating different contexts and making the framework learn different reward functions and policies for multiple tasks. To achieve the early convergence of learning and robustness during reward estimation, we introduce a term to adjust the entropy regularization coefficient in the generator's objective function. Our experiments using two setups (navigation in a discrete grid world and arm reaching in a continuous space) demonstrate that the proposed framework can acquire multiple reward functions and policies more effectively than existing frameworks. The task variable enables our framework to differentiate contexts while sharing common knowledge among multiple tasks.

W e present a set of capabilities allowing an agent planning with moral and social norms represented in temporal logic to respond to queries about its norms and behaviors in natural language, and for the human user to add and remove norms directly in natural language. The user may also pose hypothetical modifications to the agent's norms and inquire about their effects.

W e present an approach to generating natural language justifications of decisions derived from norm-based reasoning. Assuming an agent which maximally satisfies a set of rules specified in an object-oriented temporal logic, the user can ask factual questions (about the agent's rules, actions, and the extent to which the agent violated the rules) as well as "why" questions that require the agent comparing actual behavior to counterfactual trajectories with respect to these rules. To produce natural-sounding explanations, we focus on the subproblem of producing natural language clauses from statements in a fragment of temporal logic, and then describe how to embed these clauses into explanatory sentences. W e use a human judgment evaluation on a testbed task to compare our approach to variants in terms of intelligibility, mental model and perceived trust.

What kind of feelings spring to mind when you hear the words "Artificial Intelligence"? Are they positive or negative? Considering how AI is represented in the media, chances are that those feelings are somewhat negative. There are plenty of examples to support those feelings. Even the positive-sounding stories about "efficiency" and "productivity" mask a rather uncomfortable question for most people reading them that basically boils down to, "Will the robots take my job?" It's easy to focus on the negative stories, but it's important to balance this by recognising that, as humans, we're all susceptible to hard-wired cognitive biases that skew our sense of risk.

The AI Forum continues to publish the outputs of our research programme. Two new reports AI for Health in New Zealand / Haoura i te Atamai Iahiko and AI for Agriculture in New Zealand / Ahuwhenua i te Atamai Iahiko explore in depth the AI opportunities for New Zealand's crucial health and agriculture sectors. Continued thanks to the AI Forum's research programme partners for their foundational support to enable this work. AI Forum Executive Council members Christopher Laing (Xero) and Michael Witbrock (University of Auckland) were recently interviewed by Kathryn Ryan on RNZ's Nine To Noon show, listen to AI: two years for NZ to get it right. Meanwhile, I was interviewed at length by the Spinoff's Russell Brown in the latest episode of the Microsoft'Artificial Intelligence – Actually Interesting' podcast series: The cancer-fighting, wildlife-protecting, life-saving power of artificial intelligence.

Google has released new behavioural insights into 2019 holiday shoppers, with tips on how to sell to US consumers. According to Google's data, mobile searches for "best deals" have grown by 90%. So it should come as no surprise that the #1 factor when consumers decide where to buy is whichever retailer has the lowest price. Consumers also appreciate being able to do what they want all on their own. It's also interesting to note that searches around "rewards apps" and "Black Friday deals" are up 200% this year.

MOSCOW, Feb. 13, 2019 /PRNewswire-PRWeb/ -- Yandex.Mail has become one of the leading email platforms in Russia, with over 28 million active monthly users. To help users better manage their emails, the Yandex.Mail team developed a new feature that more easily manages and filters users' unwanted subscription emails. Yandex has integrated machine learning algorithms into the platform to determine which emails are coming from subscription services and can tell which emails from the same sender are relevant to users. The user can decide which subscriptions they want to hide from their inbox by selecting multiple senders and hiding their messages with one click. Along with no longer receiving future subscription emails from the hidden senders, users can also opt to delete all past messages from them.