We study offline data poisoning attacks in contextual bandits, a class of reinforcement learning problems with important applications in online recommendation and adaptive medical treatment, among others. We provide a general attack framework based on convex optimization and show that by slightly manipulating rewards in the data, an attacker can force the bandit algorithm to pull a target arm for a target contextual vector. The target arm and target contextual vector are both chosen by the attacker. That is, the attacker can hijack the behavior of a contextual bandit. We also investigate the feasibility and the side effects of such attacks, and identify future directions for defense. Experiments on both synthetic and real-world data demonstrate the efficiency of the attack algorithm.

However even these reinforcement learning algorithms couldn't transfer what they'd learned about one task to acquiring a new task. In order to realize this achievement, DeepMind supercharged a reinforcement learning algorithm called A3C. In so-called actor-critic reinforcement learning, of which A3C is one variety, acting and learning are decoupled so that one neural network, the critic, evaluates the other, the actor. Together, they drive the learning process. This was already the state of the art, but DeepMind added a new off-policy correction algorithm called V-trace to the mix, which made the learning more efficient, and crucially, better able to achieve positive transfer between tasks.

STOCKHOLM--Artificial intelligence (AI) can kick butt in games such as Pong and Space Invaders, but it comes off like a common n00b when playing Ms. Pac-Man (pictured). Now, by making AI play six classic arcade games, researchers are closer to figuring out why thinking machines excel at some games and stink at others, they reported last month at the International Conference on Machine Learning here. The team developed a new system for visualizing how Atari-playing AIs operate. They chose Atari because the games are relatively simple and a frequent focus for researchers developing "reinforcement learning" algorithms, AIs that learn behaviors through trial and error. An AI "sees" the screen (as an input of ones and zeroes) and at first randomly responds with commands for "left," "right," "fire," and so on, slowly shaping its strategy as it receives points for certain actions.

Reinforcement learning methods have been used for learning dialogue policies from the experience of conversations. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the relatively small number of successful dialogues in early learning phase. Hindsight experience replay (HER) enables an agent to learn from failure, but the vanilla HER is inapplicable to dialogue domains due to dialogue goals being implicit (c.f., explicit goals in manipulation tasks). In this work, we develop two complex HER methods providing different trade-offs between complexity and performance. Experiments were conducted using a realistic user simulator. Results suggest that our HER methods perform better than standard and prioritized experience replay methods (as applied to deep Q-networks) in learning rate, and that our two complex HER methods can be combined to produce the best performance.

We develop a reinforcement learning based search assistant which can assist users through a set of actions and sequence of interactions to enable them realize their intent. Our approach caters to subjective search where the user is seeking digital assets such as images which is fundamentally different from the tasks which have objective and limited search modalities. Labeled conversational data is generally not available in such search tasks and training the agent through human interactions can be time consuming. We propose a stochastic virtual user which impersonates a real user and can be used to sample user behavior efficiently to train the agent which accelerates the bootstrapping of the agent. We develop A3C algorithm based context preserving architecture which enables the agent to provide contextual assistance to the user. We compare the A3C agent with Q-learning and evaluate its performance on average rewards and state values it obtains with the virtual user in validation episodes. Our experiments show that the agent learns to achieve higher rewards and better states.

Despite the successful application of machine learning (ML) in a wide range of domains, adaptability---the very property that makes machine learning desirable---can be exploited by adversaries to contaminate training and evade classification. In this paper, we investigate the feasibility of applying a specific class of machine learning algorithms, namely, reinforcement learning (RL) algorithms, for autonomous cyber defence in software-defined networking (SDN). In particular, we focus on how an RL agent reacts towards different forms of causative attacks that poison its training process, including indiscriminate and targeted, white-box and black-box attacks. In addition, we also study the impact of the attack timing, and explore potential countermeasures such as adversarial training.

Deep neuroevolution, that is evolutionary policy search methods based on deep neural networks, have recently emerged as a competitor to deep reinforcement learning algorithms due to their better parallelization capabilities. However, these methods still suffer from a far worse sample efficiency. In this paper we investigate whether a mechanism known as "importance mixing" can significantly improve their sample efficiency. We provide a didactic presentation of importance mixing and we explain how it can be extended to reuse more samples. Then, from an empirical comparison based on a simple benchmark, we show that, though it actually provides better sample efficiency, it is still far from the sample efficiency of deep reinforcement learning, though it is more stable.

Dr. Li's research focuses on the strategic search for knowledge, including human information seeking behavior, memory mechanisms in cognitive architectures, inference in semantic networks, and reinforcement learning of memory use. In his office, Dr. Li has a bunch of memory aids, which encourage neural pathway creation if toyed with during deep intellectual discussion. There's also a model NASA rocket, which he built himself.

Business leaders understand the advantage of using the power of artificial intelligence and machine learning to stay ahead of their competitors. However, understanding the power of AI is a lot different than actually successfully implementing it in companies. For example, in 2017, Gartner estimated that Big Data projects have a success rate of only 15%. While organizational factors may be a primary reason for this poor success rate, another reason for such a high failure rate could be due to a lack of AI / Machine Learning talent needed to successfully pursue these types of projects. Specifically, it's been shown that there is a lack of advanced machine learning talent among data professionals; less than 20% of surveyed data professionals said they were competent in such areas as Natural Language Processing (19%), Recommendation Engines (14%), Reinforcement Learning (6%), Adversarial Learning (4%) and Neural Networks – RNNs (15%).

It aims to promote and develop friendly AI in a way to benefit humanity as a whole. It aims to "freely collaborate" by making its patents and research open to the public.