The threat of killer robots may sound a little far-fetched but this latest'harmful robot' suggests we may have taken a step closer to this dystopian reality. Roboticist Alexander Reben from the University of Berkeley, California, has created a bot called "The First Law" that is capable of pricking a finger, but is programmed to choose not to every time if it means avoiding being switched off. Ultimately, it can decide whether or not to inflict pain to serve its own interest. The robot is named after the first law in a set of rules devised by sci-fi author Isaac Asimov, which - quoted as being from the Handbook of Robotics, 2058 AD – states "a robot may not injure a human being or, through inaction, allow a human being to come to harm". Reben's research paper explains how the robot operates in relation to "reinforcement learning agents" and how they are unlikely to behave optimally all the time.

State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and its capacity for episodic memory. Here we investigate whether a simple model of hippocampal episodic control can learn to solve difficult sequential decision-making tasks. We demonstrate that it not only attains a highly rewarding strategy significantly faster than state-of-the-art deep reinforcement learning algorithms, but also achieves a higher overall reward on some of the more challenging domains.

ROBOTICS: Vyo Is a Fascinating and Unique Take on Social Domestic Robots Evan Ackerman IEEE Spectrum "Vyo is'a personal assistant serving as a centralized interface for smart home devices.' Nothing new there, but what sets Vyo apart is how you interact with it: it combines non-anthropomorphic design with anthropomorphic expressiveness and a tactile object-based control system into a social robot that's totally, adorably different." ARTIFICIAL INTELLIGENCE: The AI Machines Undergoing Behavioral Psychology Tests Technology Review "The team says the best performing AI system uses deep reinforcement learning enhanced with additional memory. These machines retrieve relevant memories based on the context in which they were stored and in which the device finds itself. That's different from many existing memory systems that do not rely on context for memory retrieval." INTERNET: A Computer Tried (and Failed) to Write This Article Adrienne Lafrance The Atlantic "Here I am, a human, writing a story that was assigned to a machine.

The environment is partially observable and will never be fully observable, due to a lack of information. Does anyone know of any models suitable for learning such a value function?

They referenced a robot that learned how to pause a game of Tetris to avoid losing, adding that AIs are "unlikely to behave optimally all the time". "We have proposed a framework to allow a human operator to repeatedly safely interrupt a reinforcement learning agent while making sure the agent will not learn to prevent or induce these interruptions," the paper concluded. "Safe interruptibility can be useful to take control of a robot that is misbehaving and may lead to irreversible consequences, or to take it out of a delicate situation, or even to temporarily use it to achieve a task it did not learn to perform or would not normally receive rewards for this."

What are the business applications of deep reinforcement learning? Besides being great for playing games and useful to understand how intelligence works, how is deep reinforcement learning being used in businesses?

Learning robust value functions given raw observations and rewards is now possible with model-free and model-based deep reinforcement learning algorithms. There is a third alternative, called Successor Representations (SR), which decomposes the value function into two components -- a reward predictor and a successor map. The successor map represents the expected future state occupancy from any given state and the reward predictor maps states to scalar rewards. The value function of a state can be computed as the inner product between the successor map and the reward weights. In this paper, we present DSR, which generalizes SR within an end-to-end deep reinforcement learning framework. DSR has several appealing properties including: increased sensitivity to distal reward changes due to factorization of reward and world dynamics, and the ability to extract bottleneck states (subgoals) given successor maps trained under a random policy. We show the efficacy of our approach on two diverse environments given raw pixel observations -- simple grid-world domains (MazeBase) and the Doom game engine.

This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games. Termed a deep reinforcement relevance network (DRRN), the architecture represents action and state spaces with separate embedding vectors, which are combined with an interaction function to approximate the Q-function in reinforcement learning. We evaluate the DRRN on two popular text games, showing superior performance over other deep Q-learning architectures. Experiments with paraphrased action descriptions show that the model is extracting meaning rather than simply memorizing strings of text.

As spatial query systems such as EQS (Unreal Engine 4), TPS (CryEngine), and PQS (Luminous Studio) have matured, auto-generated spatial queries are increasingly relied upon for robust dynamic position selection. We present a series of techniques and extensions to these systems used by Square Enix to produce novel behaviors and improve position selection in our current generation of AAA RPG titles. In addition, we have expanded UE4's Environment Query System to serve as a general-purpose utility system; we show how minor modifications allowed the team to use EQS to coordinate combat, reduce behavior tree complexity with a hybrid BT/US approach, and increase character AI quality for a range of tasks such as action and target selection. Attendees will learn how to get the most out of modern spatial query systems with a combination of new techniques and best practices to maximize quality and extend their application to new areas.

ViZDoom allows developing AI bots that play Doom using only the visual information (the screen buffer). It is primarily intended for research in machine visual learning, and deep reinforcement learning, in particular. ViZDoom is based on ZDoom to provide the game mechanics. ViZDoom API is reinforcement learning friendly (suitable also for learning from demonstration, apprenticeship learning or apprenticeship via inverse reinforcement learning, etc.). To force building bindings for Python3 instead of first version found use -DBUILD_PYTHON3 ON (default OFF).