Deep reinforcement learning has learned to play many games well, but failed on others. To better characterize the modes and reasons of failure of deep reinforcement learners, we test the widely used Asynchronous Actor-Critic (A2C) algorithm on four deceptive games, which are specially designed to provide challenges to game-playing agents. These games are implemented in the General Video Game AI framework, which allows us to compare the behavior of reinforcement learning-based agents with planning agents based on tree search. We find that several of these games reliably deceive deep reinforcement learners, and that the resulting behavior highlights the shortcomings of the learning algorithm. The particular ways in which agents fail differ from how planning-based agents fail, further illuminating the character of these algorithms. We propose an initial typology of deceptions which could help us better understand pitfalls and failure modes of (deep) reinforcement learning. Introduction In reinforcement learning (RL) (Sutton and Barto 1998) an agent is tasked with learning a policy that maximizes expected reward based only on its interactions with the environment. In general, there is no guarantee that any such procedure will lead to an optimal policy; while convergence proofs exist, they only apply to a tiny and rather uninteresting class of environments. Reinforcement learning still performs well for a wide range of scenarios not covered by those convergence proofs. However, while recent successes in game-playing with deep reinforcement learning (Justesen et al. 2017) have led to a high degree of confidence in the deep RL approach, there are still scenarios or games where deep RL fails. Some oft-mentioned reasons why RL algorithms fail are partial observability and long time spans between actions and rewards. But are there other causes?

Constrained counting is a fundamental problem in artificial intelligence. A promising new algebraic approach to constrained counting makes use of tensor networks, following a reduction from constrained counting to the problem of tensor-network contraction. Contracting a tensor network efficiently requires determining an efficient order to contract the tensors inside the network, which is itself a difficult problem. In this work, we apply graph decompositions to find contraction orders for tensor networks. We prove that finding an efficient contraction order for a tensor network is equivalent to the well-known problem of finding an optimal carving decomposition. Thus memory-optimal contraction orders for planar tensor networks can be found in cubic time. We show that tree decompositions can be used both to find carving decompositions and to factor tensor networks with high-rank, structured tensors. We implement these algorithms on top of state-of-the-art solvers for tree decompositions and show empirically that the resulting weighted model counter is quite effective and useful as part of a portfolio of counters.

In recent years, with the trend of applying deep learning (DL) in high performance scientific computing, the unique characteristics of emerging DL workloads in HPC raise great challenges in designing, implementing HPC AI systems. The community needs a new yard stick for evaluating the future HPC systems. In this paper, we propose HPC AI500 --- a benchmark suite for evaluating HPC systems that running scientific DL workloads. Covering the most representative scientific fields, each workload from HPC AI500 is based on real-world scientific DL applications. Currently, we choose 14 scientific DL benchmarks from perspectives of application scenarios, data sets, and software stack. We propose a set of metrics for comprehensively evaluating the HPC AI systems, considering both accuracy, performance as well as power and cost. We provide a scalable reference implementation of HPC AI500. HPC AI500 is a part of the open-source AIBench project, the specification and source code are publicly available from \url{http://www.benchcouncil.org/AIBench/index.html}.

Guided by artificial intelligence and powered by a robotic platform, a system developed by MIT researchers moves a step closer to automating the production of small molecules that could be used in medicine, solar energy, and polymer chemistry. The system, described in the August 8 issue of Science, could free up bench chemists from a variety of routine and time-consuming tasks, and may suggest possibilities for how to make new molecular compounds, according to the study co-leaders Klavs F. Jensen, the Warren K. Lewis Professor of Chemical Engineering, and Timothy F. Jamison, the Robert R. Taylor Professor of Chemistry and associate provost at MIT. The technology "has the promise to help people cut out all the tedious parts of molecule building," including looking up potential reaction pathways and building the components of a molecular assembly line each time a new molecule is produced, says Jensen. "And as a chemist, it may give you inspirations for new reactions that you hadn't thought about before," he adds. The new system combines three main steps.

IJCAI, the 28th International Joint Conference on Artificial Intelligence, is happening from the 10th to 16th August in Macao, China. We'll be posting updates throughout the week thanks to the AIhub Ambassadors on the ground. You can follow the tweets at #IJCAI2019 or below.

The first two days at IJCAI (International Joint Conference on Artificial Intelligence) in Macau were focussed on workshops and tutorials. First demo at #ijcai2019: grad students are still superior to robots at reception desks pic.twitter.com/49beH9qBcl

Current research is aligned with the need of rescue workers but robustness and ease of use remain significant barriers to adoption, NCCR Robotics researchers find after reviewing the field and consulting with field operators. Robots for search and rescue are developing at an impressive pace, but they must become more robust and easier to use in order to be widely adopted, and researchers in the field must devote more effort to these aspects in the future. This is one of the main findings by a group of NCCR Robotics researchers who focus on search-and-rescue applications. After reviewing the recent developments in technology and interviewing rescue workers, they have found that the work by the robotics research community is well aligned with the needs of those who work in the field. Consequently, although current adoption of state-of-the-art robotics in disaster response is still limited, it is expected to grow quickly in the future.

Using machine learning to accelerate ecological research Using machine learning to accelerate ecological research Share Pushmeet Kohli * External authors The Serengeti is one of the last remaining sites in the world that hosts an intact community of large mammals. These animals roam over vast swaths of land, some migrating thousands of miles across multiple countries following seasonal rainfall. As human encroachment around the park becomes more intense, these species are forced to alter their behaviours in order to survive. Increasing agriculture, poaching, and climate abnormalities contribute to changes in animal behaviours and population dynamics, but these changes have occurred at spatial and temporal scales which are difficult to monitor using traditional research methods. There is a great urgency to understand how these animal communities function as human pressures grow, both in order to understand the dynamics of these last pristine ecosystems, and to formulate effective management plans to conserve and protect the integrity of this unique biodiversity hotspot.

Section 8: Various research approaches and methodologies implemented in the research document.

Could a computer, at a glance, tell the difference between a joyful image and a depressing one? Could it distinguish, in a few milliseconds, a romantic comedy from a horror film? Yes, and so can your brain, according to research published this week by CU Boulder neuroscientists. "Machine learning technology is getting really good at recognizing the content of images--of deciphering what kind of object it is," said senior author Tor Wager, who worked on the study while a professor of psychology and neuroscience at CU Boulder. "We wanted to ask: Could it do the same with emotions? Part machine-learning innovation, part human brain-imaging study, the paper, published Wednesday in the journal Science Advances, marks an important step forward in the application of "neural networks"--computer systems modeled after the human brain--to the study of emotion. It also sheds a new, different light on how and where images are represented in the human brain, suggesting that what we see--even ...