Learning to reinforcement learn by Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, Matt Botvinick In recent years deep reinforcement learning (RL) systems have attained superhuman performance in a number of challenging task domains. However, a major limitation of such applications is their demand for massive amounts of training data. A critical present objective is thus to develop deep RL methods that can adapt rapidly to new tasks. In the present work we introduce a novel approach to this challenge, which we refer to as deep meta-reinforcement learning. Previous work has shown that recurrent networks can support meta-learning in a fully supervised context.

DAVOS, Switzerland: Open markets and global trade have been blamed for job losses over the last decade, but global CEOs say the real culprits are increasingly machines. And while business leaders gathered at the annual World Economic Forum (WEF) in Davos relish the productivity gains technology can bring, they warned this week that the collateral damage to jobs needs to be addressed more seriously. From taxi drivers to health care professionals, technologies such as robotics, driverless cars, artificial intelligence and 3-D printing mean more and more types of jobs are at risk. Adidas, for example, aims to use 3-D printing in the manufacture of some running shoes. "Jobs will be lost, jobs will evolve and this revolution is going to be ageless, it is going to be classless and it is going to affect everyone," said Meg Whitman, chief executive of Hewlett Packard Enterprise.

Part two of our series "A day in the life of a city" looks at the ways in which offices are changing and how cities are coping with the ever-growing problem of pollution. The morning rush hour is over and, if you live in a city in the developed world, you are likely to be settling down at your desk for the next eight or so hours. However, the office block and skyscraper, which have been part of our urban landscape since the end of the 19th Century, may also soon become surplus to requirements. Urban architect Anthony Townsend thinks cities need more creative approaches to how we work and is keen to reclaim the streets by creating pop-up workspaces in the parks and plazas of the financial district in New York. "Before the New York Stock Exchange, traders met under a tree on Wall Street to buy and sell shares. It is only in the last 50 years that we have taken that creative energy and sucked it up into office buildings and separated it from public space," he said.

A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be extended to probability measures. It can be viewed as a generalization of the original "feature map" common to support vector machines (SVMs) and other kernel methods. While initially closely associated with the latter, it has meanwhile found application in fields ranging from kernel machines and probabilistic modeling to statistical inference, causal discovery, and deep learning. The goal of this survey is to give a comprehensive review of existing work and recent advances in this research area, and to discuss the most challenging issues and open problems that could lead to new research directions. The survey begins with a brief introduction to the RKHS and positive definite kernels which forms the backbone of this survey, followed by a thorough discussion of the Hilbert space embedding of marginal distributions, theoretical guarantees, and a review of its applications. The embedding of distributions enables us to apply RKHS methods to probability measures which prompts a wide range of applications such as kernel two-sample testing, independent testing, and learning on distributional data. Next, we discuss the Hilbert space embedding for conditional distributions, give theoretical insights, and review some applications. The conditional mean embedding enables us to perform sum, product, and Bayes' rules---which are ubiquitous in graphical model, probabilistic inference, and reinforcement learning---in a non-parametric way. We then discuss relationships between this framework and other related areas. Lastly, we give some suggestions on future research directions.

Many important real-world datasets come in the form of graphs or networks: social networks, knowledge graphs, protein-interaction networks, the World Wide Web, etc. (just to name a few). Yet, until recently, very little attention has been devoted to the generalization of neural network models to such structured datasets. In the last couple of years, a number of papers re-visited this problem of generalizing neural networks to work on arbitrarily structured graphs (Bruna et al., ICLR 2014; Henaff et al., 2015; Duvenaud et al., NIPS 2015; Li et al., ICLR 2016; Defferrard et al., NIPS 2016; Kipf & Welling, 2016), some of them now achieving very promising results in domains that have previously been dominated by, e.g., kernel-based methods, graph-based regularization techniques and others. In this post, I will give a brief overview of recent developments in this field and point out strengths and drawbacks of various approaches. I wrote a short comment on Ferenc's review here (at the very end of this post).

This book presents a detailed review of the state of the art in deep learning approaches for semantic object detection and segmentation in medical image computing, microscopic image analysis, and large-scale radiology database mining. A particular focus is placed on the application of convolutional neural networks, with the theory supported by practical examples. This book describes a range of different methods that make use of deep learning for object or landmark detection tasks in 2D and 3D medical imaging; examines a varied selection of techniques for semantic segmentation using deep learning principles in medical imaging; introduces a novel approach to interleaved text and image deep mining on a large-scale radiology image database. Dr. Yang is the founder of the Biomedical Image Computing and Imaging Informatics (BICI2) lab (http://www.bme.ufl.edu/labs/yang/). His major research interests are focus on biomedical image analysis and imaging informatics, computer vision, biomedical informatics and machine learning.

DAVOS, Switzerland (Reuters) - Open markets and global trade have been blamed for job losses over the last decade, but global CEOs say the real culprits are increasingly machines. And while business leaders gathered at the annual World Economic Forum (WEF) in Davos relish the productivity gains technology can bring, they warned this week that the collateral damage to jobs needs to be addressed more seriously. From taxi drivers to healthcare professionals, technologies such as robotics, driverless cars, artificial intelligence and 3-D printing mean more and more types of jobs are at risk. Adidas, for example, aims to use 3-D printing in the manufacture of some running shoes. "Jobs will be lost, jobs will evolve and this revolution is going to be ageless, it's going to be classless and it's going to affect everyone," said Meg Whitman, chief executive of Hewlett Packard Enterprise.

Editor's Note: An update to this article has been posted here on 7/14/04. As the hype of past decades fades, the current heir to the artificial intelligence legacy may well be ontologies. Evolving from semantic network notions, modern ontologies are proving quite useful. And they are doing so without relying on the jumble of rule-based techniques common in earlier knowledge representation efforts. These structured depictions or models of known (and accepted) facts are being built today to make a number of applications more capable of handling complex and disparate information. They appear most effective when the semantic distinctions that humans take for granted are crucial to the application's purpose.

The game of poker has been identified as a beneficial domain for current AI research because of the properties it possesses such as the need to deal with hidden information and stochasticity. The identification of poker as a useful research domain has inevitably resulted in increased attention from academic researchers who have pursued many separate avenues of research in the area of computer poker. The poker domain has often featured in previous review papers that focus on games in general, however a comprehensive review paper with a specific focus on computer poker has so far been lacking in the literature. In this paper, we present a review of recent algorithms and approaches in the area of computer poker, along with a survey of the autonomous poker agents that have resulted from this research. We begin with the first serious attempts to create strong computerised poker players by constructing knowledge-based and simulation-based systems.

Artificial Intelligence (which I'll refer to hereafter by its nickname, "AI") is the subfield of Computer Science devoted to developing programs that enable computers to display behavior that can (broadly) be characterized as intelligent.[1] Most research in AI is devoted to fairly narrow applications, such as planning or speech-to-speech translation in limited, well defined task domains. But substantial interest remains in the long-range goal of building generally intelligent, autonomous agents,[2] even if the goal of fully human-like intelligence is elusive and is seldom pursued explicitly and as such. Throughout its relatively short history, AI has been heavily influenced by logical ideas. AI has drawn on many research methodologies: the value and relative importance of logical formalisms is questioned by some leading practitioners, and has been debated in the literature from time to time.[3]