Continual Reinforcement Learning in 3D Non-stationary Environments Machine Learning

High-dimensional always-changing environments constitute a hard challenge for current reinforcement learning techniques. Artificial agents, nowadays, are often trained off-line in very static and controlled conditions in simulation such that training observations can be thought as sampled i.i.d. from the entire observations space. However, in real world settings, the environment is often non-stationary and subject to unpredictable, frequent changes. In this paper we propose and openly release CRLMaze, a new benchmark for learning continually through reinforcement in a complex 3D non-stationary task based on ViZDoom and subject to several environmental changes. Then, we introduce an end-to-end model-free continual reinforcement learning strategy showing competitive results with respect to four different baselines and not requiring any access to additional supervised signals, previously encountered environmental conditions or observations.

Q&A: Travel startup paves way for industry consolidation (Includes interview)


In May 2019 Google announced the consolidation of all its travel features. Google Maps, Trips, Hotels and Flights will combine to make one Google Travel, easing the process for vacation planning. Travel startup VacationRenter, which launched last year, pioneered this model for vacation rentals, based on an artificial intelligence driven platform. According to VacationRenter's newly appointed COO, ex-Googler Marco del Rosario, both Google Travel and VacationRenter are early adopters of a pivotal strategy for today's travel technology: consolidation. Digital Journal: How has the world of travel changed in recent years?

Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay Machine Learning

Despite huge success, deep networks are unable to learn effectively in sequential multitask learning settings as they forget the past learned tasks after learning new tasks. Inspired from complementary learning systems theory, we address this challenge by learning a generative model that couples the current task to the past learned tasks through a discriminative embedding space. We learn an abstract level generative distribution in the embedding that allows the generation of data points to represent the experience. We sample from this distribution and utilize experience replay to avoid forgetting and simultaneously accumulate new knowledge to the abstract distribution in order to couple the current task with past experience. We demonstrate theoretically and empirically that our framework learns a distribution in the embedding that is shared across all task and as a result tackles catastrophic forgetting.

Attention-Based Structural-Plasticity Machine Learning

Catastrophic forgetting/interference is a critical problem for lifelong learning machines, which impedes the agents from maintaining their previously learned knowledge while learning new tasks. Neural networks, in particular, suffer plenty from the catastrophic forgetting phenomenon. Recently there has been several efforts towards overcoming catastrophic forgetting in neural networks. Here, we propose a biologically inspired method toward overcoming catastrophic forgetting. Specifically, we define an attention-based selective plasticity of synapses based on the cholinergic neuromodulatory system in the brain. We define synaptic importance parameters in addition to synaptic weights and then use Hebbian learning in parallel with backpropagation algorithm to learn synaptic importances in an online and seamless manner. We test our proposed method on benchmark tasks including the Permuted MNIST and the Split MNIST problems and show competitive performance compared to the state-of-the-art methods.

Generative Memory for Lifelong Reinforcement Learning Artificial Intelligence

Our research is focused on understanding and applying biological memory transfers to new AI systems that can fundamentally improve their performance, throughout their fielded lifetime experience. We leverage current understanding of biological memory transfer to arrive at AI algorithms for memory consolidation and replay. In this paper, we propose the use of generative memory that can be recalled in batch samples to train a multi-task agent in a pseudo-rehearsal manner. We show results motivating the need for task-agnostic separation of latent space for the generative memory to address issues of catastrophic forgetting in lifelong learning.

Policy Consolidation for Continual Reinforcement Learning Machine Learning

We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries, and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidation} model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent's policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.

CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison Artificial Intelligence

Large, labeled datasets have driven deep learning methods to achieve expert-level performance on a variety of medical imaging tasks. We present CheXpert, a large dataset that contains 224,316 chest radiographs of 65,240 patients. We design a labeler to automatically detect the presence of 14 observations in radiology reports, capturing uncertainties inherent in radiograph interpretation. We investigate different approaches to using the uncertainty labels for training convolutional neural networks that output the probability of these observations given the available frontal and lateral radiographs. On a validation set of 200 chest radiographic studies which were manually annotated by 3 board-certified radiologists, we find that different uncertainty approaches are useful for different pathologies. We then evaluate our best model on a test set composed of 500 chest radiographic studies annotated by a consensus of 5 board-certified radiologists, and compare the performance of our model to that of 3 additional radiologists in the detection of 5 selected pathologies. On Cardiomegaly, Edema, and Pleural Effusion, the model ROC and PR curves lie above all 3 radiologist operating points. We release the dataset to the public as a standard benchmark to evaluate performance of chest radiograph interpretation models. The dataset is freely available at .

7 security trends to watch in 2019: More AI, consolidation and regionalization


In reflecting on 2018 from a security perspective, some of the major themes, as I've written previously, have been about new AI security tools, industry consolidation and the blurring of lines between physical and cyber security. GRDP, the California Consumer Privacy Act and Facebook's seemingly never-ending scandals related to consumer privacy have also raised regulatory and public awareness of data privacy as a key issue and concern. These discussions will continue into next year and beyond, and there a number of other big trends that are likely to dominate the security industry in 2019. Here are seven that I believe we'll be looking at in the new year and for some time after that: As the number and range of threats continue to grow, it's clear that only AI can counter them. That's why we saw some big companies announce AI-based solutions in 2018, including Palo Alto Networks' behavioral analytics solution Magnifier and Alphabet's Chronicle.

Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation

Neural Information Processing Systems

Generating long and coherent reports to describe medical images poses challenges to bridging visual patterns with informative human linguistic descriptions. We propose a novel Hybrid Retrieval-Generation Reinforced Agent (HRGR-Agent) which reconciles traditional retrieval-based approaches populated with human prior knowledge, with modern learning-based approaches to achieve structured, robust, and diverse report generation. HRGR-Agent employs a hierarchical decision-making procedure. For each sentence, a high-level retrieval policy module chooses to either retrieve a template sentence from an off-the-shelf template database, or invoke a low-level generation module to generate a new sentence. HRGR-Agent is updated via reinforcement learning, guided by sentence-level and word-level rewards. Experiments show that our approach achieves the state-of-the-art results on two medical report datasets, generating well-balanced structured sentences with robust coverage of heterogeneous medical report contents. In addition, our model achieves the highest detection precision of medical abnormality terminologies, and improved human evaluation performance.

Breathing through your nose BOOSTS your memory

Daily Mail - Science & tech

Breathing through your nose boosts your memory, according to new research. It improves the transfer of the events we experience in our daily lives to our long-term memory bank, say scientists. In the study, participants exposed to certain odours were better at recalling them if their mouths had been taped over. The findings add to a growing body of evidence that inhaling through the nose rather than the mouth enhances cognition. Intriguingly, recent studies have also suggested a fading sense of smell is one of the first signs of Alzheimer's disease.