Nonlinear optimal control problems are often solved with numerical methods that require knowledge of system's dynamics which may be difficult to infer, and that carry a large computational cost associated with iterative calculations. We present a novel neurobiologically inspired hierarchical learning framework, Reinforcement Learning Optimal Control, which operates on two levels of abstraction and utilises a reduced number of controllers to solve nonlinear systems with unknown dynamics in continuous state and action spaces. Our approach is inspired by research at two levels of abstraction: first, at the level of limb coordination human behaviour is explained by linear optimal feedback control theory. Second, in cognitive tasks involving learning symbolic level action selection, humans learn such problems using model-free and model-based reinforcement learning algorithms. We propose that combining these two levels of abstraction leads to a fast global solution of nonlinear control problems using reduced number of controllers. Our framework learns the local task dynamics from naive experience and forms locally optimal infinite horizon Linear Quadratic Regulators which produce continuous low-level control. A top-level reinforcement learner uses the controllers as actions and learns how to best combine them in state space while maximising a long-term reward. A single optimal control objective function drives high-level symbolic learning by providing training signals on desirability of each selected controller. We show that a small number of locally optimal linear controllers are able to solve global nonlinear control problems with unknown dynamics when combined with a reinforcement learner in this hierarchical framework. Our algorithm competes in terms of computational cost and solution quality with sophisticated control algorithms and we illustrate this with solutions to benchmark problems.

Google announced today that it has made energy produced by wind farms more viable using the artificial intelligence software of its London-based subsidiary DeepMind. By using DeepMind's machine learning algorithms to predict the wind output from the farms Google uses for its green energy initiatives, the company says it can now schedule set deliveries of energy output, which are more valuable to the grid than standard, non-time-based deliveries. According to Google, this software has improved the "value" of the wind energy these farms are providing by 20 percent over a baseline where no such time-based predictions are being performed. We don't know exactly what that value is in monetary terms or in terms of energy output. We also don't know where exactly this is being deployed, although Google works with wind farms largely in the Midwest, where some of its US data centers are located.

Apple will finally fix your iPhone, even if it has someone else's battery inside. Until now, iPhones with replacement battery that were installed by someone other than Apple employees have been excluded from being fixed. They were ineligible to be looked at by the Genius Bar, for instance, meaning that getting a battery replacement could mean passing up the chance for any other service work. That was the case even if the problem was with another component and not the battery, meaning that the entire phone would be banned from repairs just for having a third-party battery. We'll tell you what's true.

There's a good chance your smartphone contains tainted cobalt. The metal is a crucial ingredient in most of the lithium-ion batteries that power our devices, and 70% of it is mined in war-torn Democratic Republic of Congo (DRC), where children are often deployed to work in toxic environments. Though global brands like Apple and Samsung are keen to clean up their supply chain, DRC's dominance of the cobalt market makes the task difficult. These brands are also pressured by growing demand for cobalt, which Citigroup estimates will outstrip supply by 2023. That's because lithium-ion batteries also power electric cars, and every car battery needs as much as 1,000 times the amount of cobalt of a smartphone battery.

Time Series forecasting (univariate and multivariate) is a problem of high complexity due the different patterns that have to be detected in the input, ranging from high to low frequencies ones. In this paper we propose a new model for timeseries prediction that utilizes convolutional layers for feature extraction, a recurrent encoder and a linear autoregressive component. We motivate the model and we test and compare it against a baseline of widely used existing architectures for univariate and multivariate timeseries. The proposed model appears to outperform the baselines in almost every case of the multivariate timeseries datasets, in some cases even with 50% improvement which shows the strengths of such a hybrid architecture in complex timeseries.

We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables. The key success of this language and its compilation scheme is in its ability to automatically distinguish parameters the density function is discontinuous with respect to, while further providing runtime checks for boundary crossings. This enables the introduction of new inference engines that are able to exploit gradient information, while remaining efficient for models which are not everywhere differentiable. We demonstrate this ability by incorporating a discontinuous Hamiltonian Monte Carlo (DHMC) inference engine that is able to deliver automated and efficient inference for non-differentiable models. Our system is backed up by a mathematical formalism that ensures that any model expressed in this language has a density with measure zero discontinuities to maintain the validity of the inference engine.

Carbon-free technologies like renewable energy help combat climate change, but many of them have not reached their full potential. Consider wind power: over the past decade, wind farms have become an important source of carbon-free electricity as the cost of turbines has plummeted and adoption has surged. However, the variable nature of wind itself makes it an unpredictable energy source--less useful than one that can reliably deliver power at a set time. In search of a solution to this problem, last year, DeepMind and Google started applying machine learning algorithms to 700 megawatts of wind power capacity in the central United States. These wind farms--part of Google's global fleet of renewable energy projects--collectively generate as much electricity as is needed by a medium-sized city.

Graph processes model a number of important problems such as identifying the epicenter of an earthquake or predicting weather. In this paper, we propose a Graph Convolutional Recurrent Neural Network (GCRNN) architecture specifically tailored to deal with these problems. GCRNNs use convolutional filter banks to keep the number of trainable parameters independent of the size of the graph and of the time sequences considered. We also put forward Gated GCRNNs, a time-gated variation of GCRNNs akin to LSTMs. When compared with GNNs and another graph recurrent architecture in experiments using both synthetic and real-word data, GCRNNs significantly improve performance while using considerably less parameters.

This paper presents a novel data-driven approach for predicting the number of vegetation-related outages that occur in power distribution systems on a monthly basis. In order to develop an approach that is able to successfully fulfill this objective, there are two main challenges that ought to be addressed. The first challenge is to define the extent of the target area. An unsupervised machine learning approach is proposed to overcome this difficulty. The second challenge is to correctly identify the main causes of vegetation-related outages and to thoroughly investigate their nature. In this paper, these outages are categorized into two main groups: growth-related and weather-related outages, and two types of models, namely time series and non-linear machine learning regression models are proposed to conduct the prediction tasks, respectively. Moreover, various features that can explain the variability in vegetation-related outages are engineered and employed. Actual outage data, obtained from a major utility in the U.S., in addition to different types of weather and geographical data are utilized to build the proposed approach. Finally, by utilizing various time series models and machine learning methods, a comprehensive case study is carried out to demonstrate how the proposed approach can be used to successfully predict the number of vegetation-related outages and to help decision-makers to detect vulnerable zones in their systems.

Besides applications on problems like digital image and speech recognition, machine learning (ML) methods are also used to predict complicated patterns in earthquake activity, say researchers. It can be used to hone predictions of seismic activity, identify earthquake centres, characterise different types of seismic waves and distinguish seismic activity from other kinds of ground "noise", according to a team of seismologists. More seismologists are using the method, driven by "the increasing size of seismic data sets, improvements in computational power, new algorithms and architecture and the availability of easy-to-use open source machine learning frameworks," said the team, including Karianne Bergen from the Harvard University in the USA, in a paper published in the journal Seismological Research Letters. These methods, called deep neural networks, can explore the complex relationships between input data and their predicted output. For instance, one kind of deep neural network can be used to develop ground motion models for natural and induced earthquakes in Oklahoma, Kansas and Texas.