In this paper, we take a step toward narrowing the gap between model-based and model-free RL methods.

We propose practical deep Gaussian process models on Riemannian manifolds, similar in spirit to residual neural networks. With manifold-to-manifold hidden layers and an arbitrary last layer, they can model manifold-and scalar-valued functions, as well as vector fields. We target data inherently supported on manifolds, which is too complex for shallow Gaussian processes thereon. For example, while the latter perform well on high-altitude wind data, they struggle with the more intricate, nonstationary patterns at low altitudes. Our models significantly improve performance in these settings, enhancing prediction quality and uncertainty calibration, and remain robust to overfitting, reverting to shallow models when additional complexity is unneeded. We further showcase our models on Bayesian optimisation problems on manifolds, using stylised examples motivated by robotics, and obtain substantial improvements in later stages of the optimisation process. Finally, we show our models to have potential for speeding up inference for nonmanifold data, when, and if, it can be mapped to a proxy manifold well enough. Gaussian processes (GPs) are a widely adopted model class for learning functions within the Bayesian framework (Rasmussen and Williams, 2006). They offer accurate uncertainty estimates and perform well even when data is scarce. Consequently, GPs have found success in decisionmaking tasks, where well-calibrated uncertainty is key, including Bayesian optimisation (Snoek et al., 2012), active (Krause et al., 2008) and reinforcement (Kamthe and Deisenroth, 2018) learning. In recent years, substantial work went into developing the analogs of practical GP models on various non-Euclidean domains (Borovitskiy et al., 2021; 2023; 2020; Fichera et al., 2023).

Marc Peter Deisenroth is DeepMind Chair in Artificial Intelligence at the Department of Computer Science, University College London. Prior to this, he was a faculty member in the Department of Computing, Imperial College London. His research areas include data-efficient learning, probabilistic modeling, and autonomous decision making. Deisenroth was Program Chair of the European Workshop on Reinforcement Learning (EWRL) 2012 and Workshops Chair of Robotics Science and Systems (RSS) 2013. His research received Best Paper Awards at the International Conference on Robotics and Automation (ICRA) 2014 and the International Conference on Control, Automation and Systems (ICCAS) 2016.

As Gaussian processes are integrated into increasingly complex problem settings, analytic solutions to quantities of interest become scarcer and scarcer. Monte Carlo methods act as a convenient bridge for connecting intractable mathematical expressions with actionable estimates via sampling. Conventional approaches for simulating Gaussian process posteriors view samples as vectors drawn from marginal distributions over process values at a finite number of input location. This distribution-based characterization leads to generative strategies that scale cubically in the size of the desired random vector. These methods are, therefore, prohibitively expensive in cases where high-dimensional vectors - let alone continuous functions - are required. In this work, we investigate a different line of reasoning. Rather than focusing on distributions, we articulate Gaussian conditionals at the level of random variables. We show how this pathwise interpretation of conditioning gives rise to a general family of approximations that lend themselves to fast sampling from Gaussian process posteriors. We analyze these methods, along with the approximation errors they introduce, from first principles. We then complement this theory, by exploring the practical ramifications of pathwise conditioning in a various applied settings.

Reinforcement learning provides a framework for learning to control which actions to take towards completing a task through trial-and-error. In many applications observing interactions is costly, necessitating sample-efficient learning. In model-based reinforcement learning efficiency is improved by learning to simulate the world dynamics. The challenge is that model inaccuracies rapidly accumulate over planned trajectories. We introduce deep Gaussian processes where the depth of the compositions introduces model complexity while incorporating prior knowledge on the dynamics brings smoothness and structure. Our approach is able to sample a Bayesian posterior over trajectories. We demonstrate highly improved early sample-efficiency over competing methods. This is shown across a number of continuous control tasks, including the half-cheetah whose contact dynamics have previously posed an insurmountable problem for earlier sample-efficient Gaussian process based models.

Gaussian processes are a versatile framework for learning unknown functions in a manner that permits one to utilize prior information about their properties. Although many different Gaussian process models are readily available when the input space is Euclidean, the choice is much more limited for Gaussian processes whose input space is an undirected graph. In this work, we leverage the stochastic partial differential equation characterization of Mat\'{e}rn Gaussian processes - a widely-used model class in the Euclidean setting - to study their analog for undirected graphs. We show that the resulting Gaussian processes inherit various attractive properties of their Euclidean and Riemannian analogs and provide techniques that allow them to be trained using standard methods, such as inducing points. This enables graph Mat\'{e}rn Gaussian processes to be employed in mini-batch and non-conjugate settings, thereby making them more accessible to practitioners and easier to deploy within larger learning frameworks.

We focus on variational inference in dynamical systems where the discrete time transition function (or evolution rule) is modelled by a Gaussian process. The dominant approach so far has been to use a factorised posterior distribution, decoupling the transition function from the system states. This is not exact in general and can lead to an overconfident posterior over the transition function as well as an overestimation of the intrinsic stochasticity of the system (process noise). We propose a new method that addresses these issues and incurs no additional computational costs.

Machine Learning has huge potential and Imperial can be at the forefront of it, say experts as a new initiative launches today. Dr Marc Deisenroth, from the Department of Computing at Imperial College London, and colleagues from across the College are launching the Machine Learning Initiative today. Colin Smith caught up with Dr Deisenroth to discover what machine learning is and why it is so important in our daily lives. Industry leaders from companies such as Microsoft and Twitter, academics and students from Imperial and representatives from the UK's major funding bodies, will all be attending. The Initiative will bring together machine learning researchers from across the College and beyond to provide a collaborative environment for learning, teaching, and research in the field.

Machine Learning has huge potential and Imperial can be at the forefront of it, experts will hear at the new launch today. Dr Marc Deisenroth, from the Department of Computing at Imperial College London, and colleagues from across the College are launching the Machine Learning Initiative today. Colin Smith caught up with Dr Deisenroth to discover what machine learning is and why it is so important in our daily lives. Industry leaders from companies such as Microsoft and Twitter, academics and students from Imperial and representatives from the UK's major funding bodies, will all be attending. The Initiative will bring together machine learning researchers from across the College and beyond to provide a collaborative environment for learning, teaching, and research in the field.

Inside a secretive AI nonprofit backed by Elon Musk and other Silicon Valley figures, a handful of robots designed to help out in warehouses are gradually learning how to do useful household chores. OpenAI, which was created to do basic AI research, is reprogramming robots developed by Fetch Robotics, a company that supplies warehouse automation hardware. Researchers at OpenAI are equipping the robots with software that lets them train themselves through trial and error. The effort reflects a bet that innovations in software and machine learning, rather than breakthroughs in hardware, are the way to give robotics remarkable new capabilities. Fetch makes a range of robots for warehouses, including systems that follow workers around a building, carrying items dropped into a basket.