In light of the inherently complex and dynamic nature of real-world environments, incorporating risk measures is crucial for the robustness evaluation of deep learning models. In this work, we propose a Risk-Averse Certification framework for Bayesian neural networks called RAC-BNN. Our method leverages sampling and optimisation to compute a sound approximation of the output set of a BNN, represented using a set of template polytopes. To enhance robustness evaluation, we integrate a coherent distortion risk measure--Conditional Value at Risk (CVaR)--into the certification framework, providing probabilistic guarantees based on empirical distributions obtained through sampling. We validate RAC-BNN on a range of regression and classification benchmarks and compare its performance with a state-of-the-art method. The results show that RAC-BNN effectively quantifies robustness under worst-performing risky scenarios, and achieves tighter certified bounds and higher efficiency in complex tasks.

Model-based reinforcement learning seeks to simultaneously learn the dynamics of an unknown stochastic environment and synthesise an optimal policy for acting in it. Ensuring the safety and robustness of sequential decisions made through a policy in such an environment is a key challenge for policies intended for safety-critical scenarios. In this work, we investigate two complementary problems: first, computing reach-avoid probabilities for iterative predictions made with dynamical models, with dynamics described by Bayesian neural network (BNN); second, synthesising control policies that are optimal with respect to a given reach-avoid specification (reaching a "target" state, while avoiding a set of "unsafe" states) and a learned BNN model. Our solution leverages interval propagation and backward recursion techniques to compute lower bounds for the probability that a policy's sequence of actions leads to satisfying the reach-avoid specification. Such computed lower bounds provide safety certification for the given policy and BNN model. We then introduce control synthesis algorithms to derive policies maximizing said lower bounds on the safety probability. We demonstrate the effectiveness of our method on a series of control benchmarks characterized by learned BNN dynamics models. On our most challenging benchmark, compared to purely data-driven policies the optimal synthesis algorithm is able to provide more than a four-fold increase in the number of certifiable states and more than a three-fold increase in the average guaranteed reach-avoid probability.

In this article, we summarise two of the invited talks from the AAAI Conference on Artificial Intelligence. We hear from Andrew Ng and Marta Kwiatkowska, who talked about data-centric AI and robust deep learning respectively. Andrew began with a definition of data-centric AI – "the discipline of systematically engineering the data used to build an AI system". AI systems tend to consist of two parts: data and code. The conventional approach for developing such systems, and one which many researchers take, is to download a dataset and then work on the code.

The Van Wijngaarden Awards 2021 are awarded to computer scientist Marta Kwiatkowska and mathematician Susan A. Murphy for the numerous and highly significant contributions they made to their respective research areas: preventing software faults and improving decision making in health. The five-yearly award is established by CWI, the national research institute for mathematics and computer science in the Netherlands, and is named after former CWI director Aad van Wijngaarden. The winners receive the prize during a festive soirée on 18 November in Amsterdam. Marta Kwiatkowska (University of Oxford) is a computer scientist who pioneered research on modelling, verification, and synthesis of probabilistic systems. She led the development of the highly influential PRISM probabilistic model checker, which is widely used for research and teaching and which has been downloaded over 80,000 times. In her research Kwiatkowska showed the relevance of PRISM by applying it in several areas, including ubiquitous computing, system biology, DNA computing, and most recently, safety for AI.