A. Relevance of Model Explanations in Real-World Contexts Complex estimation and decision-making tasks have traditionally been analyzed and judged by human experts. Hence, decisions have typically been able to be complemented with human-interpretable justifications, when needed, as experts can normally explain the line-of-thought that led to their own decision-making. However, in the past two decades, algorithmic decision-making has spread increasingly to many relevant societal contexts. Despite the notable enthusiasm for the potential benefit that this type of technology can bring, the underlying methods used are typically not inherently transparent, in the sense that they do not readily provide human-interpretable justifications for their decisions [1]. Moreover, in recent years there is a trend where the most successful algorithms, particularly in complex tasks like machine vision and natural language processing, tend to rely on highly complex models, which has led to a further increase in tension between accuracy and interpretability [2]. Relevant societal contexts where algorithmic decision systems have gained substantial traction include medical diagnosis and treatment [3], counter-terrorism [4], criminal justice [5], and risk assessments for credits and insurance [6]. In such impactful contexts, there is a legitimate need for providing human-interpretable explanations along with the estimations and decisions made. Indeed, lack of interpretability has become a barrier to the adoption of machine learning-based systems in many institutions and companies. Hence the value of complementing ML models with human-interpretable accounts of the statistical rationals behind their estimations, in a way that human decision-makers can more easily understand machine estimations, and even integrate their statistical rationals with qualitative information and human expert judgements.

In recent years, the use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise. Although these models can often bring substantial improvements in the accuracy and efficiency of organizations, many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways. Hence, these models are often regarded as black-boxes, in the sense that their internal mechanisms can be opaque to human audit. In real-world applications, particularly in domains where decisions can have a sensitive impact--e.g., criminal justice, estimating credit scores, insurance risk, health risks, etc.--model interpretability is desired. Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models. This survey reviews the most relevant and novel methods that form the state-of-the-art for addressing the particular problem of explaining individual instances in machine learning. It seeks to provide a succinct review that can guide data science and machine learning practitioners in the search for appropriate methods to their problem domain.

Interactions between several features sometimes play an important role in prediction tasks. But taking all the interactions into consideration will lead to an extremely heavy computational burden. For categorical features, the situation is more complicated since the input will be extremely high-dimensional and sparse if one-hot encoding is applied. Inspired by association rule mining, we propose a method that selects interactions of categorical features, called Random Intersection Chains. It uses random intersections to detect frequent patterns, then selects the most meaningful ones among them. At first a number of chains are generated, in which each node is the intersection of the previous node and a random chosen observation. The frequency of patterns in the tail nodes is estimated by maximum likelihood estimation, then the patterns with largest estimated frequency are selected. After that, their confidence is calculated by Bayes' theorem. The most confident patterns are finally returned by Random Intersection Chains. We show that if the number and length of chains are appropriately chosen, the patterns in the tail nodes are indeed the most frequent ones in the data set. We analyze the computation complexity of the proposed algorithm and prove the convergence of the estimators. The results of a series of experiments verify the efficiency and effectiveness of the algorithm.

Parametric statistical models are commonly used in many sub-fields of bioinformatics [1], [2]. For simplicity and computational concerns, bioinformatic scientists prefer to use differentiable and unconstrained statistical models than non-differentiable and constrained ones. For example, in pseudotime analysis (see section 3), in [3], the authors propose to regress gene expression on pseudotime using cubic B-spline so that an analytical solution is available. Other authors suggest to replace B-spline with a generalized linear model and a gradient-based method is applied to find maximum likelihood estimation [4]. In zero imputation problem, the authors construct a Gamma-Normal mixture model so that parameters can be estimated analytically [5]. In [6], the authors propose an unconstrained LASSO-type objective function and optimize it with a convex optimization algorithm. However, in real applications, it is common to impose constraints on parameters for interpretability. Besides, analytically solutions are not always available and the likelihood function is not differentiable or convex if discrete parameters are contained. Thus, constrained models without desirable mathematical properties can be more realistic and interpretable in many cases.

Active learning is a framework for supervised learning to improve the predictive performance by adaptively annotating a small number of samples. To realize efficient active learning, both an acquisition function that determines the next datum and a stopping criterion that determines when to stop learning should be considered. In this study, we propose a stopping criterion based on error stability, which guarantees that the change in generalization error upon adding a new sample is bounded by the annotation cost and can be applied to any Bayesian active learning. We demonstrate that the proposed criterion stops active learning at the appropriate timing for various learning models and real datasets.

Provision of some autonomous functions to an agricultural vehicle would lighten the job of the operator but in doing so, the accuracy should not be lost to still obtain an optimal yield. Autonomous navigation of an agricultural vehicle involves the control of different dynamic subsystems, such as the yaw angle dynamics and the longitudinal speed dynamics. In this study, a proportional-integral-derivative controller is used to control the longitudinal velocity of the tractor. For the control of the yaw angle dynamics, a proportional-derivative controller works in parallel with a type-2 fuzzy neural network. In such an arrangement, the former ensures the stability of the related subsystem, while the latter learns the system dynamics and becomes the leading controller. In this way, instead of modeling the interactions between the subsystems prior to the design of model-based control, we develop a control algorithm which learns the interactions online from the measured feedback error. In addition to the control of the stated subsystems, a kinematic controller is needed to correct the errors in both the x- and the y- axis for the trajectory tracking problem of the tractor. To demonstrate the real-time abilities of the proposed control scheme, an autonomous tractor is equipped with the use of reasonably priced sensors and actuators. Experimental results show the efficacy and efficiency of the proposed learning algorithm.

We present an efficient approach for doing approximate Bayesian inference when only a limited number of noisy likelihood evaluations can be obtained due to computational constraints, which is becoming increasingly common for applications of complex models. Our main methodological innovation is to model the log-likelihood function using a Gaussian process (GP) in a local fashion and apply this model to emulate the progression that an exact Metropolis-Hastings (MH) algorithm would take if it was applicable. New log-likelihood evaluation locations are selected using sequential experimental design strategies such that each MH accept/reject decision is done within a pre-specified error tolerance. The resulting approach is conceptually simple and sample-efficient as it takes full advantage of the GP model. It is also more robust to violations of GP modelling assumptions and better suited for the typical situation where the posterior is substantially more concentrated than the prior, compared with various existing inference methods based on global GP surrogate modelling. We discuss the probabilistic interpretations and central theoretical aspects of our approach, and we then demonstrate the benefits of the resulting algorithm in the context of likelihood-free inference for simulator-based statistical models.

Federated Bayesian learning offers a principled framework for the definition of collaborative training algorithms that are able to quantify epistemic uncertainty and to produce trustworthy decisions. Upon the completion of collaborative training, an agent may decide to exercise her legal "right to be forgotten", which calls for her contribution to the jointly trained model to be deleted and discarded. This paper studies federated learning and unlearning in a decentralized network within a Bayesian framework. It specifically develops federated variational inference (VI) solutions based on the decentralized solution of local free energy minimization problems within exponential-family models and on local gossip-driven communication. The proposed protocols are demonstrated to yield efficient unlearning mechanisms.

In this paper, we present a state-of-the-art reinforcement learning method for autonomous driving. Our approach employs temporal difference learning in a Bayesian framework to learn vehicle control signals from sensor data. The agent has access to images from a forward facing camera, which are preprocessed to generate semantic segmentation maps. We trained our system using both ground truth and estimated semantic segmentation input. Based on our observations from a large set of experiments, we conclude that training the system on ground truth input data leads to better performance than training the system on estimated input even if estimated input is used for evaluation. The system is trained and evaluated in a realistic simulated urban environment using the CARLA simulator. The simulator also contains a benchmark that allows for comparing to other systems and methods. The required training time of the system is shown to be lower and the performance on the benchmark superior to competing approaches.

Remaining Useful Life (RUL) estimation is the problem of inferring how long a certain industrial asset can be expected to operate within its defined specifications. Deploying successful RUL prediction methods in real-life applications is a prerequisite for the design of intelligent maintenance strategies with the potential of drastically reducing maintenance costs and machine downtimes. In light of their superior performance in a wide range of engineering fields, Machine Learning (ML) algorithms are natural candidates to tackle the challenges involved in the design of intelligent maintenance systems. In particular, given the potentially catastrophic consequences or substantial costs associated with maintenance decisions that are either too late or too early, it is desirable that ML algorithms provide uncertainty estimates alongside their predictions. However, standard data-driven methods used for uncertainty estimation in RUL problems do not scale well to large datasets or are not sufficiently expressive to model the high-dimensional mapping from raw sensor data to RUL estimates. In this work, we consider Deep Gaussian Processes (DGPs) as possible solutions to the aforementioned limitations. We perform a thorough evaluation and comparison of several variants of DGPs applied to RUL predictions. The performance of the algorithms is evaluated on the N-CMAPSS (New Commercial Modular Aero-Propulsion System Simulation) dataset from NASA for aircraft engines. The results show that the proposed methods are able to provide very accurate RUL predictions along with sensible uncertainty estimates, providing more reliable solutions for (safety-critical) real-life industrial applications.