Inter-domain Gaussian processes (GPs) allow for high flexibility and low computational cost when performing approximate inference in GP models. They are particularly suitable for modeling data exhibiting global structure but are limited to stationary covariance functions and thus fail to model non-stationary data effectively. We propose Inter-domain Deep Gaussian Processes, an extension of inter-domain shallow GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs), and demonstrate how to leverage existing approximate inference methods to perform simple and scalable approximate inference using inter-domain features in DGPs. We assess the performance of our method on a range of regression tasks and demonstrate that it outperforms inter-domain shallow GPs and conventional DGPs on challenging large-scale real-world datasets exhibiting both global structure as well as a high-degree of non-stationarity.

Savage (1972) lays down the foundation of Bayesian decision theory, but asserts that it is not applicable in big worlds where the environment is complex. Using the theory of finite automaton to model belief formation, this paper studies the characteristics of optimal learning behavior in small and big worlds, where the complexity of the environment is low and high, respectively, relative to the cognitive ability of the decision maker. Confirming Savage's claim, optimal learning behavior is closed to Bayesian in small worlds but significantly different in big worlds. In addition, I show that in big worlds, the optimal learning behavior could exhibit a wide range of well-documented non-Bayesian learning behavior, including the use of heuristic, correlation neglect, persistent over-confidence, inattentive learning, and other behaviors of model simplification or misspecification. These results establish a clear and testable relationship between the prominence of non-Bayesian learning behavior, complexity and cognitive ability.

This paper studies the source (event) localization problem in decentralized wireless sensor networks (WSNs) under the fault model without knowing the sensor parameters. Event localizations have many applications such as localizing intruders, Wifi hotspots and users, and faults in power systems. Previous studies assume the true knowledge (or good estimates) of sensor parameters (e.g., fault model probability or Region of Influence (ROI) of the source) for source localization. However, we propose two methods to estimate the source location in this paper under the fault model: hitting set approach and feature selection method, which only utilize the noisy data set at the fusion center for estimation of the source location without knowing the sensor parameters. The proposed methods have been shown to localize the source effectively. We also study the lower bound on the sample complexity requirement for hitting set method. These methods have also been extended for multiple sources localizations. In addition, we modify the proposed feature selection approach to use maximum likelihood. Finally, extensive simulations are carried out for different settings (i.e., the number of sensor nodes and sample complexity) to validate our proposed methods in comparison to centroid, maximum likelihood, FTML, SNAP estimators.

In this paper, we propose and prove the theorem regarding the stability of attributes in a decision system. Based on the theorem, we propose the LRA framework for accelerating rough set algorithms. It is a general-purpose framework which can be applied to almost all rough set methods significantly . Theoretical analysis guarantees high efficiency. Note that the enhancement of efficiency will not lead to any decrease of the classification accuracy. Besides, we provide a simpler prove for the positive approximation acceleration framework.

In recent years, bankruptcy forecasting has gained lot of attention from researchers as well as practitioners in the field of financial risk management. For bankruptcy prediction, various approaches proposed in the past and currently in practice relies on accounting ratios and using statistical modeling or machine learning methods. These models have had varying degrees of successes. Models such as Linear Discriminant Analysis or Artificial Neural Network employ discriminative classification techniques. They lack explicit provision to include prior expert knowledge. In this paper, we propose another route of generative modeling using Expert Bayesian framework. The biggest advantage of the proposed framework is an explicit inclusion of expert judgment in the modeling process. Also the proposed methodology provides a way to quantify uncertainty in prediction. As a result the model built using Bayesian framework is highly flexible, interpretable and intuitive in nature. The proposed approach is well suited for highly regulated or safety critical applications such as in finance or in medical diagnosis. In such cases accuracy in the prediction is not the only concern for decision makers. Decision makers and other stakeholders are also interested in uncertainty in the prediction as well as interpretability of the model. We empirically demonstrate these benefits of proposed framework on real world dataset using Stan, a probabilistic programming language. We found that the proposed model is either comparable or superior to the other existing methods. Also resulting model has much less False Positive Rate compared to many existing state of the art methods. The corresponding R code for the experiments is available at Github repository.

Gaussian Process (GPs) models are a rich distribution over functions with inductive biases controlled by a kernel function. Learning occurs through the optimisation of kernel hyperparameters using the marginal likelihood as the objective. This classical approach known as Type-II maximum likelihood (ML-II) yields point estimates of the hyperparameters, and continues to be the default method for training GPs. However, this approach risks underestimating predictive uncertainty and is prone to overfitting especially when there are many hyperparameters. Furthermore, gradient based optimisation makes ML-II point estimates highly susceptible to the presence of local minima. This work presents an alternative learning procedure where the hyperparameters of the kernel function are marginalised using Nested Sampling (NS), a technique that is well suited to sample from complex, multi-modal distributions. We focus on regression tasks with the spectral mixture (SM) class of kernels and find that a principled approach to quantifying model uncertainty leads to substantial gains in predictive performance across a range of synthetic and benchmark data sets. In this context, nested sampling is also found to offer a speed advantage over Hamiltonian Monte Carlo (HMC), widely considered to be the gold-standard in MCMC based inference.

Normalizing flows and variational autoencoders are powerful generative models that can represent complicated density functions. However, they both impose constraints on the models: Normalizing flows use bijective transformations to model densities whereas VAEs learn stochastic transformations that are non-invertible and thus typically do not provide tractable estimates of the marginal likelihood. In this paper, we introduce SurVAE Flows: A modular framework of composable transformations that encompasses VAEs and normalizing flows. SurVAE Flows bridge the gap between normalizing flows and VAEs with surjective transformations, wherein the transformations are deterministic in one direction -- thereby allowing exact likelihood computation, and stochastic in the reverse direction -- hence providing a lower bound on the corresponding likelihood. We show that several recently proposed methods, including dequantization and augmented normalizing flows, can be expressed as SurVAE Flows. Finally, we introduce common operations such as the max value, the absolute value, sorting and stochastic permutation as composable layers in SurVAE Flows.

Brittle optimization has been observed to adversely impact model likelihoods for regression and VAEs when simultaneously fitting neural network mappings from a (random) variable onto the mean and variance of a dependent Gaussian variable. Previous works have bolstered optimization and improved likelihoods, but fail other basic posterior predictive checks (PPCs). Under the PPC framework, we propose critiques to test predictive mean and variance calibration and the predictive distribution's ability to generate sensible data. We find that our attractively simple solution, to treat heteroscedastic variance variationally, sufficiently regularizes variance to pass these PPCs. We consider a diverse gamut of existing and novel priors and find our methods preserve or outperform existing model likelihoods while significantly improving parameter calibration and sample quality for regression and VAEs.

Clinical decision making about treatments and interventions based on personal characteristics leads to effective health improvement. Machine learning (ML) has been the central concern of the diagnosis support and disease prediction based on comprehensive patient information. Because the black-box problem in ML is serious for medical applications, explainable artificial intelligence (XAI) techniques to explain the reasons for ML models predictions have been focused. A remaining important issue in clinical situations is discovery of concrete and realistic treatment processes. This paper proposes an innovative framework to plan concrete treatment processes based on an ML model. A key point of our proposed framework is to evaluate an "actionability" of the treatment process using a stochastic surrogate model constructed through hierarchical Bayesian modeling. The actionability is an essential concept for suggesting a realistic treatment process, which leads to clinical applications for personal health improvement. This paper also presents two experiments to evaluate our framework. We first demonstrate the feasibility of our framework from the viewpoint of the methodology using a synthetic dataset. Subsequently, our framework is applied to an actual health checkup dataset, which comprises 3,132 participants, considering an application to improve systolic blood pressure values at a personal level. We confirmed that the computed treatment processes are actionable and consistent with clinical knowledge for lowering blood pressure. These results demonstrate that our framework can contribute to decision making in the medical field. Our framework can be expected to provide clinicians deeper insights by proposing concrete and actionable treatment process based on the ML model.

We present a reformulation of the regression and classification, which aims to validate the result of a machine learning algorithm. Our reformulation simplifies the original problem and validates the result of the machine learning algorithm using the training data. Since the validation of machine learning algorithms must always be explainable, we perform our experiments with the kNN algorithm as well as with an algorithm based on conditional probabilities, which is proposed in this work. For the evaluation of our approach, three publicly available data sets were used and three classification and two regression problems were evaluated. The presented algorithm based on conditional probabilities is also online capable and requires only a fraction of memory compared to the kNN algorithm.