We propose a new approach to explain Bayesian Networks. The approach revolves around a new definition of a probabilistic argument and the evidence it provides. We define a notion of independent arguments, and propose an algorithm to extract a list of relevant, independent arguments given a Bayesian Network, a target node and a set of observations. To demonstrate the relevance of the arguments, we show how we can use the extracted arguments to approximate message passing. Finally, we show a simple scheme to explain the arguments in natural language.

It is more flexible for decision makers to evaluate by interval-valued q-rung orthopair fuzzy set (IVq-ROFS),which offers fuzzy decision-making more applicational space. Meanwhile, Choquet integralses non-additive set function (fuzzy measure) to describe the interaction between attributes directly.In particular, there are a large number of practical issues that have relevance between attributes.Therefore,this paper proposes the correlation operator and group decision-making method based on the interval-valued q-rung orthopair fuzzy set Choquet integral.First,interval-valued q-rung orthopair fuzzy Choquet integral average operator (IVq-ROFCA) and interval-valued q-rung orthopair fuzzy Choquet integral geometric operator (IVq-ROFCG) are inves-tigated,and their basic properties are proved.Furthermore, several operators based on IVq-ROFCA and IVq-ROFCG are developed. Then, a group decision-making method based on IVq-ROFCA is developed,which can solve the decision making problems with interaction between attributes.Finally,through the implementation of the warning management system for hypertension,it is shown that the operator and group decision-making method proposed in this paper can handle complex decision-making cases in reality, and the decision result is consistent with the doctor's diagnosis result.Moreover,the comparison with the results of other operators shows that the proposed operators and group decision-making method are correct and effective,and the decision result will not be affected by the change of q value.

Changepoint analysis deals with unsupervised detection and/or estimation of time-points in time-series data, when the distribution generating the data changes. In this article, we consider \emph{offline} changepoint detection in the context of large scale textual data. We build a specialised temporal topic model with provisions for changepoints in the distribution of topic proportions. As full likelihood based inference in this model is computationally intractable, we develop a computationally tractable approximate inference procedure. More specifically, we use sample splitting to estimate topic polytopes first and then apply a likelihood ratio statistic together with a modified version of the wild binary segmentation algorithm of Fryzlewicz et al. (2014). Our methodology facilitates automated detection of structural changes in large corpora without the need of manual processing by domain experts. As changepoints under our model correspond to changes in topic structure, the estimated changepoints are often highly interpretable as marking the surge or decline in popularity of a fashionable topic. We apply our procedure on two large datasets: (i) a corpus of English literature from the period 1800-1922 (Underwoodet al., 2015); (ii) abstracts from the High Energy Physics arXiv repository (Clementet al., 2019). We obtain some historically well-known changepoints and discover some new ones.

Independent Component Analysis (ICA) is intended to recover the mutually independent sources from their linear mixtures, and F astICA is one of the most successful ICA algorithms. Although it seems reasonable to improve the performance of F astICA by introducing more nonlinear functions to the negentropy estimation, the original fixed-point method (approximate Newton method) in F astICA degenerates under this circumstance. To alleviate this problem, we propose a novel method based on the second-order approximation of minimum discrimination information (MDI). The joint maximization in our method is consisted of minimizing single weighted least squares and seeking unmixing matrix by the fixed-point method. Experimental results validate its efficiency compared with other popular ICA algorithms.

The Bayesian brain hypothesis postulates that the brain accurately operates on statistical distributions according to Bayes' theorem. The random failure of presynaptic vesicles to release neurotransmitters may allow the brain to sample from posterior distributions of network parameters, interpreted as epistemic uncertainty. It has not been shown previously how random failures might allow networks to sample from observed distributions, also known as aleatoric or residual uncertainty. Sampling from both distributions enables probabilistic inference, efficient search, and creative or generative problem solving. We demonstrate that under a population-code based interpretation of neural activity, both types of distribution can be represented and sampled with synaptic failure alone. We first define a biologically constrained neural network and sampling scheme based on synaptic failure and lateral inhibition. Within this framework, we derive dropout based epistemic uncertainty, then prove an analytic mapping from synaptic efficacy to release probability that allows networks to sample from arbitrary, learned distributions represented by a receiving layer. Second, our result leads to a local learning rule by which synapses adapt their release probabilities. Our result demonstrates complete Bayesian inference, related to the variational learning method of dropout, in a biologically constrained network using only locally-learned synaptic failure rates. Introduction The Bayesian Brain hypothesis has led to a number of important insights about neural coding in the brain (Knill and Pouget, 2004; Friston, 2010, 2012; Pouget et al., 2013; Lee and Mumford, 2003) by characterizing neural representation and processing in terms of formal probabilistic inference and sampling. Furthermore, the introduction of related probabilistic representations and sampling processes in modern deep learning variational models has led to improved performance on a range of different tasks (Zhang et al., 2019; Blei et al., 2017; Kingma and Welling, 2014; Detorakis et al., 2019). The widely-used dropout technique in deep learning can be seen as a form of variational inference and sampling (Srivastava et al., 2014; Gal and Ghahramani, 2016) with direct analogy to the random failure of synapses in the brain. This link has led to biologically-motivated models of variational deep learning that use network weight dropout to simulate synaptic failure (Mostafa and Cauwenberghs, 2018; Wan et al., 2013; Neftci et al., 2016). In this paper, we build on these and other recent findings in machine learning and neurobiology to show how the brain can accurately represent the two primary components of probabilistic inference, distributions of observed data and distributions of unobserved values (such as model parameters), with the single, biologically established mechanism of synaptic failure.

In recent years, large-scale Bayesian learning draws a great deal of attention. However, in big-data era, the amount of data we face is growing much faster than our ability to deal with it. Fortunately, it is observed that large-scale datasets usually own rich internal structure and is somewhat redundant. In this paper, we attempt to simplify the Bayesian posterior via exploiting this structure. Specifically, we restrict our interest to the so-called well-clustered datasets and construct an \emph{approximate posterior} according to the clustering information. Fortunately, the clustering structure can be efficiently obtained via a particular clustering algorithm. When constructing the approximate posterior, the data points in the same cluster are all replaced by the centroid of the cluster. As a result, the posterior can be significantly simplified. Theoretically, we show that under certain conditions the approximate posterior we construct is close (measured by KL divergence) to the exact posterior. Furthermore, thorough experiments are conducted to validate the fact that the constructed posterior is a good approximation to the true posterior and much easier to sample from.

Mixture model is a probabilistic model that is able to infer subpopulations from total population without additional information (within the paradigm of unsupervised learning). Mixture models closely correspond to the mixture distributions of the probabilistic distributions of observations. In general, in the structure of mixture model, we make assumptions over latent variables that evaluate membership of each observation. Given the dataset, we can assume that it is a sample and then mixture model can estimate the parameters of the probability distributions that created points of this dataset, as well as assign each observation vector of probabilities indicating the original distribution. Comparing different mixture models can be considered a generalization of the problem of comparing different distributions. From the viewpoint of optimal transport theory, the Wasserstein distance is an important method for measuring similarities and the maintenance of the explainable nature of mixture models.

Hierarchical models with gamma hyperpriors provide a flexible, sparse-promoting framework to bridge $L^1$ and $L^2$ regularizations in Bayesian formulations to inverse problems. Despite the Bayesian motivation for these models, existing methodologies are limited to \textit{maximum a posteriori} estimation. The potential to perform uncertainty quantification has not yet been realized. This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement. In addition, it lends itself naturally to conduct model selection for the choice of hyperparameters. We illustrate the performance of our methodology in several computed examples, including a deconvolution problem and sparse identification of dynamical systems from time series data.

Inspired by the foundational works by Spivak and Fong and Cruttwell et al., we introduce a categorical framework to formalize Bayesian inference and learning. The two key ideas at play here are the notions of Bayesian inversions and the functor GL as constructed by Cruttwell et al.. In this context, we find that Bayesian learning is the simplest case of the learning paradigm. We then obtain categorical formulations of batch and sequential Bayes updates while also verifying that the two coincide in a specific example.

All three of us are authors of the PyMC Probabilistic Programming Language, a production grade package used at leading organizations around the world. Ravin learned the power of Bayes Theorem at SpaceX when improving the supply chains of the world's most advanced rockets. He's now an advocate of applied Bayesian methods and has since authored a textbook about Bayes Theorem and writes about appllied data science on his blog. Thomas is enthusiastic about teaching statistics using code and examples, rather than arduous math. Through his many talks and blog posts, he has shown that there is a different way to teach statistics.