How to estimate heterogeneity, e.g. the effect of some variable differing across observations, is a key question in political science. Methods for doing so make simplifying assumptions about the underlying nature of the heterogeneity to draw reliable inferences. This paper allows a common way of simplifying complex phenomenon (placing observations with similar effects into discrete groups) to be integrated into regression analysis. The framework allows researchers to (i) use their prior knowledge to guide which groups are permissible and (ii) appropriately quantify uncertainty. The paper does this by extending work on "structured sparsity" from a traditional penalized likelihood approach to a Bayesian one by deriving new theoretical results and inferential techniques. It shows that this method outperforms state-of-the-art methods for estimating heterogeneous effects when the underlying heterogeneity is grouped and more effectively identifies groups of observations with different effects in observational data.

The potential of digital twin technology is immense, specifically in the infrastructure, aerospace, and automotive sector. However, practical implementation of this technology is not at an expected speed, specifically because of lack of application-specific details. In this paper, we propose a novel digital twin framework for stochastic nonlinear multi-degree of freedom (MDOF) dynamical systems. The approach proposed in this paper strategically decouples the problem into two time-scales -- (a) a fast time-scale governing the system dynamics and (b) a slow time-scale governing the degradation in the system. The proposed digital twin has four components - (a) a physics-based nominal model (low-fidelity), (b) a Bayesian filtering algorithm a (c) a supervised machine learning algorithm and (d) a high-fidelity model for predicting future responses. The physics-based nominal model combined with Bayesian filtering is used combined parameter state estimation and the supervised machine learning algorithm is used for learning the temporal evolution of the parameters. While the proposed framework can be used with any choice of Bayesian filtering and machine learning algorithm, we propose to use unscented Kalman filter and Gaussian process. Performance of the proposed approach is illustrated using two examples. Results obtained indicate the applicability and excellent performance of the proposed digital twin framework.

We develop a formalism for constructing stochastic upper bounds on the expected full sample risk for supervised classification tasks via the Hilbert coresets approach within a transductive framework. We explicitly compute tight and meaningful bounds for complex datasets and complex hypothesis classes such as state-of-the-art deep neural network architectures. The bounds we develop exhibit nice properties: i) the bounds are non-uniform in the hypothesis space H, ii) in many practical examples, the bounds become effectively deterministic by appropriate choice of prior and training data-dependent posterior distributions on the hypothesis space, and iii) the bounds become significantly better with increase in the size of the training set. We also lay out some ideas to explore for future research. Generalization bounds for learning provide a theoretical guarantee on the performance of a learning algorithm on unseen data. The goal of such bounds is to provide control of the error on unseen data with pre-specified confidence. In certain situations, such bounds may also help in designing new learning algorithms.

Markov networks are popular models for discrete multivariate systems where the dependence structure of the variables is specified by an undirected graph. To allow for more expressive dependence structures, several generalizations of Markov networks have been proposed. Here we consider the class of contextual Markov networks which takes into account possible context-specific independences among pairs of variables. Structure learning of contextual Markov networks is very challenging due to the extremely large number of possible structures. One of the main challenges has been to design a score, by which a structure can be assessed in terms of model fit related to complexity, without assuming chordality. Here we introduce the marginal pseudo-likelihood as an analytically tractable criterion for general contextual Markov networks. Our criterion is shown to yield a consistent structure estimator. Experiments demonstrate the favorable properties of our method in terms of predictive accuracy of the inferred models.

Convolutional dictionary learning (CDL), the problem of estimating shift-invariant templates from data, is typically conducted in the absence of a prior/structure on the templates. In data-scarce or low signal-to-noise ratio (SNR) regimes, which have received little attention from the community, learned templates overfit the data and lack smoothness, which can affect the predictive performance of downstream tasks. To address this limitation, we propose GPCDL, a convolutional dictionary learning framework that enforces priors on templates using Gaussian Processes (GPs). With the focus on smoothness, we show theoretically that imposing a GP prior is equivalent to Wiener filtering the learned templates, thereby suppressing high-frequency components and promoting smoothness. We show that the algorithm is a simple extension of the classical iteratively reweighted least squares, which allows the flexibility to experiment with different smoothness assumptions. Through simulation, we show that GPCDL learns smooth dictionaries with better accuracy than the unregularized alternative across a range of SNRs. Through an application to neural spiking data from rats, we show that learning templates by GPCDL results in a more accurate and visually-interpretable smooth dictionary, leading to superior predictive performance compared to non-regularized CDL, as well as parametric alternatives.

Unobserved confounding is one of the greatest challenges for causal discovery. The case in which unobserved variables have a widespread effect on many of the observed ones is particularly difficult because most pairs of variables are conditionally dependent given any other subset, rendering the causal effect unidentifiable. In this paper we show that beyond conditional independencies, under the principle of independent mechanisms, unobserved confounding in this setting leaves a statistical footprint in the observed data distribution that allows for disentangling spurious and causal effects. Using this insight, we demonstrate that a sparse linear Gaussian directed acyclic graph among observed variables may be recovered approximately and propose an adjusted score-based causal discovery algorithm that may be implemented with general purpose solvers and scales to high-dimensional problems. We find, in addition, that despite the conditions we pose to guarantee causal recovery, performance in practice is robust to large deviations in model assumptions.

Trained AI systems and expert decision makers can make errors that are often difficult to identify and understand. Determining the root cause for these errors can improve future decisions. This work presents Generative Error Model (GEM), a generative model for inferring representational errors based on observations of an actor's behavior (either simulated agent, robot, or human). The model considers two sources of error: those that occur due to representational limitations -- "blind spots" -- and non-representational errors, such as those caused by noise in execution or systematic errors present in the actor's policy. Disambiguating these two error types allows for targeted refinement of the actor's policy (i.e., representational errors require perceptual augmentation, while other errors can be reduced through methods such as improved training or attention support). We present a Bayesian inference algorithm for GEM and evaluate its utility in recovering representational errors on multiple domains. Results show that our approach can recover blind spots of both reinforcement learning agents as well as human users.

Recently it was introduced a negation of a probability distribution. The need for such negation arises when a knowledge-based system can use the terms like NOT HIGH, where HIGH is represented by a probability distribution (pd). For example, HIGH PROFIT or HIGH PRICE can be considered. The application of this negation in Dempster-Shafer theory was considered in many works. Although several negations of probability distributions have been proposed, it was not clear how to construct other negation. In this paper, we consider negations of probability distributions as point-by-point transformations of pd using decreasing functions defined on [0,1] called negators. We propose the general method of generation of negators and corresponding negations of pd, and study their properties. We give a characterization of linear negators as a convex combination of Yager's and uniform negators. Keywords: Probability distribution, Negation, Dempster-Shafer theory 1. Introduction The concept of negation of probability distribution (pd) was recently introduced by Yager [18].

Network inference is the process of learning the properties of complex networks from data. Besides using information about known links in the network, node attributes and other forms of network metadata can help to solve network inference problems. Indeed, several approaches have been proposed to introduce metadata into probabilistic network models and to use them to make better inferences. However, we know little about the effect of such metadata in the inference process. Here, we investigate this issue. We find that, rather than affecting inference gradually, adding metadata causes abrupt transitions in the inference process and in our ability to make accurate predictions, from a situation in which metadata does not play any role to a situation in which metadata completely dominates the inference process. When network data and metadata are partly correlated, metadata optimally contributes to the inference process at the transition between data-dominated and metadata-dominated regimes.

The article provides a review of the publications on the current trends and developments in Dempster-Shafer theory and its different applications in science, engineering, and technologies. The review took account of the following provisions with a focus on some specific aspects of the theory. Firstly, the article considers the research directions whose results are known not only in scientific and academic community but understood by a wide circle of potential designers and developers of advanced engineering solutions and technologies. Secondly, the article shows the theory applications in some important areas of human activity such as manufacturing systems, diagnostics of technological processes, materials and products, building and construction, product quality control, economic and social systems. The particular attention is paid to the current state of research in the domains under consideration and, thus, the papers published, as a rule, in recent years and presenting the achievements of modern research on Dempster-Shafer theory and its application are selected and analyzed.