State space models (SSMs) are a flexible approach to modeling complex time series. However, inference in SSMs is often computationally prohibitive for long time series. Stochastic gradient MCMC (SGMCMC) is a popular method for scalable Bayesian inference for large independent data. Unfortunately when applied to dependent data, such as in SSMs, SGMCMC's stochastic gradient estimates are biased as they break crucial temporal dependencies. To alleviate this, we propose stochastic gradient estimators that control this bias by performing additional computation in a `buffer' to reduce breaking dependencies. Furthermore, we derive error bounds for this bias and show a geometric decay under mild conditions. Using these estimators, we develop novel SGMCMC samplers for discrete, continuous and mixed-type SSMs. Our experiments on real and synthetic data demonstrate the effectiveness of our SGMCMC algorithms compared to batch MCMC, allowing us to scale inference to long time series with millions of time points.

A Dynamic Bayesian Network (DBN) [1-3] is a widely used family of graphical model for representing and reasoning within dynamic systems whose progress is recorded over a discrete time intervals [4-10]. However, in some context a DBN model is not able to represent all structural information of the target process [11]. This is particularly the case when the process is more naturally described by concatenations of unfolding events rather than by a product space of preassigned set of random variables. In other situations, a relevant statement corresponding to a conditioned variable cannot be directly incorporated into a DBN model using directed edges because it is valid only for a certain combinations of values assumed by the conditioning variables. In the literature, this type of statements is sometimes referred to context-specific information [12, 13]. To circumvent these issues, collections of networks and embellishments in the form of trees have been added to the DBN framework and computationally implemented using the object-oriented programming paradigm [14]: for instance, see the developments on context-specific BNs [11, 13, 15], Bayesian Multinet [16], Similarity Networks [17] and Object-Oriented BNs [18, 19].

Understanding the uncertainty of a neural network's (NN) predictions is essential for many applications. The Bayesian framework provides a principled approach to this, however applying it to NNs is challenging due to the large number of parameters and data. Ensembling NNs provides a practical and scalable method for uncertainty quantification. Its drawback is that its justification is heuristic rather than Bayesian. In this work we propose one modification to the usual ensembling process, that does result in Bayesian behaviour: regularising parameters about values drawn from a prior distribution. Hence, we present an easily implementable, scalable technique for performing approximate Bayesian inference in NNs.

Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly. We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any derived quantities, but can be shown to be equivalent to maximizing likelihood under some conditions. Our result holds in the non-asymptotic parametric setting, where both the capacity of the model and the number of data examples are finite. We also demonstrate encouraging experimental results.

In this chapter we discuss soft concept analysis, a study which identifies an enriched notion of "conceptual scale" as developed in formal concept analysis with an enriched notion of "linguistic variable" as discussed in fuzzy logic. The identification "enriched conceptual scale" = "enriched linguistic variable" was made in a previous paper (Enriched interpretation, Robert E. Kent). In this chapter we offer further arguments for the importance of this identification by discussing the philosophy, spirit, and practical application of conceptual scaling to the discovery, conceptual analysis, interpretation, and categorization of networked information resources. We argue that a linguistic variable, which has been defined at just the right generalization of valuated categories, provides a natural definition for the process of soft conceptual scaling. This enrichment using valuated categories models the relation of indiscernability, a notion of central importance in rough set theory. At a more fundamental level for soft concept analysis, it also models the derivation of formal concepts, a process of central importance in formal concept analysis. Soft concept analysis is synonymous with enriched concept analysis. From one viewpoint, the study of soft concept analysis that is initiated here extends formal concept analysis to soft computational structures. From another viewpoint, soft concept analysis provides a natural foundation for soft computation by unifying and explaining notions from soft computation in terms of suitably generalized notions from formal concept analysis, rough set theory and fuzzy set theory.

In this paper we present a hybrid active sampling strategy for pairwise preference aggregation, which aims at recovering the underlying rating of the test candidates from sparse and noisy pairwise labelling. Our method employs Bayesian optimization framework and Bradley-Terry model to construct the utility function, then to obtain the Expected Information Gain (EIG) of each pair. For computational efficiency, Gaussian-Hermite quadrature is used for estimation of EIG. In this work, a hybrid active sampling strategy is proposed, either using Global Maximum (GM) EIG sampling or Minimum Spanning Tree (MST) sampling in each trial, which is determined by the test budget. The proposed method has been validated on both simulated and real-world datasets, where it shows higher preference aggregation ability than the state-of-the-art methods.

In this paper, we propose a new systematic approach based on nonquadratic Lyapunov function and technique of introducing slack matrices, for a class of affine nonlinear systems with disturbance. To achieve the goal, first, the affine nonlinear system is represented via Takagi-Sugeno (T-S) fuzzy bilinear model. Subsequently, the robust H_inf controller is designed based on parallel distributed compensation (PDC) scheme. Then, the stability conditions are derived in terms of linear matrix inequalities (LMIs) by utilizing Lyapunov function. Moreover, some slack matrices are proposed to reduce the conservativeness of the LMI stability conditions. Finally, for illustrating the merits and verifying the effectiveness of the proposed approach, the application of an isothermal continuous stirred tank reactor (CSTR) for Van de Vusse reactor is discussed in details.

In many applications of statistics and machine learning, we wish to learn models that achieve uniformly good performance over almost all input values. This is important for safety-and fairnesscritical systems such as medical diagnosis, autonomous vehicles, criminal justice and credit evaluations, where poor performance on the tails of the inputs leads to high-cost system failures. Methods that optimize average performance, however, often produce models that suffer low performance on the "hard" instances of the population. For example, standard regressors obtained from maximum likelihood estimation can lose their predictive power on certain regions of covariates [57], so that high average performance comes at the expense of low performance on minority subpopulations. In this work, we propose and study a procedure that explicitly optimizes performance on tail inputs that suffer high loss. Modern datasets incorporate heterogeneous (but latent) subpopulations, and a natural goal is to perform well across all of these [57, 65, 21]. While many statistical models show strong average performance, their performance often deteriorates on minority groups underrepresented in the dataset. For example, speech recognition systems are inaccurate for people with minority accents [4]. In numerous other applications--such as facial recognition, automatic video captioning, language identification, academic recommender systems--performance varies significantly over different demographic groupings, such as race, gender, or age [38, 42, 18, 68, 76].

Score based learning (SBL) is a promising approach for learning Bayesian networks in the discrete domain. However, when employing SBL in the continuous domain, one is either forced to move the problem to the discrete domain or use metrics such as BIC/AIC, and these approaches are often lacking. Discretization can have an undesired impact on the accuracy of the results, and BIC/AIC can fall short of achieving the desired accuracy. In this paper, we introduce two new scoring metrics for scoring Bayesian networks in the continuous domain: the three-part minimum description length and the renormalized normalized maximum likelihood metric. We rely on the minimum description length principle in formulating these metrics. The metrics proposed are free of hyperparameters, decomposable, and are asymptotically consistent. We evaluate our solution by studying the convergence rate of the learned graph to the generating network and, also, the structural hamming distance of the learned graph to the generating network. Our evaluations show that the proposed metrics outperform their competitors, the BIC/AIC metrics. Furthermore, using the proposed RNML metric, SBL will have the fastest rate of convergence with the smallest structural hamming distance to the generating network.

We propose Lomax delegate racing (LDR) to explicitly model the mechanism of survival under competing risks and to interpret how the covariates accelerate or decelerate the time to event. LDR explains non-monotonic covariate effects by racing a potentially infinite number of sub-risks, and consequently relaxes the ubiquitous proportional-hazards assumption which may be too restrictive. Moreover, LDR is naturally able to model not only censoring, but also missing event times or event types. For inference, we develop a Gibbs sampler under data augmentation for moderately sized data, along with a stochastic gradient descent maximum a posteriori inference algorithm for big data applications. Illustrative experiments are provided on both synthetic and real datasets, and comparison with various benchmark algorithms for survival analysis with competing risks demonstrates distinguished performance of LDR.