Making predictions robust is an important challenge. A separate challenge in federated learning (FL) is to reduce the number of communication rounds, particularly since doing so reduces performance in heterogeneous data settings. To tackle both issues, we take a Bayesian perspective on the problem of learning a global model. We show how the global predictive posterior can be approximated using client predictive posteriors. This is unlike other works which aggregate the local model space posteriors into the global model space posterior, and are susceptible to high approximation errors due to the posterior's high dimensional multimodal nature. In contrast, our method performs the aggregation on the predictive posteriors, which are typically easier to approximate owing to the low-dimensionality of the output space. We present an algorithm based on this idea, which performs MCMC sampling at each client to obtain an estimate of the local posterior, and then aggregates these in one round to obtain a global ensemble model. Through empirical evaluation on several classification and regression tasks, we show that despite using one round of communication, the method is competitive with other FL techniques, and outperforms them on heterogeneous settings. The code is publicly available at https://github.com/hasanmohsin/FedPredSpace_1Round.

Tensor decompositions have proven to be effective in analyzing the structure of multidimensional data. However, most of these methods require a key parameter: the number of desired components. In the case of the CANDECOMP/PARAFAC decomposition (CPD), this value is known as the canonical rank and greatly affects the quality of the results. Existing methods use heuristics or Bayesian methods to estimate this value by repeatedly calculating the CPD, making them extremely computationally expensive. In this work, we propose FRAPPE and Self-FRAPPE: a cheaply supervised and a self-supervised method to estimate the canonical rank of a tensor without ever having to compute the CPD. We call FRAPPE cheaply supervised because it uses a fully synthetic training set without requiring real-world examples. We evaluate these methods on synthetic tensors, real tensors of known rank, and the weight tensor of a convolutional neural network. We show that FRAPPE and Self-FRAPPE offer large improvements in both effectiveness and speed, with a respective $15\%$ and $10\%$ improvement in MAPE and an $4000\times$ and $13\times$ improvement in evaluation speed over the best-performing baseline.

Multi-task Gaussian process (MTGP) is a well-known non-parametric Bayesian model for learning correlated tasks effectively by transferring knowledge across tasks. But current MTGPs are usually limited to the multi-task scenario defined in the same input domain, leaving no space for tackling the heterogeneous case, i.e., the features of input domains vary over tasks. To this end, this paper presents a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) model for simultaneously learning the tasks with varied input domains. Particularly, we develop the stochastic variational framework with Bayesian calibration that (i) takes into account the effect of dimensionality reduction raised by domain mappings in order to achieve effective input alignment; and (ii) employs a residual modeling strategy to leverage the inductive bias brought by prior domain mappings for better model inference. Finally, the superiority of the proposed model against existing LMC models has been extensively verified on diverse heterogeneous multi-task cases and a practical multi-fidelity steam turbine exhaust problem.

The photoelectric factor (PEF) is an important well logging tool to distinguish between different types of reservoir rocks because PEF measurement is sensitive to elements with high atomic number. Furthermore, the ratio of rock minerals could be determined by combining PEF log with other well logs. However, PEF log could be missing in some cases such as in old well logs and wells drilled with barite-based mud. Therefore, developing models for estimating missing PEF log is essential in those circumstances. In this work, we developed various machine learning models to predict PEF values using the following well logs as inputs: bulk density (RHOB), neutron porosity (NPHI), gamma ray (GR), compressional and shear velocity. The predictions of PEF values using adaptive-network-fuzzy inference system (ANFIS) and artificial neural network (ANN) models have errors of about 16% and 14% average absolute percentage error (AAPE) in the testing dataset, respectively. Thus, a different approach was proposed that is based on the concept of automated machine learning. It works by automatically searching for the optimal model type and optimizes its hyperparameters for the dataset under investigation. This approach selected a Gaussian process regression (GPR) model for accurate estimation of PEF values. The developed GPR model decreases the AAPE of the predicted PEF values in the testing dataset to about 10% AAPE. This error could be further decreased to about 2% by modeling the potential noise in the measurements using the GPR model.

Temperature and humidity are two of the rudimentary factors that must be controlled during egg incubation. Improper temperature and humidity levels during the incubation period often result in unwanted conditions. This paper proposes the design of an efficient Mamdani fuzzy interference system instead of the widely used Takagi-Sugeno system in this field for controlling the temperature and humidity levels of an egg incubator. Though the optimum incubation temperature and humidity levels used here are that of chicken egg, the proposed methodology is applicable to other avian species as well. Theinput functions have been used here as per estimated values forsafe hatching using Mamdani whereas defuzzification method, COA, has been applied for output. From the model output,a stabilized heat from temperature level and fan speed to control the humidity level of an egg incubator can be obtained. This maximizes the hatching rate of healthy chicks under any conditions in the field.

For many applications, analyzing the uncertainty of a machine learning model is indispensable. While research of uncertainty quantification (UQ) techniques is very advanced for computer vision applications, UQ methods for spatio-temporal data are less studied. In this paper, we focus on models for online handwriting recognition, one particular type of spatio-temporal data. The data is observed from a sensor-enhanced pen with the goal to classify written characters. We conduct a broad evaluation of aleatoric (data) and epistemic (model) UQ based on two prominent techniques for Bayesian inference, Stochastic Weight Averaging-Gaussian (SWAG) and Deep Ensembles. Next to a better understanding of the model, UQ techniques can detect out-of-distribution data and domain shifts when combining right-handed and left-handed writers (an underrepresented group).

A common way to learn and analyze statistical models is to consider operations in the model parameter space. But what happens if we optimize in the parameter space and there is no one-to-one mapping between the parameter space and the underlying statistical model space? Such cases frequently occur for hierarchical models which include statistical mixtures or stochastic neural networks, and these models are said to be singular. Singular models reveal several important and well-studied problems in machine learning like the decrease in convergence speed of learning trajectories due to attractor behaviors. In this work, we propose a relative reparameterization technique of the parameter space, which yields a general method for extracting regular submodels from singular models. Our method enforces model identifiability during training and we study the learning dynamics for gradient descent and expectation maximization for Gaussian Mixture Models (GMMs) under relative parameterization, showing faster experimental convergence and a improved manifold shape of the dynamics around the singularity. Extending the analysis beyond GMMs, we furthermore analyze the Fisher information matrix under relative reparameterization and its influence on the generalization error, and show how the method can be applied to more complex models like deep neural networks.

Currently available quantum computers suffer from constraints including hardware noise and a limited number of qubits. As such, variational quantum algorithms that utilise a classical optimiser in order to train a parameterised quantum circuit have drawn significant attention for near-term practical applications of quantum technology. In this work, we take a probabilistic point of view and reformulate the classical optimisation as an approximation of a Bayesian posterior. The posterior is induced by combining the cost function to be minimised with a prior distribution over the parameters of the quantum circuit. We describe a dimension reduction strategy based on a maximum a posteriori point estimate with a Laplace prior. Experiments on the Quantinuum H1-2 computer show that the resulting circuits are faster to execute and less noisy than the circuits trained without the dimension reduction strategy. We subsequently describe a posterior sampling strategy based on stochastic gradient Langevin dynamics. Numerical simulations on three different problems show that the strategy is capable of generating samples from the full posterior and avoiding local optima.

In the complex multi-state system (MSS), reliability analysis is a significant research content, both for equipment design, manufacturing, usage and maintenance. Universal Generating Function (UGF) is an important method in the reliability analysis, which efficiently obtains the system reliability by a fast algebraic procedure. However, when structural relationships between subsystems or components are not clear or without explicit expressions, the UGF method is difficult to use or not applicable at all. Bayesian Network (BN) has a natural advantage in terms of uncertainty inference for the relationship without explicit expressions. For the number of components is extremely large, though, it has the defects of low efficiency. To overcome the respective defects of UGF and BN, a novel reliability analysis method called UGF-BN is proposed for the complex MSS. In the UGF-BN framework, the UGF method is firstly used to analyze the bottom components with a large number. Then probability distributions obtained are taken as the input of BN. Finally, the reliability of the complex MSS is modeled by the BN method. This proposed method improves the computational efficiency, especially for the MSS with the large number of bottom components. Besides, the aircraft reliability-based design optimization based on the UGF-BN method is further studied with budget constraints on mass, power, and cost. Finally, two cases are used to demonstrate and verify the proposed method.

Prognostics aid in the longevity of fielded systems or products. Quantifying the system's current health enable prognosis to enhance the operator's decision-making to preserve the system's health. Creating a prognosis for a system can be difficult due to (a) unknown physical relationships and/or (b) irregularities in data appearing well beyond the initiation of a problem. Traditionally, three different modeling paradigms have been used to develop a prognostics model: physics-based (PbM), data-driven (DDM), and hybrid modeling. Recently, the hybrid modeling approach that combines the strength of both PbM and DDM based approaches and alleviates their limitations is gaining traction in the prognostics domain. In this paper, a novel hybrid modeling approach for prognostics applications based on combining concepts from fuzzy logic and generative adversarial networks (GANs) is outlined. The FuzzyGAN based method embeds a physics-based model in the aggregation of the fuzzy implications. This technique constrains the output of the learning method to a realistic solution. Results on a bearing problem showcases the efficacy of adding a physics-based aggregation in a fuzzy logic model to improve GAN's ability to model health and give a more accurate system prognosis.