Learning Graphical Models
Meta Learning as Bayes Risk Minimization
Maeda, Shin-ichi, Nakanishi, Toshiki, Koyama, Masanori
We show that, when we cast meta-learning problem as BRM, the optimal solution Meta-Learning is a family of methods that use is given by the predictive distribution computed from a set of interrelated tasks to learn a model that the posterior distribution of the latent variable conditioned can quickly learn a new query task from a possibly against the contextual dataset. This result justifies the use of small contextual dataset. In this study, we the predictive distribution in many previous studies of meta use a probabilistic framework to formalize what learning, such as (Edwards & Storkey, 2017; Gordon et al., it means for two tasks to be related and reframe 2018; Garnelo et al., 2018). However, the optimality of the the meta-learning problem into the problem of predictive distribution cannot be guaranteed if one uses an Bayesian risk minimization (BRM). In our formulation, approximation of the posterior distribution that violates the the BRM optimal solution is given by the way the posterior distribution changes with the contextual predictive distribution computed from the posterior dataset, and this is unfortunately the case for most of the distribution of the task-specific latent variable aforementioned works. For example, the variance of the conditioned on the contextual dataset, and this posterior in these works do not converge to 0 as we take justifies the philosophy of Neural Process.
Fully probabilistic quasar continua predictions near Lyman-{\alpha} with conditional neural spline flows
Reiman, David M., Tamanas, John, Prochaska, J. Xavier, Ďurovčíková, Dominika
Measurement of the red damping wing of neutral hydrogen in quasar spectra provides a probe of the epoch of reionization in the early Universe. Such quantification requires precise and unbiased estimates of the intrinsic continua near Lyman-$\alpha$ (Ly$\alpha$), a challenging task given the highly variable Ly$\alpha$ emission profiles of quasars. Here, we introduce a fully probabilistic approach to intrinsic continua prediction. We frame the problem as a conditional density estimation task and explicitly model the distribution over plausible blue-side continua ($1190\ \unicode{xC5} \leq \lambda_{\text{rest}} < 1290\ \unicode{xC5}$) conditional on the red-side spectrum ($1290\ \unicode{xC5} \leq \lambda_{\text{rest}} < 2900\ \unicode{xC5}$) using normalizing flows. Our approach achieves state-of-the-art precision and accuracy, allows for sampling one thousand plausible continua in less than a tenth of a second, and can natively provide confidence intervals on the blue-side continua via Monte Carlo sampling. We measure the damping wing effect in two $z>7$ quasars and estimate the volume-averaged neutral fraction of hydrogen from each, finding $\bar{x}_\text{HI}=0.304 \pm 0.042$ for ULAS J1120+0641 ($z=7.09$) and $\bar{x}_\text{HI}=0.384 \pm 0.133$ for ULAS J1342+0928 ($z=7.54$).
Variational Bayesian Inference for Crowdsourcing Predictions
Cai, Desmond, Nguyen, Duc Thien, Lim, Shiau Hong, Wynter, Laura
Crowdsourcing has emerged as an effective means for performing a number of machine learning tasks such as annotation and labelling of images and other data sets. In most early settings of crowdsourcing, the task involved classification, that is assigning one of a discrete set of labels to each task. Recently, however, more complex tasks have been attempted including asking crowdsource workers to assign continuous labels, or predictions. In essence, this involves the use of crowdsourcing for function estimation. We are motivated by this problem to drive applications such as collaborative prediction, that is, harnessing the wisdom of the crowd to predict quantities more accurately. To do so, we propose a Bayesian approach aimed specifically at alleviating overfitting, a typical impediment to accurate prediction models in practice. In particular, we develop a variational Bayesian technique for two different worker noise models - one that assumes workers' noises are independent and the other that assumes workers' noises have a latent low-rank structure. Our evaluations on synthetic and real-world datasets demonstrate that these Bayesian approaches perform significantly better than existing non-Bayesian approaches and are thus potentially useful for this class of crowdsourcing problems.
Sampling Techniques in Bayesian Target Encoding
Target encoding is an effective encoding technique of categorical variables and is often used in machine learning systems for processing tabular data sets with mixed numeric and categorical variables. Recently en enhanced version of this encoding technique was proposed by using conjugate Bayesian modeling. This paper presents a further development of Bayesian encoding method by using sampling techniques, which helps in extracting information from intra-category distribution of the target variable, improves generalization and reduces target leakage.
Reinforcement learning and Bayesian data assimilation for model-informed precision dosing in oncology
Maier, Corinna, Hartung, Niklas, Kloft, Charlotte, Huisinga, Wilhelm, de Wiljes, Jana
Model-informed precision dosing (MIPD) using therapeutic drug/biomarker monitoring offers the opportunity to significantly improve the efficacy and safety of drug therapies. Current strategies comprise model-informed dosing tables or are based on maximum a-posteriori estimates. These approaches, however, lack a quantification of uncertainty and/or consider only part of the available patient-specific information. We propose three novel approaches for MIPD employing Bayesian data assimilation (DA) and/or reinforcement learning (RL) to control neutropenia, the major dose-limiting side effect in anticancer chemotherapy. These approaches have the potential to substantially reduce the incidence of life-threatening grade 4 and subtherapeutic grade 0 neutropenia compared to existing approaches. We further show that RL allows to gain further insights by identifying patient factors that drive dose decisions. Due to its flexibility, the proposed combined DA-RL approach can easily be extended to integrate multiple endpoints or patient-reported outcomes, thereby promising important benefits for future personalized therapies.
Uniform Convergence Rates for Maximum Likelihood Estimation under Two-Component Gaussian Mixture Models
Finite mixture models are a widely-used tool for modeling heterogeneous data, consisting of hidden subpopulations with distinct distributions. For applications exhibiting continuous data, location-scale Gaussian mixtures are arguably the most popular family of parametric mixture models. Beyond their broad applications as a modeling and clustering tool in the social, physical and life sciences (McLachlan & Peel 2004), Gaussian mixtures provide a flexible approach to density estimation (Genovese & Wasserman 2000, Ghosal & van der Vaart 2001). Estimating the parameters of a mixture model is crucial for quantifying the underlying heterogeneity of the data. One of the most widely-used approaches is the maximum likelihood estimator (MLE). A Gaussian mixture model with a known number of components K, all of which are well-separated, forms a regular parametric model for which the MLE achieves the standard parametric estimation rate (Ho & Nguyen 2016b, Chen 2017). Such rates are typically understood in terms of convergence of mixing measures, quantified using the Wasserstein distance as a means of avoiding label switching issues inherent in mixture modeling (Nguyen 2013).
Spatial-Temporal Mitosis Detection in Phase-Contrast Microscopy via Likelihood Map Estimation by 3DCNN
Nishimura, Kazuya, Bise, Ryoma
Automated mitotic detection in time-lapse phasecontrast microscopy provides us much information for cell behavior analysis, and thus several mitosis detection methods have been proposed. However, these methods still have two problems; 1) they cannot detect multiple mitosis events when there are closely placed. 2) they do not consider the annotation gaps, which may occur since the appearances of mitosis cells are very similar before and after the annotated frame. In this paper, we propose a novel mitosis detection method that can detect multiple mitosis events in a candidate sequence and mitigate the human annotation gap via estimating a spatiotemporal likelihood map by 3DCNN. In this training, the loss gradually decreases with the gap size between ground truth and estimation. This mitigates the annotation gaps. Our method outperformed the compared methods in terms of F1- score using a challenging dataset that contains the data under four different conditions.
Bayesian Optimisation vs. Input Uncertainty Reduction
Ungredda, Juan, Pearce, Michael, Branke, Juergen
Simulators often require calibration inputs estimated from real world data and the quality of the estimate can significantly affect simulation output. Particularly when performing simulation optimisation to find an optimal solution, the uncertainty in the inputs significantly affects the quality of the found solution. One remedy is to search for the solution that has the best performance on average over the uncertain range of inputs yielding an optimal compromise solution. We consider the more general setting where a user may choose between either running simulations or instead collecting real world data. A user may choose an input and a solution and observe the simulation output, or instead query an external data source improving the input estimate enabling the search for a more focused, less compromised solution. We explicitly examine the trade-off between simulation and real data collection in order to find the optimal solution of the simulator with the true inputs. Using a value of information procedure, we propose a novel unified simulation optimisation procedure called Bayesian Information Collection and Optimisation (BICO) that, in each iteration, automatically determines which of the two actions (running simulations or data collection) is more beneficial. Numerical experiments demonstrate that the proposed algorithm is able to automatically determine an appropriate balance between optimisation and data collection.
QuLBIT: Quantum-Like Bayesian Inference Technologies for Cognition and Decision
Moreira, Catarina, Hammes, Matheus, Kurdoglu, Rasim Serdar, Bruza, Peter
This paper provides the foundations of a unified cognitive decision-making framework (QulBIT) which is derived from quantum theory. The main advantage of this framework is that it can cater for paradoxical and irrational human decision making. Although quantum approaches for cognition have demonstrated advantages over classical probabilistic approaches and bounded rationality models, they still lack explanatory power. To address this, we introduce a novel explanatory analysis of the decision-maker's belief space. This is achieved by exploiting quantum interference effects as a way of both quantifying and explaining the decision-maker's uncertainty. We detail the main modules of the unified framework, the explanatory analysis method, and illustrate their application in situations violating the Sure Thing Principle.
Sequence to Point Learning Based on Bidirectional Dilated Residual Network for Non Intrusive Load Monitoring
Jia, Ziyue, Yang, Linfeng, Zhang, Zhenrong, Liu, Hui, Kong, Fannie
Non-Intrusive Load Monitoring (NILM) or Energy Disaggregation (ED), seeks to save energy by decomposing corresponding appliances power reading from an aggregate power reading of the whole house. It is a single channel blind source separation problem (SCBSS) and difficult prediction problem because it is unidentifiable. Recent research shows that deep learning has become a growing popularity for NILM problem. The ability of neural networks to extract load features is closely related to its depth. However, deep neural network is difficult to train because of exploding gradient, vanishing gradient and network degradation. To solve these problems, we propose a sequence to point learning framework based on bidirectional (non-casual) dilated convolution for NILM. To be more convincing, we compare our method with the state of art method--Seq2point (Zhang) directly and compare with existing algorithms indirectly via two same datasets and metrics. Experiments based on REDD and UK-DALE data sets show that our proposed approach is far superior to existing approaches in all appliances.