Hamiltonian Monte Carlo (HMC) is a a Markov Chain Monte Carlo (MCMC) method for sampling from complex probability measures whose normalizing constant is unknown. It originated under the name "Hybrid Monte Carlo" in Duane et al. (1987) in statistical physics. The'target' measure that HMC can sample from has the form dπ(q) exp( Φ(q)) dq, (1) i.e. the target measure has a positive density with respect to the Lebesgue measure dq. The means that the density of π w.r.t.

Semi-continuous data comes from a distribution that is a mixture of the point mass at zero and a continuous distribution with support on the positive real line. A clear example is the daily rainfall data. In this paper, we present a novel algorithm to estimate the density function for semi-continuous data using the principle of maximum entropy. Unlike existing methods in the literature, our algorithm needs only the sample values of the constraint functions in the entropy maximization problem and does not need the entire sample. Using simulations, we show that the estimate of the entropy produced by our algorithm has significantly less bias compared to existing methods. An application to the daily rainfall data is provided.

Federated Learning (FL) is a distributed learning paradigm which scales on-device learning collaboratively and privately. Standard FL algorithms such as Federated Averaging (FedAvg) are primarily geared towards smooth unconstrained settings. In this paper, we study the Federated Composite Optimization (FCO) problem, where the objective function in FL includes an additive (possibly) non-smooth component. Such optimization problems are fundamental to machine learning and arise naturally in the context of regularization (e.g., sparsity, low-rank, monotonicity, and constraint). To tackle this problem, we propose different primal/dual averaging approaches and study their communication and computation complexities. Of particular interest is Federated Dual Averaging (FedDualAvg), a federated variant of the dual averaging algorithm. FedDualAvg uses a novel double averaging procedure, which involves gradient averaging step in standard dual averaging and an average of client updates akin to standard federated averaging. Our theoretical analysis and empirical experiments demonstrate that FedDualAvg outperforms baselines for FCO.

We introduce a new scalable approximation for Gaussian processes with provable guarantees which hold simultaneously over its entire parameter space. Our approximation is obtained from an improved sample complexity analysis for sparse spectrum Gaussian processes (SSGPs). In particular, our analysis shows that under a certain data disentangling condition, an SSGP's prediction and model evidence (for training) can well-approximate those of a full GP with low sample complexity. We also develop a new auto-encoding algorithm that finds a latent space to disentangle latent input coordinates into well-separated clusters, which is amenable to our sample complexity analysis. We validate our proposed method on several benchmarks with promising results supporting our theoretical analysis.

A model involving Gaussian processes (GPs) is introduced to simultaneously handle multi-task learning, clustering, and prediction for multiple functional data. This procedure acts as a model-based clustering method for functional data as well as a learning step for subsequent predictions for new tasks. The model is instantiated as a mixture of multi-task GPs with common mean processes. A variational EM algorithm is derived for dealing with the optimisation of the hyper-parameters along with the hyper-posteriors' estimation of latent variables and processes. We establish explicit formulas for integrating the mean processes and the latent clustering variables within a predictive distribution, accounting for uncertainty on both aspects. This distribution is defined as a mixture of cluster-specific GP predictions, which enhances the performances when dealing with group-structured data. The model handles irregular grid of observations and offers different hypotheses on the covariance structure for sharing additional information across tasks. The performances on both clustering and prediction tasks are assessed through various simulated scenarios and real datasets. The overall algorithm, called MagmaClust, is publicly available as an R package.

We present a method for hierarchical clustering of directed acyclic graphs and other strictly partially ordered data that preserves the data structure. In particular, if we have $a

Support Vector Machines are very versatile Machine Learning algorithms. The main reason for their popularity is for their ability to perform both linear and non-linear classification and regression using what is known as the kernel trick; if you don't know what that is, don't worry. So, without further ado, let's dive in! You may be wondering why you need to have another one in your toolkit! Here we see three potential decision boundaries for classifying the data: H1, H2, and H3. First off, H1 does not separate the classes at all, so it is not a good hyperplane.

The use of cumulative incidence functions for characterizing the risk of one type of event in the presence of others has become increasingly popular over the past decade. The problems of modeling, estimation and inference have been treated using parametric, nonparametric and semi-parametric methods. Efforts to develop suitable extensions of machine learning methods, such as regression trees and related ensemble methods, have begun only recently. In this paper, we develop a novel approach to building regression trees for estimating cumulative incidence curves in a competing risks setting. The proposed methods employ augmented estimators of the Brier score risk as the primary basis for building and pruning trees.

Curve fitting is a type of optimization that finds an optimal set of parameters for a defined function that best fits a given set of observations. Unlike supervised learning, curve fitting requires that you define the function that maps examples of inputs to outputs. The mapping function, also called the basis function can have any form you like, including a straight line (linear regression), a curved line (polynomial regression), and much more. This provides the flexibility and control to define the form of the curve, where an optimization process is used to find the specific optimal parameters of the function. In this tutorial, you will discover how to perform curve fitting in Python.

I know you're itching to see something real. So let's dive into an example. I will say that the code shown here is from a Mac. It'll work on Linux with extremely minor changes, and it can be made to work on Windows with some minor changes. For brevity, I'll just assert that everything I'm showing here is on a *nix terminal, specifically on a Mac. What I'm going to do is first demonstrate local development.