In this paper, we aimed at reviewing several different approaches present today in the search for more accurate diagnostic and treatment management in mental healthcare. Our focus is on mood disorders, and in particular on the major depressive disorder (MDD). We are reviewing and discussing findings based on neuroimaging studies (MRI and fMRI) first to get the impression of the body of knowledge about the anatomical and functional differences in depression. Then, we are focusing on less expensive data-driven approach, applicable for everyday clinical practice, in particular, those based on electroencephalographic (EEG) recordings. Among those studies utilizing EEG, we are discussing a group of applications used for detecting of depression based on the resting state EEG (detection studies) and interventional studies (using stimulus in their protocols or aiming to predict the outcome of therapy). We conclude with a discussion and review of guidelines to improve the reliability of developed models that could serve improvement of diagnostic of depression in psychiatry.

We present a scalable nonparametric Bayesian method to perform network reconstruction from observed functional behavior, that at the same time infers the communities present in the network. We show that the joint reconstruction with community detection has a synergistic effect, where the edge correlations used to inform the existence of communities are inherently also used to improve the accuracy of the reconstruction, which in turn can better inform the uncovering of communities. We illustrate the use of our method with observations arising from epidemic models and the Ising model, both on synthetic and empirical networks, as well as on data containing only functional information.

The paper deals with learning probability distributions of observed data by artificial neural networks. We suggest a so-called gradient conjugate prior (GCP) update appropriate for neural networks, which is a modification of the classical Bayesian update for conjugate priors. We establish a connection between the gradient conjugate prior update and the maximization of the log-likelihood of the predictive distribution. Unlike for the Bayesian neural networks, we use deterministic weights of neural networks, but rather assume that the ground truth distribution is normal with unknown mean and variance and learn by the neural networks the parameters of a prior (normal-gamma distribution) for these unknown mean and variance. The update of the parameters is done, using the gradient that, at each step, directs towards minimizing the Kullback--Leibler divergence from the prior to the posterior distribution (both being normal-gamma). We obtain a corresponding dynamical system for the prior's parameters and analyze its properties. In particular, we study the limiting behavior of all the prior's parameters and show how it differs from the case of the classical full Bayesian update. The results are validated on synthetic and real world data sets.

In this paper we contribute a novel algorithm family, which generalizes many unsupervised techniques including unnormalized and energy models, and allows to infer different statistical modalities (e.g.~data likelihood and ratio between densities) from data samples. The proposed unsupervised technique Probabilistic Surface Optimization (PSO) views a neural network (NN) as a flexible surface which can be pushed according to loss-specific virtual stochastic forces, where a dynamical equilibrium is achieved when the point-wise forces on the surface become equal. Concretely, the surface is pushed up and down at points sampled from two different distributions, with overall up and down forces becoming functions of these two distribution densities and of force intensity magnitudes defined by loss of a particular PSO instance. The eventual force equilibrium upon convergence enforces the NN to be equal to various statistical functions depending on the used magnitude functions, such as data density. Furthermore, this dynamical-statistical equilibrium is extremely intuitive and useful, providing many implications and possible usages in probabilistic inference. Further, we provide new PSO-based approaches as demonstration of PSO exceptional usability. We also analyze PSO convergence and optimization stability, and relate them to the gradient similarity function over NN input space. Further, we propose new ways to improve the above stability. Finally, we present new instances of PSO, termed PSO-LDE, for data density estimation on logarithmic scale and also provide a new NN block-diagonal architecture for increased surface flexibility, which significantly improves estimation accuracy. Both PSO-LDE and the new architecture are combined together as a new density estimation technique. In our experiments we demonstrate this technique to produce highly accurate density estimation for 20D data.

Complex networks can be modeled as a probabilistic graphical model, where the interactions between binary variables, "spins", on nodes are described by a coupling matrix that is inferred from observations. The inverse statistical problem of finding direct interactions is difficult, especially for large systems, because of the exponential growth in the possible number of states and the possible number of networks. In the context of the experimental sciences, well-controlled perturbations can be applied to a system, shedding light on the internal structure of the network. Therefore, we propose a method to improve the accuracy and efficiency of inference by iteratively applying perturbations to a network that are advantageous under a Bayesian framework. The spectrum of the empirical Fisher information can be used as a measure for the difficulty of the inference during the training process. We significantly improve the accuracy and efficiency of inference in medium-sized networks based on this strategy with a reasonable number of experimental queries. Our method could be powerful in the analysis of complex networks as well as in the rational design of experiments.

Linear regression is the most basic and the most widely used technique in machine learning; yet for all its simplicity, studying it can unlock some of the most important concepts in statistics. If you have a basic undestanding of linear regression expressed as $ \hat{Y} \theta_0 \theta_1X$, but don't have a background in statistics and find statements like "ridge regression is equivalent to the maximum a posteriori (MAP) estimate with a zero-mean Gaussian prior" bewildering, then this post is for you. With a superficial goal of understanding that somewhat obtuse statement, its main objective is to explore the topic, starting from the standard formulation of linear regression, moving on to the probabilistic approach (maximum likelihood formulation) and from there to Bayesian linear regression. I'll use the $\theta$ character throughout to refer to the coefficients (weights) of a regression model, either explicitly broken out as $\theta_0$ and $\theta_1$ for intercept and slope respectively, or just $\theta$ referring to the vector of coefficients. I'll usually use the expression $\theta Tx_i$ for the prediction a model gives at $x_i$, the assumption being that a 1 has been added to the vector of values at $x_i$. 1 In the single predictor case, we know that the least squares fit is the line that minimizes the sum of the squared distances between observed data and predicted values, i.e. it minimizes the Residual Sum of Squares (RSS): These residuals are pretty important in how we reason about our model.

The 2020 Democratic candidates are out of the gate and the pollsters have the call! Bernie Sanders is leading by two lengths with Kamala Harris and Elizabeth Warren right behind, but Cory Booker and Beto O'Rourke are coming on fast! The political horse-race season is upon us and I bet I know what you are thinking: "Stop!" Every election we complain about horse-race coverage and every election we stay glued to it all the same. The problem with this kind of coverage is not that it's unimportant.

Scalable Data Augmentation (SDA) provides a framework for training deep neural networks (DNNs). Our methodology exploits auxiliary hidden units which are designed to avoid backtracking and traverse local modes in an efficient way. This allows us to exploit recent advantages in high performance computing such as scalable linear algebra (CUDA, XLA). We show how to implement standard activation and objective functions, including ReLU (Polson and Ročková, 2018), logit (Zhou et al., 2012) and SVM (Mallick et al., 2005) are all available as data augmentation schemes. Data augmentation strategies are commonplace in statistical applications such as EM, ECM and MM algorithms, as they accelerate convergence and can use Nesterov acceleration (Nesterov, 1983).

This note is concerned with an accurate and computationally efficient variational bayesian treatment of mixed-effects modelling. We focus on group studies, i.e. empirical studies that report multiple measurements acquired in multiple subjects. When approached from a bayesian perspective, such mixed-effects models typically rely upon a hierarchical generative model of the data, whereby both within- and between-subject effects contribute to the overall observed variance. The ensuing VB scheme can be used to assess statistical significance at the group level and/or to capture inter-individual differences. Alternatively, it can be seen as an adaptive regularization procedure, which iteratively learns the corresponding within-subject priors from estimates of the group distribution of effects of interest (cf. so-called "empirical bayes" approaches). We outline the mathematical derivation of the ensuing VB scheme, whose open-source implementation is available as part the VBA toolbox.

A detailed understanding of wind turbine performance status classification can improve operations and maintenance in the wind energy industry. Due to different engineering properties of wind turbines, the standard supervised learning models used for classification do not generalize across data sets obtained from different wind sites. We propose two methods to deal with the transferability of the trained models: first, data normalization in the form of power curve alignment, and second, a robust method based on convolutional neural networks and feature-space extension. We demonstrate the success of our methods on real-world data sets with industrial applications. Keywords: Machine learning, classification, generalization, CNN, wind turbine, wind energy 1. Introduction Classification of operational status is an important step for performance analysis of wind farms from data of SCADA (Supervisory Control and Data Acquisition) type.