In the realm of artificial intelligence (AI), not all machine learning approaches are considered equal. This is an important consideration in fields such as neuroscience, medicine, biotechnology, life sciences, health care, genomics, pharmaceuticals, and other industries where accuracy may directly impact human health and safety. In a new study published earlier this month in Nature Communications, researchers at Georgia State University show the advantages of deep learning (DL) over standard machine learning (SML) in brain research. "Our findings highlight the presence of nonlinearities in neuroimaging data that DL can exploit to generate superior task-discriminative representations for characterizing the human brain," wrote the paper's lead author Anees Abrol along with Sergey Plis, Vince Calhoun, Yuhui Du, Rogers Silva, Mustafa Salman, and Zening Fu. In standard machine learning, predictions are a result of processing prediction functions via inference rules, and the decision boundaries are determined in the "native, kernel-transformed, or feature-engineered input spaces."

The plug-in estimator of the Wasserstein distance is known to be conservative, however its usefulness is severely limited when the distributions are similar as its bias does not decay to zero with the true Wasserstein distance. We propose a linear combination of plug-in estimators for the squared 2-Wasserstein distance with a reduced bias that decays to zero with the true distance. The new estimator is provably conservative provided one distribution is appropriately overdispersed with respect the other, and is unbiased when the distributions are equal. We apply it to approximately bound from above the 2-Wasserstein distance between the target and current distribution in Markov chain Monte Carlo, running multiple identically distributed chains which start, and remain, overdispersed with respect to the target. Our bound consistently outperforms the current state-of-the-art bound, which uses coupling, improving mixing time bounds by up to an order of magnitude.

Most convergence guarantees for stochastic gradient descent with momentum (SGDm) rely on iid sampling. Yet, SGDm is often used outside this regime, in settings with temporally correlated input samples such as continual learning and reinforcement learning. Existing work has shown that SGDm with a decaying step-size can converge under Markovian temporal correlation. In this work, we show that SGDm under covariate shift with a fixed step-size can be unstable and diverge. In particular, we show SGDm under covariate shift is a parametric oscillator, and so can suffer from a phenomenon known as resonance. We approximate the learning system as a time varying system of ordinary differential equations, and leverage existing theory to characterize the system's divergence/convergence as resonant/nonresonant modes. The theoretical result is limited to the linear setting with periodic covariate shift, so we empirically supplement this result to show that resonance phenomena persist even under non-periodic covariate shift, nonlinear dynamics with neural networks, and optimizers other than SGDm.

Stragglers, Byzantine workers, and data privacy are the main bottlenecks in distributed cloud computing. Some prior works proposed coded computing strategies to jointly address all three challenges. They require either a large number of workers, a significant communication cost or a significant computational complexity to tolerate Byzantine workers. Much of the overhead in prior schemes comes from the fact that they tightly couple coding for all three problems into a single framework. In this paper, we propose Adaptive Verifiable Coded Computing (AVCC) framework that decouples the Byzantine node detection challenge from the straggler tolerance. AVCC leverages coded computing just for handling stragglers and privacy, and then uses an orthogonal approach that leverages verifiable computing to mitigate Byzantine workers. Furthermore, AVCC dynamically adapts its coding scheme to trade-off straggler tolerance with Byzantine protection. We evaluate AVCC on a compute-intensive distributed logistic regression application. Our experiments show that AVCC achieves up to $4.2\times$ speedup and up to $5.1\%$ accuracy improvement over the state-of-the-art Lagrange coded computing approach (LCC). AVCC also speeds up the conventional uncoded implementation of distributed logistic regression by up to $7.6\times$, and improves the test accuracy by up to $12.1\%$.

We present a clustering method and provide a theoretical analysis and an explanation to a phenomenon encountered in the applied statistical literature since the 1990's. This phenomenon is the natural adaptability of the order when using a clustering method derived from the famous EM algorithm. We define a new statistic, the relative entropic order, that represents the number of clumps in the target distribution. We prove in particular that the empirical version of this relative entropic order is consistent. Our approach is easy to implement and has a high potential of applications. Perspectives of this works are algorithmic and theoretical, with possible natural extensions to various cases such as dependent or multidimensional data.

The field of extrasolar planet studies is undergoing a seismic shift. To date, 4,940 exoplanets have been confirmed in 3,711 planetary systems, with another 8,709 candidates awaiting confirmation. With so many planets available for study and improvements in telescope sensitivity and data analysis, the focus is transitioning from discovery to characterization. Instead of simply looking for more planets, astrobiologists will examine "potentially-habitable" worlds for potential "biosignatures." This refers to the chemical signatures associated with life and biological processes, one of the most important of which is water. As the only known solvent that life (as we know it) cannot exist, water is considered the divining rod for finding life.

Linear regression finds the optimal linear relationship between independent variables and dependent variables, thus makes prediction accordingly. The simplest form is y b0 b1x. When there is only one input feature, linear regression model fits the line in a 2 dimensional space, in order to minimize the residuals between predicted values and actual values. The common cost function to measure the magnitude of residuals is residual sum of squared (RSS). As more features are introduced, simple linear regression evolves into multiple linear regression y b0 b1x1 b2x2 … bnxn. Feel free to check out my article if you want the specific guide to simple linear regression model.

Scikit-learn is one of the most popular Python libraries for Machine Learning. It provides models, datasets, and other useful functions. In this article, I will describe the most popular techniques provided by scikit-learn for Model Evaluation. Model Evaluation permits us to evaluate the performance of a model, and compare different models, to choose the best one to send into production. There are different techniques for Model Evaluation, which depend on the specific task we want to solve.

Identifying the causal structure of systems with multiple dynamic elements is critical to several scientific disciplines. The conventional approach is to conduct statistical tests of causality, for example with Granger Causality, between observed signals that are selected a priori. Here it is posited that, in many systems, the causal relations are embedded in a latent space that is expressed in the observed data as a linear mixture. A technique for blindly identifying the latent sources is presented: the observations are projected into pairs of components -- driving and driven -- to maximize the strength of causality between the pairs. This leads to an optimization problem with closed form expressions for the objective function and gradient that can be solved with off-the-shelf techniques. After demonstrating proof-of-concept on synthetic data with known latent structure, the technique is applied to recordings from the human brain and historical cryptocurrency prices. In both cases, the approach recovers multiple strong causal relationships that are not evident in the observed data. The proposed technique is unsupervised and can be readily applied to any multiple time series to shed light on the causal relationships underlying the data.

We demonstrate and discuss nonasymptotic bounds in probability for the cost of a regression scheme with a general loss function from the perspective of the Rademacher theory, and for the optimality with respect to the average $L^{2}$-distance to the underlying conditional expectations of least squares regression outcomes from the perspective of the Vapnik-Chervonenkis theory. The results follow from an analysis involving independent but possibly nonstationary training samples and can be extended, in a manner that we explain and illustrate, to relevant cases in which the training sample exhibits dependence.