Recent theoretical studies illustrated that kernel ridgeless regression can guarantee good generalization ability without an explicit regularization. In this paper, we investigate the statistical properties of ridgeless regression with random features and stochastic gradient descent. We explore the effect of factors in the stochastic gradient and random features, respectively. Specifically, random features error exhibits the double-descent curve. Motivated by the theoretical findings, we propose a tunable kernel algorithm that optimizes the spectral density of kernel during training. Our work bridges the interpolation theory and practical algorithm.

Principal Component Regression (PCR) is a regression technique that serves the same goal as standard linear regression -- model the relationship between a target variable and the predictor variables. The difference is that PCR uses the principal components as the predictor variables for regression analysis instead of the original features. The idea is that the smaller number of principal components represents most of the variability in the data and (presumptively) the relationship with the target variable. Therefore, instead of using all the original features for regression, we only utilize a subset of the principal components. Although the assumption of a relationship with the target variable does not always hold, it is often a reasonable enough approximation to yield good results.

A new approach (SMOTE-RF-SVM) is proposed to identify SARS-CoV-2 epitopes that can be used in vaccine design. Epitope candidates that can be used in vaccine design were determined using machine learning-based in silico and bioinformatics tools. In the unbalanced dataset, generating artificial data with the SMOTE technique increased the model performance. Nonallergic, high antigen (antigen score ≥1.0) and nontoxic 11 possible epitopes candidates were proposed. The search space for vaccine studies was narrowed by SMOTE-RF-SVM.

April 28, 2022 – Researchers have developed a convolutional neural network (CNN) model, a type of deep learning model, for classifying epileptic seizures that is designed to provide maximum accuracy and minor computational complexity, according to a study published in Soft Computing. The researchers developed their algorithm by integrating CNN architecture with a hierarchical attention mechanism, which was expected to enhance the model's performance. The model comprises three parts: a feature extraction layer, a hierarchical attention layer, and a classification layer. The model, which also uses a support vector machine (SVM) algorithm, analyzes a feature map obtained from the raw EEG signal and determines whether the EEGs it was taken from are "healthy" or "seizure."

This course will help you develop Machine Learning skills for solving real-life problems in the new digital world. Machine Learning combines computer science and statistics to analyze raw real-time data, identify trends, and make predictions. The participants will explore key techniques and tools to build Machine Learning solutions for businesses. You don't need to have any technical knowledge to learn this skill. You'll start with the History of Machine Learning; Difference Between Traditional Programming and Machine Learning; What does Machine Learning do; Definition of Machine Learning; Apply Apple Sorting Example Experiences; Role of Machine Learning; Machine Learning Key Terms; Basic Terminologies of Statistics; Descriptive Statistics-Types of Statistics; Types of Descriptive Statistics; What is Inferential Statistics; What is Analysis and its types; Probability and Real-life Examples; How Probability is a Process; Views of Probability; Base Theory of Probability.

Identification of unknown physical processes and parameters of groundwater contaminant sources is a challenging task due to their ill-posed and non-unique nature. Numerous works have focused on determining nonlinear physical processes through model selection methods. However, identifying corresponding nonlinear systems for different physical phenomena using numerical methods can be computationally prohibitive. With the advent of machine learning (ML) algorithms, more efficient surrogate models based on neural networks (NNs) have been developed in various disciplines. In this work, a theory-guided U-net (TgU-net) framework is proposed for surrogate modeling of three-dimensional (3D) groundwater contaminant problems in order to efficiently elucidate their involved processes and unknown parameters. In TgU-net, the underlying governing equations are embedded into the loss function of U-net as soft constraints. For the considered groundwater contaminant problem, sorption is considered to be a potential process of an uncertain type, and three equilibrium sorption isotherm types (i.e., linear, Freundlich, and Langmuir) are considered. Different from traditional approaches in which one model corresponds to one equation, these three sorption types are modeled through only one TgU-net surrogate. The three mentioned sorption terms are integrated into one equation by assigning indicators. Accurate predictions illustrate the satisfactory generalizability and extrapolability of the constructed TgU-net. Furthermore, based on the constructed TgU-net surrogate, a data assimilation method is employed to identify the physical process and parameters simultaneously. This work shows the possibility of governing equation discovery of physical problems that contain multiple and even uncertain processes by using deep learning and data assimilation methods.

With the increase of data volume and the continuous development in deep learning, although more and more traditional machine learning techniques are outperformed by artificial neural networks, tree-based methods are still popular. Random forest (Breiman, 2001) is commonly used as a benchmark to evaluate the performance of nonparametric models, while XGBoost (Chen and Guestrin, 2016) performs well in Kaggle competitions and often competes with artificial neural networks. Also, instead of relying on a specific method, people prefer to make decisions based on a combination of multiple models, which shows a better performance than a single one. Therefore, identifying the importance of each model by weights assignment is critical.

It is a computationally feasible yet statistically principled alternative to likelihood-based learning when the likelihood function is too expensive to compute and thus has wide applicability. In this paper we focus on the statistical side of contrastive learning rather than on a particular application domain. We first explain the principles of contrastive learning and then show how we can use it to solve a diverse set of difficult statistical tasks, namely (1) parameter estimation for energy-based models, (2) Bayesian inference for simulator-based models, as well as (3) experimental design. We will introduce these problems in detail and explain when and why likelihood-based learning becomes computationally infeasible. The three problems involve different models as well as tasks--inference versus experimental design.

In statistics and machine learning, it is common to optimize an objective function that is a finitesum. SMD efficiently optimizes such an objective by using a subset of data to do one step update of the variable/parameter. Further adopting the variance reduction technique to SMD, we get the VRSMD algorithm that enjoys fast convergence [1], [2]. The implicit regularization is a relatively new concept [3] that explains why a result of an algorithm generalizes well in some overparameterized models [3], [4]. It refers to the fact that an algorithm can automatically select a minimum norm solution, which is not explicitly induced by the objective function. There are works on implicit regularization for Gradient Descent [5]- [8], Stochastic Gradient Descent [9]-[12], and Stochastic Mirror Descent [13]. Considering the computational advantage of VRSMD compared to all the algorithms above, it would be even better if VRSMD also has the useful implicit regularization property. From technical point of view, our work contains the following two results: In linear regression (including underfitting and overfitting), we show that the solution sequence of VRSMD converges to the minimum mirror interpolant, which is the implicit regularization property of VRSMD, and we also specify the convergence rate.

The Stochastic Gradient Descent (SGD) is a popular optimization algorithm that has a wide range of applications, including generalized linear model in statistics and deep Neural Network in machine learning. One main advantage of the SGD is the computational scalability due to low cost per iteration. Recent work also indicates that the SGD might also lead to outcomes that possess nice statistical properties under the linear regression framework, see [19]. In this paper, we study the statistical properties of the SGD under nonparametric regression models. We focus on the Reproducing Kernel Hilbert Space (RKHS) model, which is popular in both statistics and machine learning communities and is often simply referred to as the "kernel trick," see [2, 27].