Regression
The leap to ordinal: detailed functional prognosis after traumatic brain injury with a flexible modelling approach
Bhattacharyay, Shubhayu, Milosevic, Ioan, Wilson, Lindsay, Menon, David K., Stevens, Robert D., Steyerberg, Ewout W., Nelson, David W., Ercole, Ari, investigators/participants, the CENTER-TBI
When a patient is admitted to the intensive care unit (ICU) after a traumatic brain injury (TBI), an early prognosis is essential for baseline risk adjustment and shared decision making. TBI outcomes are commonly categorised by the Glasgow Outcome Scale-Extended (GOSE) into 8, ordered levels of functional recovery at 6 months after injury. Existing ICU prognostic models predict binary outcomes at a certain threshold of GOSE (e.g., prediction of survival [GOSE>1] or functional independence [GOSE>4]). We aimed to develop ordinal prediction models that concurrently predict probabilities of each GOSE score. From a prospective cohort (n=1,550, 65 centres) in the ICU stratum of the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) patient dataset, we extracted all clinical information within 24 hours of ICU admission (1,151 predictors) and 6-month GOSE scores. We analysed the effect of 2 design elements on ordinal model performance: (1) the baseline predictor set, ranging from a concise set of 10 validated predictors to a token-embedded representation of all possible predictors, and (2) the modelling strategy, from ordinal logistic regression to multinomial deep learning. With repeated k-fold cross-validation, we found that expanding the baseline predictor set significantly improved ordinal prediction performance while increasing analytical complexity did not. Half of these gains could be achieved with the addition of 8 high-impact predictors (2 demographic variables, 4 protein biomarkers, and 2 severity assessments) to the concise set. At best, ordinal models achieved 0.76 (95% CI: 0.74-0.77) ordinal discrimination ability (ordinal c-index) and 57% (95% CI: 54%-60%) explanation of ordinal variation in 6-month GOSE (Somers' D). Our results motivate the search for informative predictors for higher GOSE and the development of ordinal dynamic prediction models.
Partial Maximum Correntropy Regression for Robust Trajectory Decoding from Noisy Epidural Electrocorticographic Signals
Li, Yuanhao, Chen, Badong, Wang, Gang, Yoshimura, Natsue, Koike, Yasuharu
The Partial Least Square Regression (PLSR) exhibits admirable competence for predicting continuous variables from inter-correlated brain recordings in the brain-computer interface. However, PLSR is in essence formulated based on the least square criterion, thus, being non-robust with respect to noises. The aim of this study is to propose a new robust implementation for PLSR. To this end, the maximum correntropy criterion (MCC) is used to propose a new robust variant of PLSR, called as Partial Maximum Correntropy Regression (PMCR). The half-quadratic optimization is utilized to calculate the robust projectors for the dimensionality reduction, and the regression coefficients are optimized by a fixed-point approach. We evaluate the proposed PMCR with a synthetic example and the public Neurotycho electrocorticography (ECoG) datasets. The extensive experimental results demonstrate that, the proposed PMCR can achieve better prediction performance than the conventional PLSR and existing variants with three different performance indicators in high-dimensional and noisy regression tasks. PMCR can suppress the performance degradation caused by the adverse noise, ameliorating the decoding robustness of the brain-computer interface.
Scalable Regularised Joint Mixture Models
Lartigue, Thomas, Mukherjee, Sach
In many applications, data can be heterogeneous in the sense of spanning latent groups with different underlying distributions. When predictive models are applied to such data the heterogeneity can affect both predictive performance and interpretability. Building on developments at the intersection of unsupervised learning and regularised regression, we propose an approach for heterogeneous data that allows joint learning of (i) explicit multivariate feature distributions, (ii) high-dimensional regression models and (iii) latent group labels, with both (i) and (ii) specific to latent groups and both elements informing (iii). The approach is demonstrably effective in high dimensions, combining data reduction for computational efficiency with a re-weighting scheme that retains key signals even when the number of features is large. We discuss in detail these aspects and their impact on modelling and computation, including EM convergence. The approach is modular and allows incorporation of data reductions and high-dimensional estimators that are suitable for specific applications. We show results from extensive simulations and real data experiments, including highly non-Gaussian data. Our results allow efficient, effective analysis of high-dimensional data in settings, such as biomedicine, where both interpretable prediction and explicit feature space models are needed but hidden heterogeneity may be a concern.
Machine Learning in Nuclear Physics
Boehnlein, Amber, Diefenthaler, Markus, Fanelli, Cristiano, Hjorth-Jensen, Morten, Horn, Tanja, Kuchera, Michelle P., Lee, Dean, Nazarewicz, Witold, Orginos, Kostas, Ostroumov, Peter, Pang, Long-Gang, Poon, Alan, Sato, Nobuo, Schram, Malachi, Scheinker, Alexander, Smith, Michael S., Wang, Xin-Nian, Ziegler, Veronique
Advances in machine learning methods provide tools that have broad applicability in scientific research. These techniques are being applied across the diversity of nuclear physics research topics, leading to advances that will facilitate scientific discoveries and societal applications. This Review gives a snapshot of nuclear physics research which has been transformed by machine learning techniques.
Improving forecasting by learning quantile functions
The quantile function is a mathematical function that takes a quantile (a percentage of a distribution, from 0 to 1) as input and outputs the value of a variable. It can answer questions like, "If I want to guarantee that 95% of my customers receive their orders within 24 hours, how much inventory do I need to keep on hand?" As such, the quantile function is commonly used in the context of forecasting questions. In practical cases, however, we rarely have a tidy formula for computing the quantile function. Instead, statisticians usually use regression analysis to approximate it for a single quantile level at a time.
Principal Component Regression -- Clearly Explained and Implemented
Principal Component Regression (PCR) is a regression technique that serves the same goal as standard linear regression -- model the relationship between a target variable and the predictor variables. The difference is that PCR uses the principal components as the predictor variables for regression analysis instead of the original features. The idea is that the smaller number of principal components represents most of the variability in the data and (presumptively) the relationship with the target variable. Therefore, instead of using all the original features for regression, we only utilize a subset of the principal components. Although the assumption of a relationship with the target variable does not always hold, it is often a reasonable enough approximation to yield good results.
Machine Learning in Python
This course will help you develop Machine Learning skills for solving real-life problems in the new digital world. Machine Learning combines computer science and statistics to analyze raw real-time data, identify trends, and make predictions. The participants will explore key techniques and tools to build Machine Learning solutions for businesses. You don't need to have any technical knowledge to learn this skill. You'll start with the History of Machine Learning; Difference Between Traditional Programming and Machine Learning; What does Machine Learning do; Definition of Machine Learning; Apply Apple Sorting Example Experiences; Role of Machine Learning; Machine Learning Key Terms; Basic Terminologies of Statistics; Descriptive Statistics-Types of Statistics; Types of Descriptive Statistics; What is Inferential Statistics; What is Analysis and its types; Probability and Real-life Examples; How Probability is a Process; Views of Probability; Base Theory of Probability.
The Directional Bias Helps Stochastic Gradient Descent to Generalize in Kernel Regression Models
Luo, Yiling, Huo, Xiaoming, Mei, Yajun
The Stochastic Gradient Descent (SGD) is a popular optimization algorithm that has a wide range of applications, including generalized linear model in statistics and deep Neural Network in machine learning. One main advantage of the SGD is the computational scalability due to low cost per iteration. Recent work also indicates that the SGD might also lead to outcomes that possess nice statistical properties under the linear regression framework, see [19]. In this paper, we study the statistical properties of the SGD under nonparametric regression models. We focus on the Reproducing Kernel Hilbert Space (RKHS) model, which is popular in both statistics and machine learning communities and is often simply referred to as the "kernel trick," see [2, 27].
Using Machine Learning and Regression Techniques to Rank Liberal Arts Colleges on Social Mobility and the Advancement of Underrepresented Groups - The College of Wooster
The purpose of higher education is to contribute to the advancement of society by graduating students of all backgrounds and providing them the skills and knowledge to be successful in life. One way colleges and universities can contribute to this purpose is to promote the goal of social mobility. The data shows that elite schools are enrolling mostly students from the highest income families. The current college ranking systems are highly weighted towards wealth, rather than social mobility and the advancement of all students. Therefore, there is a need to redirect the focus of college rankings to social mobility, not for the few but for all, especially those who have been traditionally excluded.