Statistical Learning
How Proximal gradient descent works part1(Machine Learning Optimization)
Abstract: In this letter, we focus on the problem of millimeter-Wave channels estimation in massive MIMO communication systems. Inspired by the sparsity of mmWave MIMO channel in the angular domain, we formulate the estimation problem as a sparse signal recovery problem. We propose a deep learning based trainable proximal gradient descent network (TPGD-Net) for mmWave channel estimation. Specifically, we unfold the iterative proximal gradient descent (PGD) algorithm into a layer-wise network. Different from the PGD algorithm, the gradient descent step size in TPGD-Net is set as a trainable parameter.
Linear Regression -- Occam's Razor of Predictive Machine Learning Modeling
Linear regression is when linear predictors (or independent variables) are used to predict a dependent variable. In this case, y is the dependent variable and x is the independent variable (c is a constant). The goal of a linear regression model is to determine the best coefficient (a in the example above) for x to most accurately predict y. Now let's generalize that example, which is also known as mulitple linear regression. In a multiple linear regression model, the goal is to find the line of best fit that describes the relationship between the dependent variable and multiple independent variables.
Dual-sPLS: a family of Dual Sparse Partial Least Squares regressions for feature selection and prediction with tunable sparsity; evaluation on simulated and near-infrared (NIR) data
Alsouki, Louna, Duval, Laurent, Marteau, Clément, Haddad, Rami El, Wahl, François
Relating a set of variables X to a response y is crucial in chemometrics. A quantitative prediction objective can be enriched by qualitative data interpretation, for instance by locating the most influential features. When high-dimensional problems arise, dimension reduction techniques can be used. Most notable are projections (e.g. Partial Least Squares or PLS ) or variable selections (e.g. lasso). Sparse partial least squares combine both strategies, by blending variable selection into PLS. The variant presented in this paper, Dual-sPLS, generalizes the classical PLS1 algorithm. It provides balance between accurate prediction and efficient interpretation. It is based on penalizations inspired by classical regression methods (lasso, group lasso, least squares, ridge) and uses the dual norm notion. The resulting sparsity is enforced by an intuitive shrinking ratio parameter. Dual-sPLS favorably compares to similar regression methods, on simulated and real chemical data. Code is provided as an open-source package in R: \url{https://CRAN.R-project.org/package=dual.spls}.
Weight Matrix Dimensionality Reduction in Deep Learning via Kronecker Multi-layer Architectures
Hogue, Jarom D., Kirby, Robert M., Narayan, Akil
Deep learning using neural networks is an effective technique for generating models of complex data. However, training such models can be expensive when networks have large model capacity resulting from a large number of layers and nodes. For training in such a computationally prohibitive regime, dimensionality reduction techniques ease the computational burden, and allow implementations of more robust networks. We propose a novel type of such dimensionality reduction via a new deep learning architecture based on fast matrix multiplication of a Kronecker product decomposition; in particular our network construction can be viewed as a Kronecker product-induced sparsification of an "extended" fully connected network. Analysis and practical examples show that this architecture allows a neural network to be trained and implemented with a significant reduction in computational time and resources, while achieving a similar error level compared to a traditional feedforward neural network.
From Risk Prediction to Risk Factors Interpretation. Comparison of Neural Networks and Classical Statistics for Dementia Prediction
It is proposed to investigate the onset of a disease D, based on several risk factors., with a specific interest in Alzheimer occurrence. For that purpose, two classes of techniques are available, whose properties are quite different in terms of interpretation, which is the focus of this paper: classical statistics based on probabilistic models and artificial intelligence (mainly neural networks) based on optimization algorithms. Both methods are good at prediction, with a preference for neural networks when the dimension of the potential predictors is high. But the advantage of the classical statistics is cognitive : the role of each factor is generally summarized in the value of a coefficient which is highly positive for a harmful factor, close to 0 for an irrelevant one, and highly negative for a beneficial one.
Investigation of a Machine learning methodology for the SKA pulsar search pipeline
Bhat, Shashank Sanjay, Prabu, Thiagaraj, Stappers, Ben, Ghalame, Atul, Saha, Snehanshu, Sudarshan, T. S. B, Hosenie, Zafiirah
The SKA pulsar search pipeline will be used for real time detection of pulsars. Modern radio telescopes such as SKA will be generating petabytes of data in their full scale of operation. Hence experience-based and datadriven algorithms are being investigated for applications such as candidate detection. Here we describe our findings from testing a state of the art object detection algorithm called Mask R-CNN to detect candidate signatures in the SKA pulsar search pipeline. We have trained the Mask R-CNN model to detect candidate images. A custom semi-auto annotation tool was developed and investigated to rapidly mark the regions of interest in large datasets. We have used a simulation dataset to train and build the candidate detection algorithm. A more detailed analysis is planned. The paper presents details of this initial investigation highlighting the future prospects.
Explainable Data Poison Attacks on Human Emotion Evaluation Systems based on EEG Signals
Zhang, Zhibo, Umar, Sani, Hammadi, Ahmed Y. Al, Yoon, Sangyoung, Damiani, Ernesto, Ardagna, Claudio Agostino, Bena, Nicola, Yeun, Chan Yeob
The major aim of this paper is to explain the data poisoning attacks using label-flipping during the training stage of the electroencephalogram (EEG) signal-based human emotion evaluation systems deploying Machine Learning models from the attackers' perspective. Human emotion evaluation using EEG signals has consistently attracted a lot of research attention. The identification of human emotional states based on EEG signals is effective to detect potential internal threats caused by insider individuals. Nevertheless, EEG signal-based human emotion evaluation systems have shown several vulnerabilities to data poison attacks. The findings of the experiments demonstrate that the suggested data poison assaults are model-independently successful, although various models exhibit varying levels of resilience to the attacks. In addition, the data poison attacks on the EEG signal-based human emotion evaluation systems are explained with several Explainable Artificial Intelligence (XAI) methods, including Shapley Additive Explanation (SHAP) values, Local Interpretable Model-agnostic Explanations (LIME), and Generated Decision Trees. And the codes of this paper are publicly available on GitHub.
Outlier-Based Domain of Applicability Identification for Materials Property Prediction Models
Panapitiya, Gihan, Saldanha, Emily
Machine learning models have been widely applied for material property prediction. However, practical application of these models can be hindered by a lack of information about how well they will perform on previously unseen types of materials. Because machine learning model predictions depend on the quality of the available training data, different domains of the material feature space are predicted with different accuracy levels by such models. The ability to identify such domains enables the ability to find the confidence level of each prediction, to determine when and how the model should be employed depending on the prediction accuracy requirements of different tasks, and to improve the model for domains with high errors. In this work, we propose a method to find domains of applicability using a large feature space and also introduce analysis techniques to gain more insight into the detected domains and subdomains.
MAFUS: a Framework to predict mortality risk in MAFLD subjects
Lofù, Domenico, Sorino, Paolo, Colafiglio, Tommaso, Bonfiglio, Caterina, Narducci, Fedelucio, Di Noia, Tommaso, Di Sciascio, Eugenio
Metabolic (dysfunction) associated fatty liver disease (MAFLD) establishes new criteria for diagnosing fatty liver disease independent of alcohol consumption and concurrent viral hepatitis infection. However, the long-term outcome of MAFLD subjects is sparse. Few articles are focused on mortality in MAFLD subjects, and none investigate how to predict a fatal outcome. In this paper, we propose an artificial intelligence-based framework named MAFUS that physicians can use for predicting mortality in MAFLD subjects. The framework uses data from various anthropometric and biochemical sources based on Machine Learning (ML) algorithms. The framework has been tested on a state-of-the-art dataset on which five ML algorithms are trained. Support Vector Machines resulted in being the best model. Furthermore, an Explainable Artificial Intelligence (XAI) analysis has been performed to understand the SVM diagnostic reasoning and the contribution of each feature to the prediction. The MAFUS framework is easy to apply, and the required parameters are readily available in the dataset.
Towards the design of user-centric strategy recommendation systems for collaborative Human-AI tasks
Dodeja, Lakshita, Tambwekar, Pradyumna, Hedlund-Botti, Erin, Gombolay, Matthew
Artificial Intelligence is being employed by humans to collaboratively solve complicated tasks for search and rescue, manufacturing, etc. Efficient teamwork can be achieved by understanding user preferences and recommending different strategies for solving the particular task to humans. Prior work has focused on personalization of recommendation systems for relatively well-understood tasks in the context of e-commerce or social networks. In this paper, we seek to understand the important factors to consider while designing user-centric strategy recommendation systems for decision-making. We conducted a human-subjects experiment (n=60) for measuring the preferences of users with different personality types towards different strategy recommendation systems. We conducted our experiment across four types of strategy recommendation modalities that have been established in prior work: (1) Single strategy recommendation, (2) Multiple similar recommendations, (3) Multiple diverse recommendations, (4) All possible strategies recommendations. While these strategy recommendation schemes have been explored independently in prior work, our study is novel in that we employ all of them simultaneously and in the context of strategy recommendations, to provide us an in-depth overview of the perception of different strategy recommendation systems. We found that certain personality traits, such as conscientiousness, notably impact the preference towards a particular type of system (p < 0.01). Finally, we report an interesting relationship between usability, alignment and perceived intelligence wherein greater perceived alignment of recommendations with one's own preferences leads to higher perceived intelligence (p < 0.01) and higher usability (p < 0.01).