Statistical Learning
5 Papers to Read on Dimensionality Reduction Method in 2022
Abstract: Dimension reduction is an important tool for analyzing high-dimensional data. The predictor envelope is a method of dimension reduction for regression that assumes certain linear combinations of the predictors are immaterial to the regression. The method can result in substantial gains in estimation efficiency and prediction accuracy over traditional maximum likelihood and least squares estimates. While predictor envelopes have been developed and studied for independent data, no work has been done adapting predictor envelopes to spatial data. In this work, the predictor envelope is adapted to a popular spatial model to form the spatial predictor envelope (SPE).
Apple ML Research: Private On-Device Machine Learning to Recognize People in Photos
People use Photos (on iOS, iPad OS, and macOS) to browse, search, and relive life's memories with their friends and family. Photos curate and organize photographs, live images, and videos using several machine learning algorithms that operate secretly on the device. A vital component of this goal is an algorithm that recognizes people based on their appearance. In a variety of ways, photos rely on personal information. A user can scroll upon an image, tap on the circle representing the person who has been recognized in that image, and later pivot to browse their library for photographs including that person, as shown in Figure 1A.
S-DABT: Schedule and Dependency-Aware Bug Triage in Open-Source Bug Tracking Systems
Jahanshahi, Hadi, Cevik, Mucahit
Fixing bugs in a timely manner lowers various potential costs in software maintenance. However, manual bug fixing scheduling can be time-consuming, cumbersome, and error-prone. In this paper, we propose the Schedule and Dependency-aware Bug Triage (S-DABT), a bug triaging method that utilizes integer programming and machine learning techniques to assign bugs to suitable developers. Unlike prior works that largely focus on a single component of the bug reports, our approach takes into account the textual data, bug fixing costs, and bug dependencies. We further incorporate the schedule of developers in our formulation to have a more comprehensive model for this multifaceted problem. As a result, this complete formulation considers developers' schedules and the blocking effects of the bugs while covering the most significant aspects of the previously proposed methods. Our numerical study on four open-source software systems, namely, EclipseJDT, LibreOffice, GCC, and Mozilla, shows that taking into account the schedules of the developers decreases the average bug fixing times. We find that S-DABT leads to a high level of developer utilization through a fair distribution of the tasks among the developers and efficient use of the free spots in their schedules. Via the simulation of the issue tracking system, we also show how incorporating the schedule in the model formulation reduces the bug fixing time, improves the assignment accuracy, and utilizes the capability of each developer without much comprising in the model run times. We find that S-DABT decreases the complexity of the bug dependency graph by prioritizing blocking bugs and effectively reduces the infeasible assignment ratio due to bug dependencies. Consequently, we recommend considering developers' schedules while automating bug triage.
Automated Learning of Interpretable Models with Quantified Uncertainty
Bomarito, G. F., Leser, P. E., Strauss, N. C. M, Garbrecht, K. M., Hochhalter, J. D.
Machine learning (ML) has become ubiquitous in scientific disciplines. In some applications, accurate data-driven predictions are all that is required; however, in many others, interpretability and explainability of the model is equally important. Interpretability and explainability can provide justification for decisions, promote scientific discovery and ultimately lead to better control/improvement of models [1, 2]. In a complementary fashion, ML models can provide further insight by conveying their level of uncertainty in predictions [3]. Especially in cases of low risk tolerance this type of insight is crucial for building trust in ML models [4]. Rather than focus on black-box ML methods (e.g., neural networks or Gaussian process regression) combined with post hoc explainability tools, the current work focuses on inherently interpretable methods. Interpretable ML methods can be competitive with black-box ML in terms of accuracy and do not require a separate explainability toolkit [4, 5]. Symbolic regression is one such inherently interpretable form of ML wherein an analytic equation is produced that best models input data.
Distributed learning optimisation of Cox models can leak patient data: Risks and solutions
Brink, Carsten, Hansen, Christian Rønn, Field, Matthew, Price, Gareth, Thwaites, David, Sarup, Nis, Bernchou, Uffe, Holloway, Lois
Medical data are often highly sensitive, and frequently there are missing data. Due to the data's sensitive nature, there is an interest in creating modelling methods where the data are kept in each local centre to preserve their privacy, but yet the model can be trained on and learn from data across multiple centres. Such an approach might be distributed machine learning (federated learning, collaborative learning) in which a model is iteratively calculated based on aggregated local model information from each centre. However, even though no specific data are leaving the centre, there is a potential risk that the exchanged information is sufficient to reconstruct all or part of the patient data, which would hamper the safety-protecting rationale idea of distributed learning. This paper demonstrates that the optimisation of a Cox survival model can lead to patient data leakage. Following this, we suggest a way to optimise and validate a Cox model that avoids these problems in a secure way. The feasibility of the suggested method is demonstrated in a provided Matlab code that also includes methods for handling missing data.
Local Random Feature Approximations of the Gaussian Kernel
Wacker, Jonas, Filippone, Maurizio
A fundamental drawback of kernel-based statistical models is their limited scalability to large data sets, which requires resorting to approximations. In this work, we focus on the popular Gaussian kernel and on techniques to linearize kernel-based models by means of random feature approximations. In particular, we do so by studying a less explored random feature approximation based on Maclaurin expansions and polynomial sketches. We show that such approaches yield poor results when modelling high-frequency data, and we propose a novel localization scheme that improves kernel approximations and downstream performance significantly in this regime. We demonstrate these gains on a number of experiments involving the application of Gaussian process regression to synthetic and real-world data of different data sizes and dimensions.
Robust online joint state/input/parameter estimation of linear systems
Brouillon, Jean-Sébastien, Moffat, Keith, Dörfler, Florian, Ferrari-Trecate, Giancarlo
This paper presents a method for jointly estimating the state, input, and parameters of linear systems in an online fashion. The method is specially designed for measurements that are corrupted with non-Gaussian noise or outliers, which are commonly found in engineering applications. In particular, it combines recursive, alternating, and iteratively-reweighted least squares into a single, one-step algorithm, which solves the estimation problem online and benefits from the robustness of least-deviation regression methods. The convergence of the iterative method is formally guaranteed. Numerical experiments show the good performance of the estimation algorithm in presence of outliers and in comparison to state-of-the-art methods.
Difference between R square and Adjusted R square?
The R-square is a measure of how well the linear regression model fits the observed data. It is calculated by squaring the correlation coefficient and dividing by the standard deviation of errors. It is the square of the correlation coefficient divided by its standard deviation (r2/s2). The R-square value of 1 indicates that the model explains 100% of the variation in Y. The R-square values greater than 1 indicate that the model explains more than 100% of the variation in Y.
Does predict function work in parallel when predicting k-nearest neighbour?
I have a k-nearest neighbour classifier which I have trained with fitcknn. I am wondering, when predicting labels on the model using predicit does it work in parallel? I have tested using predict in a for loop and parfor loop. The simple for loop performs a bit faster which makes me think there is some optimisation and built in parallelisation that the predict function is taking advantage of. However, the documentation makes no reference to this, and I thought MATLAB always runs in a single thread unless specifically using a parallel pool?