Goto

Collaborating Authors

 Accuracy


Fast cross-validation for multi-penalty ridge regression

arXiv.org Machine Learning

Prediction based on multiple high-dimensional data types needs to account for the potentially strong differences in predictive signal. Ridge regression is a simple, yet versatile and interpretable model for high-dimensional data that has challenged the predictive performance of many more complex models and learners, in particular in dense settings. Moreover, it allows using a specific penalty per data type to account for differences between those. Then, the largest challenge for multi-penalty ridge is to optimize these penalties efficiently in a cross-validation (CV) setting, in particular for GLM and Cox ridge regression, which require an additional loop for fitting the model by iterative weighted least squares (IWLS). Our main contribution is a computationally very efficient formula for the multi-penalty, sample-weighted hat-matrix, as used in the IWLS algorithm. As a result, nearly all computations are in the low-dimensional sample space. We show that our approach is several orders of magnitude faster than more naive ones. We developed a very flexible framework that includes prediction of several types of response, allows for unpenalized covariates, can optimize several performance criteria and implements repeated CV. Moreover, extensions to pair data types and to allow a preferential order of data types are included and illustrated on several cancer genomics survival prediction problems. The corresponding R-package, multiridge, serves as a versatile standalone tool, but also as a fast benchmark for other more complex models and multi-view learners.


Using AI to predict retinal disease progression

#artificialintelligence

However, we know there's still a lot to do โ€“ this work does not yet represent a product that could be implemented in routine clinical practice. While our model can make better predictions than clinical experts, there are many other factors to consider for such systems to be impactful in a clinical setting. While the model was trained and evaluated on a population representative of the largest eye hospital in Europe, additional work would be needed to evaluate performance in the context of very different demographics. A recent study examining the use of a different AI system in a clinical setting highlighted just some of the sociotechnical issues for such systems in practice. Another difficult point to contend with is that any prediction system will have a certain rate of false positives: that is, when a patient is found to have a condition, or predicted to develop one, that they don't actually have.


Sparse Methods for Automatic Relevance Determination

arXiv.org Machine Learning

This work considers methods for imposing sparsity in Bayesian regression with applications in nonlinear system identification. We first review automatic relevance determination (ARD) and analytically demonstrate the need to additional regularization or thresholding to achieve sparse models. We then discuss two classes of methods, regularization based and thresholding based, which build on ARD to learn parsimonious solutions to linear problems. In the case of orthogonal covariates, we analytically demonstrate favorable performance with regards to learning a small set of active terms in a linear system with a sparse solution. Several example problems are presented to compare the set of proposed methods in terms of advantages and limitations to ARD in bases with hundreds of elements. The aim of this paper is to analyze and understand the assumptions that lead to several algorithms and to provide theoretical and empirical results so that the reader may gain insight and make more informed choices regarding sparse Bayesian regression.


Optimal survival trees ensemble

arXiv.org Machine Learning

Recent studies have adopted an approach of selecting accurate and diverse trees based on individual or collective performance within an ensemble for classification and regression problems. This work follows in the wake of these investigations and considers the possibility of growing a forest of optimal survival trees. Initially, a large set of survival trees are grown using the method of random survival forest. The grown trees are then ranked from smallest to highest value of their prediction error using out-of-bag observations for each respective survival tree. The top ranked survival trees are then assessed for their collective performance as an ensemble. This ensemble is initiated with the survival tree which stands first in rank, then further trees are tested one by one by adding them to the ensemble in order of rank. A survival tree is selected for the resultant ensemble if the performance improves after an assessment using independent training data. This ensemble is called an optimal trees ensemble (OSTE). The proposed method is assessed using 17 benchmark datasets and the results are compared with those of random survival forest, conditional inference forest, bagging and a non tree based method, the Cox proportional hazard model. In addition to improve predictive performance, the proposed method reduces the number of survival trees in the ensemble as compared to the other tree based methods. The method is implemented in an R package called "OSTE".


Insights into Performance Fitness and Error Metrics for Machine Learning

arXiv.org Machine Learning

Machine learning (ML) is the field of training machines to achieve high level of cognition and perform human-like analysis. Since ML is a data-driven approach, it seemingly fits into our daily lives and operations as well as complex and interdisciplinary fields. With the rise of commercial, open-source and user-catered ML tools, a key question often arises whenever ML is applied to explore a phenomenon or a scenario: what constitutes a good ML model? Keeping in mind that a proper answer to this question depends on a variety of factors, this work presumes that a good ML model is one that optimally performs and best describes the phenomenon on hand. From this perspective, identifying proper assessment metrics to evaluate performance of ML models is not only necessary but is also warranted. As such, this paper examines a number of the most commonly-used performance fitness and error metrics for regression and classification algorithms, with emphasis on engineering applications.


How Might AI and Chest Imaging Help Unravel COVID-19's Mysteries?

#artificialintelligence

Artificial intelligence (AI) has the potential to expand the role of chest imaging in COVID-19 beyond diagnosis to enable risk stratification, treatment monitoring, and discovery of novel therapeutic targets. AI's power to generate models from large volumes of information โ€“ fusing molecular, clinical, epidemiological, and imaging data โ€“ may accelerate solutions to detect, contain, and treat COVID-19. Two healthcare workers fell ill in Wuhan, China, where the first Coronavirus Disease 2019 (COVID-19) case was reported. Both were 29 years old and were hospitalized after contracting the virus. One survived, the other died. In a global pandemic that has suddenly pushed doctors, scientists, and healthcare workers to the frontlines, why some patients are falling critically ill while others have minimal or no symptoms is one of the most mysterious aspects of the disease caused by Severe Acute Respiratory Syndrome coronavirus 2 (SARS-CoV-2).


Confusion Matrix and it's 25 offspring: or the link between machine learning and epidemiology Dr. Yury Zablotski

#artificialintelligence

For instance, an LR of 3 suggests that for every false positive, there are 3 true positives. The greater the value of the LR for a particular test, the more likely a positive test result is a true positive. On the other hand, an LR 1 would imply that an individual with a positive test result is more likely to be non-diseased than diseased. The rationale for the diagnostic odds ratio is that it is a single indicator of test performance (like accuracy and Youden's J index, explained below) which is independent of prevalence (unlike accuracy) and is presented as an odds ratio, which is familiar to epidemiologists. Similarly to a usual odds ratio, the diagnostic odds ratio ranges from zero to infinity, where DOR greater then one is already good, and the higher DOR goes, the better the test performs. DOR of less than one indicates that the test performs bad, or even gives wrong information.


Machine Learning for Exploring Spatial Affordance Patterns

arXiv.org Machine Learning

This dissertation uses supervised and unsupervised data mining techniques to analyse office floor plans in an attempt to gain a better understanding of their geometry-to-function relationship. This question was deemed relevant after a background review of the state-of-the-art in automated floor-plan generation tools showed that such tools have been prototyped since the 1960s, but their search space is ill-informed because there are few formalisms to describe spatial affordance. To show and evaluate the relationship of geometry and use, data from visual graph analysis were used to train three supervised learners and compare these to a baseline accuracy established with a ZeroR classifier. This showed that for the office dataset examined, visual mean depth and integration are most tightly linked to usage and that the supervised learning algorithm J48 can correctly predict class performance on unseen examples to up to 79.5%. The thesis also includes an evaluation of the layout case studies with unsupervised learners, which showed that use could not be immediately reverse-engineered based solemnly on the VGA information to achieve a strong cluster-to-class evaluation.


Improving Classification Accuracy by Mining Deterministic and Frequent Rules

AAAI Conferences

Patterns underlying the data sometimes take the form of IF conditions THEN outcome. However, not all the classifiers can detect such rules, resulting in compromised classification accuracy. In this paper we proposed an Add-on Rule-based Classifier (ARC) that can be paired with any existing classifier (base). The idea of ARC is improving the accuracy of the base by 1) mining deterministic and frequent rules, and 2) using such rules to assist the base in classification. Key novelty includes 1) a greedy search algorithm that identifies the rules by alternating between adding the ``best'' condition and removing the ``worst", and 2) new heuristics for selecting the best and worst conditions. We theoretically proved that rules detected by ARC are sound, complete, and minimal, indicating that ARC will almost never degrade the accuracy of the base, but instead, could often improve it. To experimentally verify this claim, we paired ARC with 9 leading classifiers and tested the ensembles on 12 UCI datasets. Empirical results show that, ARC never lowers the accuracy of the base and, more importantly, usually increases it (where some of the increases are statistically significant), echoing what we theoretically proved.


FDA investigates COVID-19 test with false negatives

PBS NewsHour

Food and Drug Administration Commissioner Steve Hahn says it will be up to the White House to determine whether it continues to use a coronavirus test that has falsely cleared patients of infection. Hahn Told CBS on Friday the FDA will keep "providing guidance to the White House regarding this test" but whether to keep using the test "will be a White House decision." The test is used daily at the White House to test President Donald Trump and key members of his staff, including the coronavirus task force. The FDA said late Thursday it was investigating preliminary data suggesting Abbott Laboratories' 15-minute test can miss COVID-19 cases, producing false negatives. Hahn told CBS the test is on the market and the FDA continues to "recommend its use or to have it available for use."