Goto

Collaborating Authors

 Performance Analysis


Selective Classification via One-Sided Prediction

arXiv.org Machine Learning

We propose a novel method for selective classification (SC), a problem which allows a classifier to abstain from predicting some instances, thus trading off accuracy against coverage (the fraction of instances predicted). In contrast to prior gating or confidence-set based work, our proposed method optimises a collection of class-wise decoupled one-sided empirical risks, and is in essence a method for explicitly finding the largest decision sets for each class that have few false positives. This one-sided prediction (OSP) based relaxation yields an SC scheme that attains near-optimal coverage in the practically relevant high target accuracy regime, and further admits efficient implementation, leading to a flexible and principled method for SC. We theoretically derive generalization bounds for SC and OSP, and empirically we show that our scheme strongly outperforms state of the art methods in coverage at small error levels.


Acoustics Based Intent Recognition Using Discovered Phonetic Units for Low Resource Languages

arXiv.org Artificial Intelligence

With recent advancements in language technologies, humansare now interacting with technology through speech. To in-crease the reach of these technologies, we need to build suchsystems in local languages. A major bottleneck here are theunderlying data-intensive parts that make up such systems,including automatic speech recognition (ASR) systems thatrequire large amounts of labelled data. With the aim of aidingdevelopment of dialog systems in low resourced languages,we propose a novel acoustics based intent recognition systemthat uses discovered phonetic units for intent classification.The system is made up of two blocks - the first block gen-erates a transcript of discovered phonetic units for the inputaudio, and the second block which performs intent classifi-cation from the generated phonemic transcripts. Our workpresents results for such a system for two languages families- Indic languages and Romance languages, for two differentintent recognition tasks. We also perform multilingual train-ing of our intent classifier and show improved cross-lingualtransfer and performance on an unknown language with zeroresources in the same language family.


Instant Payments

#artificialintelligence

PelicanPayments is a complete payments solution delivering exceptional levels of efficiency, control and flexibility. PelicanSecure is a comprehensive suite of real-time financial crime compliance and anti-fraud solutions. Pelican Open Banking Hub connects 6000 banks in Europe with an ability to connect to over 500 million customers. This Open Banking webinar focuses on opportunities for banks in opting for the an aggressive and proactive approach to winning new customers. To read a full report, click "Learn More:


FDA: Antigen tests for COVID-19 are rapid but can lead to false positives

Boston Herald

The U.S. Food and Drug Administration is alerting clinical laboratory staff and health care providers that false positive results can occur with antigen tests for the virus that causes COVID-19. In a letter to stakeholders, the FDA said Tuesday that while antigen tests can be used for the rapid detection of SARS-CoV-2, false positive results can occur, especially if users don't follow the instructions. "The FDA is aware of reports of false positive results associated with antigen tests used in nursing homes and other settings and continues to monitor and evaluate these reports and other available information about device safety and performance," the letter said. A Boston-area infectious disease expert said the antigen tests are good for large scale screening, when used properly, but must be followed up with more accurate testing. "If you are testing a population at low risk, it's fine to do these tests for screening," said Dr. Daniel Kuritzkes, chief of the Division of Infectious Diseases at Brigham and Women's Hospital.


Re-Assessing the "Classify and Count" Quantification Method

arXiv.org Artificial Intelligence

Learning to quantify (a.k.a.\ quantification) is a task concerned with training unbiased estimators of class prevalence via supervised learning. This task originated with the observation that "Classify and Count" (CC), the trivial method of obtaining class prevalence estimates, is often a biased estimator, and thus delivers suboptimal quantification accuracy; following this observation, several methods for learning to quantify have been proposed that have been shown to outperform CC. In this work we contend that previous works have failed to use properly optimised versions of CC. We thus reassess the real merits of CC (and its variants), and argue that, while still inferior to some cutting-edge methods, they deliver near-state-of-the-art accuracy once (a) hyperparameter optimisation is performed, and (b) this optimisation is performed by using a true quantification loss instead of a standard classification-based loss. Experiments on three publicly available binary sentiment classification datasets support these conclusions.


Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection

arXiv.org Artificial Intelligence

This paper proposes a new defense against neural network backdooring attacks that are maliciously trained to mispredict in the presence of attacker-chosen triggers. Our defense is based on the intuition that the feature extraction layers of a backdoored network embed new features to detect the presence of a trigger and the subsequent classification layers learn to mispredict when triggers are detected. Therefore, to detect backdoors, the proposed defense uses two synergistic anomaly detectors trained on clean validation data: the first is a novelty detector that checks for anomalous features, while the second detects anomalous mappings from features to outputs by comparing with a separate classifier trained on validation data. The approach is evaluated on a wide range of backdoored networks (with multiple variations of triggers) that successfully evade state-of-the-art defenses. Additionally, we evaluate the robustness of our approach on imperceptible perturbations, scalability on large-scale datasets, and effectiveness under domain shift. This paper also shows that the defense can be further improved using data augmentation.


Stochastic Hard Thresholding Algorithms for AUC Maximization

arXiv.org Machine Learning

In this paper, we aim to develop stochastic hard thresholding algorithms for the important problem of AUC maximization in imbalanced classification. The main challenge is the pairwise loss involved in AUC maximization. We overcome this obstacle by reformulating the U-statistics objective function as an empirical risk minimization (ERM), from which a stochastic hard thresholding algorithm (\texttt{SHT-AUC}) is developed. To our best knowledge, this is the first attempt to provide stochastic hard thresholding algorithms for AUC maximization with a per-iteration cost $\O(b d)$ where $d$ and $b$ are the dimension of the data and the minibatch size, respectively. We show that the proposed algorithm enjoys the linear convergence rate up to a tolerance error. In particular, we show, if the data is generated from the Gaussian distribution, then its convergence becomes slower as the data gets more imbalanced. We conduct extensive experiments to show the efficiency and effectiveness of the proposed algorithms.


Ensuring Fairness Beyond the Training Data

arXiv.org Machine Learning

We initiate the study of fair classifiers that are robust to perturbations in the training distribution. Despite recent progress, the literature on fairness has largely ignored the design of fair and robust classifiers. In this work, we develop classifiers that are fair not only with respect to the training distribution, but also for a class of distributions that are weighted perturbations of the training samples. We formulate a min-max objective function whose goal is to minimize a distributionally robust training loss, and at the same time, find a classifier that is fair with respect to a class of distributions. We first reduce this problem to finding a fair classifier that is robust with respect to the class of distributions. Based on online learning algorithm, we develop an iterative algorithm that provably converges to such a fair and robust solution. Experiments on standard machine learning fairness datasets suggest that, compared to the state-of-the-art fair classifiers, our classifier retains fairness guarantees and test accuracy for a large class of perturbations on the test set. Furthermore, our experiments show that there is an inherent trade-off between fairness robustness and accuracy of such classifiers.


NeuMiss networks: differentiable programming for supervised learning with missing values

arXiv.org Artificial Intelligence

The presence of missing values makes supervised learning much more challenging. Indeed, previous work has shown that even when the response is a linear function of the complete data, the optimal predictor is a complex function of the observed entries and the missingness indicator. As a result, the computational or sample complexities of consistent approaches depend on the number of missing patterns, which can be exponential in the number of dimensions. In this work, we derive the analytical form of the optimal predictor under a linearity assumption and various missing data mechanisms including Missing at Random (MAR) and self-masking (Missing Not At Random). Based on a Neumann-series approximation of the optimal predictor, we propose a new principled architecture, named NeuMiss networks. Their originality and strength come from the use of a new type of non-linearity: the multiplication by the missingness indicator. We provide an upper bound on the Bayes risk of NeuMiss networks, and show that they have good predictive accuracy with both a number of parameters and a computational complexity independent of the number of missing data patterns. As a result they scale well to problems with many features, and remain statistically efficient for medium-sized samples. Moreover, we show that, contrary to procedures using EM or imputation, they are robust to the missing data mechanism, including difficult MNAR settings such as self-masking.


Rapid coronavirus antigen tests may give false positives, FDA warns

FOX News

Our technology has advanced, our diagnostics have improved and our testing capability has advanced since the beginning of this pandemic, says Dr. Nicole Saphier, Fox News medical contributor. The Food and Drug Administration (FDA) warned about the possibility of false positives that can occur when using rapid antigen tests to detect coronavirus, particularly if the test is not used correctly. The regulatory agency said it has received reports of false-positive results occurring in nursing homes and other health care settings. The agency warned that reading the test results either before or after the specified time provided in the instructions can result in false-positive or false-negative results. It also referenced the antigen EUA conditions of authorization, which specifies that authorized laboratories are to follow the manufacturer's instructions for use regarding administering the test and reading the results.