Goto

Collaborating Authors

 Oceania


Know Your Limits: Monotonicity & Softmax Make Neural Classifiers Overconfident on OOD Data

arXiv.org Artificial Intelligence

A crucial requirement for reliable deployment of deep learning models for safety-critical applications is the ability to identify out-of-distribution (OOD) data points, samples which differ from the training data and on which a model might underperform. Previous work has attempted to tackle this problem using uncertainty estimation techniques. However, there is empirical evidence that a large family of these techniques do not detect OOD reliably in classification tasks. This paper puts forward a theoretical explanation for said experimental findings. We prove that such techniques are not able to reliably identify OOD samples in a classification setting, provided the models satisfy weak assumptions about the monotonicity of feature values and resulting class probabilities. This result stems from the interplay between the saturating nature of activation functions like sigmoid or softmax, coupled with the most widely-used uncertainty metrics.


Beyond Occam's Razor in System Identification: Double-Descent when Modeling Dynamics

arXiv.org Machine Learning

System identification aims to build models of dynamical systems from data. Traditionally, choosing the model requires the designer to balance between two goals of conflicting nature; the model must be rich enough to capture the system dynamics, but not so flexible that it learns spurious random effects from the dataset. It is typically observed that model validation performance follows a U-shaped curve as the model complexity increases. Recent developments in machine learning and statistics, however, have observed situations where a "double-descent" curve subsumes this U-shaped model-performance curve. With a second decrease in performance occurring beyond the point where the model has reached the capacity of interpolating - i.e., (near) perfectly fitting - the training data. To the best of our knowledge, however, such phenomena have not been studied within the context of the identification of dynamic systems. The present paper aims to answer the question: "Can such a phenomenon also be observed when estimating parameters of dynamic systems?" We show the answer is yes, verifying such behavior experimentally both for artificially generated and real-world datasets.


Sequential Estimation of Nonparametric Correlation using Hermite Series Estimators

arXiv.org Machine Learning

In this article we describe a new Hermite series based sequential estimator for the Spearman's rank correlation coefficient and provide algorithms applicable in both the stationary and non-stationary settings. To treat the non-stationary setting, we introduce a novel, exponentially weighted estimator for the Spearman's rank correlation, which allows the local nonparametric correlation of a bivariate data stream to be tracked. To the best of our knowledge this is the first algorithm to be proposed for estimating a time-varying Spearman's rank correlation that does not rely on a moving window approach. We explore the practical effectiveness of the Hermite series based estimators through real data and simulation studies demonstrating good practical performance. The simulation studies in particular reveal competitive performance compared to an existing algorithm. The potential applications of this work are manifold. The Hermite series based Spearman's rank correlation estimator can be applied to fast and robust online calculation of correlation which may vary over time. Possible machine learning applications include, amongst others, fast feature selection and hierarchical clustering on massive data sets.


NAB partners with credit risk AI specialist

#artificialintelligence

National Australia Bank (NAB) has signed an agreement with artificial intelligence (AI) specialist Rich Data Corporation to use its AI prediction and decisioning capability software Delta. Under the agreement, NAB would use Delta's credit risk management capability, which links traditional and alternative data sources to provide prediction and credit decisions, for its SME customers with little or no credit history or who might fall outside traditional credit bureau criteria. The capability will also use analytics and monitoring tools to identify portfolio risks and opportunities to support businesses that experience major change from events like the coronavirus pandemic. Commenting on the partnership with Rich Data Corporation, NAB executive innovation and partnerships Howard Silby said: Their Delta platform will enable us to innovate and accelerate our lending options, leveraging alternative data and AI techniques and providing a means to assess SMEs who fall outside traditional credit models. "We remain committed to finding new ways to serve and meet the needs of our small-business customers, as we pioneered with our NAB QuickBiz offering in 2016 and several innovations since."


AI to judge small business loans at NAB

#artificialintelligence

National Australia Bank will use artificial intelligence technology to make credit decisions on small business loans, illustrating how AI applications are shifting from the periphery to the heart of banking operations. NAB hopes the system that is being built by Rich Data, a Sydney-based AI company, will increase the supply of credit by widening the pool of small businesses that can qualify for a loan. It will also help NAB push towards real-time loan assessments, tapping data from cloud accounting platforms, transaction systems and other macroeconomic sources to profile small to medium enterprises and predict their likelihood of repayment. Howard Silby, chief innovation officer at NAB: "This will be put live with real customers early in the 2021 calendar year." NAB is the first major Australian customer for Rich Data, which supplies its Delta platform to banks in Asia and North America. NAB uses AI to triage of customer complaints and to detect money-laundering and fraud.


A transmissible cancer shifts from emergence to endemism in Tasmanian devils

Science

The emergence of a devastating transmissible facial cancer among Tasmanian devils over the past few decades has caused substantial concern for their future because these animals are already threatened by a regional distribution and other stressors. Little is known about the overall history and trajectory of this disease. Patton et al. used an epidemiological phylodynamic approach to reveal the pattern of disease emergence and spread. They found that low Tasmanian devil densities appear to be contributing to slower disease growth and spread, which is good news for Tasmanian devil persistence and suggests that care should be taken when considering options for increasing devil populations. Science , this issue p. [eabb9772][1] ### INTRODUCTION Emerging infectious diseases pose one of the greatest threats to human health and biodiversity. Phylodynamics is an effective tool for inferring epidemiological parameters to guide intervention strategies, particularly for human viruses such as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). However, phylodynamic analysis has historically been limited to the study of rapidly evolving viruses and, in rare cases, bacteria. Nonetheless, application of phylodynamics to nonviral pathogens has immense potential, such as for predicting disease spread and informing the management of wildlife diseases. We conducted a phylodynamics analysis of devil facial tumor disease (DFTD), a transmissible cancer that has spread across nearly the entire geographic range of Tasmanian devils and threatens the species with extinction. DFTD is transmitted as an allograft through biting during common social interactions, susceptibility is nearly universal, and case fatality rates approach 100%. The goals of our study were to (i) characterize the geographic spread of DFTD, (ii) identify whether there are different circulating tumor lineages, and (iii) quantify rates of transmission among lineages. ### RATIONALE In principle, phylodynamics should be readily extended to the study of slowly evolving pathogens with large genomes through careful interrogation of genes to identify those that are measurably evolving. By testing individual genes for a clocklike signal, these genes may then be used for phylodynamic analysis. We demonstrate this proof of concept in DFTD. ### RESULTS We screened >11,000 genes across the DFTD genome, identifying 28 that exhibited a strong, clocklike signal, and performed the first phylodynamic analysis of a genome larger than a bacterium. We demonstrate here, contrary to field observations, that DFTD spread omnidirectionally throughout the epizootic, leaving little signal of geographic structuring of tumor lineages across Tasmania. Despite predictions of devil extinction, we found that the effective reproduction number ( R E), a summary of the rate at which disease spreads, has declined precipitously after the initial epidemic spread of DFTD. Specifically, R E peaked at a high of ~3.5 shortly after the discovery of DFTD in 1996 and is now ~1 in both extant tumor lineages. This is consistent with a shift from emergence to endemism. Except for a single gene, we found little evidence for convergent molecular evolution among tumor lineages. ### CONCLUSION We have demonstrated that phylodynamics can be applied to virtually any pathogen. In doing so, we show that through careful interrogation of the pathogen genome, a measurably evolving set of genes can be identified to characterize epidemiological dynamics of nonviral pathogens with large genomes. By applying this approach to DFTD, we have shown that the disease appears to be transitioning from emergence to endemism. Consistent with recent models, our inference that R E ~1 predicts that coexistence between devils and DFTD is a more likely outcome than devil extinction. Therefore, our findings present cautious optimism for the continued survival of the iconic Tasmanian devil but emphasize the need for evolutionarily informed conservation management to ensure their persistence. ![Figure][2] Tasmanian devils and their transmissible cancer. Healthy (top) and DFTD-infected (bottom) Tasmanian devils. Photos: David G. Hamilton (top), Alexandra K. Fraik (bottom). Emerging infectious diseases pose one of the greatest threats to human health and biodiversity. Phylodynamics is often used to infer epidemiological parameters essential for guiding intervention strategies for human viruses such as severe acute respiratory syndrome coronavirus 2 (SARS-Cov-2). Here, we applied phylodynamics to elucidate the epidemiological dynamics of Tasmanian devil facial tumor disease (DFTD), a fatal, transmissible cancer with a genome thousands of times larger than that of any virus. Despite prior predictions of devil extinction, transmission rates have declined precipitously from ~3.5 secondary infections per infected individual to ~1 at present. Thus, DFTD appears to be transitioning from emergence to endemism, lending hope for the continued survival of the endangered Tasmanian devil. More generally, our study demonstrates a new phylodynamic analytical framework that can be applied to virtually any pathogen. [1]: /lookup/doi/10.1126/science.abb9772 [2]: pending:yes


Machine learning helps to map invasive plant from space

AIHub

Researchers from CSIRO, Charles Darwin University and The University of Western Australia have developed a machine-learning approach that reliably detects invasive gamba grass from high-resolution satellite imagery. Gamba grass is listed as a Weed of National Significance, and is one of five introduced grass species that pose extensive and significant threats to Australia's biodiversity. The perennial grass can grow to four metres in height and forms dense tussocks which can burn as large, hot fires late in the dry season. Mapping where gamba grass occurs is essential to managing it effectively, but northern Australia is so vast and remote that on-the-ground mapping and even airborne detection of the weed is too labour-intensive. So, the researchers turned to high-quality satellite imagery and developed a technique that could help detect and prioritise gamba grass for management.


Upheaval at Google signals pushback against biased algorithms and unaccountable AI

#artificialintelligence

Artificial intelligence (AI) is no longer the stuff of science fiction. AI determines what news you get served up on the internet. It plays a key role in online matchmaking, which is now the way most romantic couples get together. It will tell you how to get to your next meeting, and what time to leave home so you're not late. AI often appears both omniscient and neutral, but on closer inspection we find AI learns from and adopts human biases.


Six researchers who are shaping the future of artificial intelligence

#artificialintelligence

As artificial intelligence (AI) becomes ubiquitous in fields such as medicine, education and security, there are significant ethical and technical challenges to overcome. While the credits to Star Wars drew to a close in a 1970s cinema, 10-year-old Cynthia Breazeal remained fixated on C-3PO, the anxious robot. "Typically, when you saw robots in science fiction, they were mindless, but in Star Wars they had rich personalities and could form friendships," says Breazeal, associate director of the Massachusetts Institute of Technology (MIT) Media Lab in Cambridge, Massachusetts. "I assumed these robots would never exist in my lifetime." A pioneer of social robotics and human–robot interaction, Breazeal has made a career of conceptualizing and building robots with personality.


Argument Mining Driven Analysis of Peer-Reviews

arXiv.org Artificial Intelligence

Peer reviewing is a central process in modern research and essential for ensuring high quality and reliability of published work. At the same time, it is a time-consuming process and increasing interest in emerging fields often results in a high review workload, especially for senior researchers in this area. How to cope with this problem is an open question and it is vividly discussed across all major conferences. In this work, we propose an Argument Mining based approach for the assistance of editors, meta-reviewers, and reviewers. We demonstrate that the decision process in the field of scientific publications is driven by arguments and automatic argument identification is helpful in various use-cases. One of our findings is that arguments used in the peer-review process differ from arguments in other domains making the transfer of pre-trained models difficult. Therefore, we provide the community with a new peer-review dataset from different computer science conferences with annotated arguments. In our extensive empirical evaluation, we show that Argument Mining can be used to efficiently extract the most relevant parts from reviews, which are paramount for the publication decision. The process remains interpretable since the extracted arguments can be highlighted in a review without detaching them from their context.