Temporal concept drift refers to the problem of data changing over time. In NLP, that would entail that language (e.g. new expressions, meaning shifts) and factual knowledge (e.g. new concepts, updated facts) evolve over time. Focusing on the latter, we benchmark $11$ pretrained masked language models (MLMs) on a series of tests designed to evaluate the effect of temporal concept drift, as it is crucial that widely used language models remain up-to-date with the ever-evolving factual updates of the real world. Specifically, we provide a holistic framework that (1) dynamically creates temporal test sets of any time granularity (e.g. month, quarter, year) of factual data from Wikidata, (2) constructs fine-grained splits of tests (e.g. updated, new, unchanged facts) to ensure comprehensive analysis, and (3) evaluates MLMs in three distinct ways (single-token probing, multi-token generation, MLM scoring). In contrast to prior work, our framework aims to unveil how robust an MLM is over time and thus to provide a signal in case it has become outdated, by leveraging multiple views of evaluation.

Deep neural network based speech enhancement approaches aim to learn a noisy-to-clean transformation using a supervised learning paradigm. However, such a trained-well transformation is vulnerable to unseen noises that are not included in training set. In this work, we focus on the unsupervised noise adaptation problem in speech enhancement, where the ground truth of target domain data is completely unavailable. Specifically, we propose a generative adversarial network based method to efficiently learn a converse clean-to-noisy transformation using a few minutes of unpaired target domain data. Then this transformation is utilized to generate sufficient simulated data for domain adaptation of the enhancement model. Experimental results show that our method effectively mitigates the domain mismatch between training and test sets, and surpasses the best baseline by a large margin.

The US federal government spends more than a trillion dollars per year on health care, largely provided by private third parties and reimbursed by the government. A major concern in this system is overbilling, waste and fraud by providers, who face incentives to misreport on their claims in order to receive higher payments. In this paper, we develop novel machine learning tools to identify providers that overbill Medicare, the US federal health insurance program for elderly adults and the disabled. Using large-scale Medicare claims data, we identify patterns consistent with fraud or overbilling among inpatient hospitalizations. Our proposed approach for Medicare fraud detection is fully unsupervised, not relying on any labeled training data, and is explainable to end users, providing reasoning and interpretable insights into the potentially suspicious behavior of the flagged providers. Data from the Department of Justice on providers facing anti-fraud lawsuits and several case studies validate our approach and findings both quantitatively and qualitatively.

Decision-makers in the humanitarian sector rely on timely and exact information during crisis events. Knowing how many civilians were injured during an earthquake is vital to allocate aids properly. Information about such victim counts is often only available within full-text event descriptions from newspapers and other reports. Extracting numbers from text is challenging: numbers have different formats and may require numeric reasoning. This renders purely string matching-based approaches insufficient. As a consequence, fine-grained counts of injured, displaced, or abused victims beyond fatalities are often not extracted and remain unseen. We cast victim count extraction as a question answering (QA) task with a regression or classification objective. We compare regex, dependency parsing, semantic role labeling-based approaches, and advanced text-to-text models. Beyond model accuracy, we analyze extraction reliability and robustness which are key for this sensitive task. In particular, we discuss model calibration and investigate few-shot and out-of-distribution performance. Ultimately, we make a comprehensive recommendation on which model to select for different desiderata and data domains. Our work is among the first to apply numeracy-focused large language models in a real-world use case with a positive impact.

The cosmic microwave background (CMB) is a significant source of knowledge about the origin and evolution of our universe. However, observations of the CMB are contaminated by foreground emissions, obscuring the CMB signal and reducing its efficacy in constraining cosmological parameters. We employ deep learning as a data-driven approach to CMB cleaning from multi-frequency full-sky maps. In particular, we develop a graph-based Bayesian convolutional neural network based on the U-Net architecture that predicts cleaned CMB with pixel-wise uncertainty estimates. We demonstrate the potential of this technique on realistic simulated data based on the Planck mission. We show that our model accurately recovers the cleaned CMB sky map and resulting angular power spectrum while identifying regions of uncertainty. Finally, we discuss the current challenges and the path forward for deploying our model for CMB recovery on real observations.

A critical factor in utilizing agents with Artificial Intelligence (AI) is their robustness to novelty. AI agents include models that are either engineered or trained. Engineered models include knowledge of those aspects of the environment that are known and considered important by the engineers. Learned models form embeddings of aspects of the environment based on connections made through the training data. In operation, however, a rich environment is likely to present challenges not seen in training sets or accounted for in engineered models. Worse still, adversarial environments are subject to change by opponents. A program at the Defense Advanced Research Project Agency (DARPA) seeks to develop the science necessary to develop and evaluate agents that are robust to novelty. This capability will be required, before AI has the role envisioned within mission critical environments. As part of the Science of AI and Learning for Open-world Novelty (SAIL-ON), we are mapping possible military domain novelty types to a domain-independent ontology developed as part of a theory of novelty. Characterizing the possible space of novelty mathematically and ontologically will allow us to experiment with agent designs that are coming from the DARPA SAIL-ON program in relevant military environments. Utilizing the same techniques as being used in laboratory experiments, we will be able to measure agent ability to detect, characterize, and accommodate novelty.

Since its introduction in 2017, physics-informed deep learning (PIDL) has garnered growing popularity in understanding the evolution of systems governed by physical laws in terms of partial differential equations (PDEs). However, empirical evidence points to the limitations of PIDL for learning certain types of PDEs. In this paper, we (a) present the challenges in training PIDL architecture, (b) contrast the performance of PIDL architecture in learning a first order scalar hyperbolic conservation law and its parabolic counterpart, (c) investigate the effect of training data sampling, which corresponds to various sensing scenarios in traffic networks, (d) comment on the implications of PIDL limitations for traffic flow estimation and prediction in practice. Detailed in the case study, we present the contradistinction in PIDL results between learning the traffic flow model (LWR PDE) and its variation with diffusion. The outcome indicates that PIDL experiences significant challenges in learning the hyperbolic LWR equation due to the non-smoothness of its solution. On the other hand, the architecture with parabolic PDE, augmented with the diffusion term, leads to the successful reassembly of the density data even with the shockwaves present.

A large amount of feedback was collected over the years. Many feedback analysis models have been developed focusing on the English language. Recognizing the concept of feedback is challenging and crucial in languages which do not have applicable corpus and tools employed in Natural Language Processing (i.e., vocabulary corpus, sentence structure rules, etc). However, in this paper, we study a feedback classification in Mongolian language using two different word embeddings for deep learning. We compare the results of proposed approaches. We use feedback data in Cyrillic collected from 2012-2018. The result indicates that word embeddings using their own dataset improve the deep learning based proposed model with the best accuracy of 80.1% and 82.7% for two classification tasks.

Sydney, Australia Xinhua A global study released on Wednesday has found that three out of five people, about 61 percent, are wary about trusting artificial intelligence (AI) systems, reporting either ambivalence or an unwillingness to trust. Researchers from the University of Queensland and KPMG Australia surveyed more than 17,000 people from 17 countries across the world, such as Australia, China, France and the United States, which were regarded by the study as leaders in AI activity and readiness within their region. According to the study, only 39 percent said they are willing to trust AI systems, while a third of people worldwide reported high acceptance. Though about 85 percent believed the use of AI will result in a wide range of benefits, including improved efficiency, innovation, reduced costs and better use of resources, 73 percent of the respondents also voiced concern over potential risks from AI use. Among the nine listed risks, cybersecurity is the dominant concern raised by 84 percent of people, followed by manipulation or harmful use of AI and job loss due to automation.

The lessons of the Ukraine war are still being debated and assessed. After all, there still is no official winner in this conflict. But the war has clearly accelerated certain military trends, experts have said, indicating how future wars will be fought. Social media is perhaps the greatest innovation in this war. The internet has been replete with videos of Russian armour being destroyed by Ukrainian operatives, an underdog narrative amplified repeatedly by official Ukrainian channels.