Government
Examining spatial heterogeneity of ridesourcing demand determinants with explainable machine learning
Zhang, Xiaojian, Yan, Xiang, Zhou, Zhengze, Xu, Yiming, Zhao, Xilei
The growing significance of ridesourcing services in recent years suggests a need to examine the key determinants of ridesourcing demand. However, little is known regarding the nonlinear effects and spatial heterogeneity of ridesourcing demand determinants. This study applies an explainable-machine-learning-based analytical framework to identify the key factors that shape ridesourcing demand and to explore their nonlinear associations across various spatial contexts (airport, downtown, and neighborhood). We use the ridesourcing-trip data in Chicago for empirical analysis. The results reveal that the importance of built environment varies across spatial contexts, and it collectively contributes the largest importance in predicting ridesourcing demand for airport trips. Additionally, the nonlinear effects of built environment on ridesourcing demand show strong spatial variations. Ridesourcing demand is usually most responsive to the built environment changes for downtown trips, followed by neighborhood trips and airport trips. These findings offer transportation professionals nuanced insights for managing ridesourcing services.
IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective
With the wide spread of sensors and smart devices in recent years, the data generation speed of the Internet of Things (IoT) systems has increased dramatically. In IoT systems, massive volumes of data must be processed, transformed, and analyzed on a frequent basis to enable various IoT services and functionalities. Machine Learning (ML) approaches have shown their capacity for IoT data analytics. However, applying ML models to IoT data analytics tasks still faces many difficulties and challenges, specifically, effective model selection, design/tuning, and updating, which have brought massive demand for experienced data scientists. Additionally, the dynamic nature of IoT data may introduce concept drift issues, causing model performance degradation. To reduce human efforts, Automated Machine Learning (AutoML) has become a popular field that aims to automatically select, construct, tune, and update machine learning models to achieve the best performance on specified tasks. In this paper, we conduct a review of existing methods in the model selection, tuning, and updating procedures in the area of AutoML in order to identify and summarize the optimal solutions for every step of applying ML algorithms to IoT data analytics. To justify our findings and help industrial users and researchers better implement AutoML approaches, a case study of applying AutoML to IoT anomaly detection problems is conducted in this work. Lastly, we discuss and classify the challenges and research directions for this domain.
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval
Litschko, Robert, Vulić, Ivan, Glavaš, Goran
State-of-the-art neural (re)rankers are notoriously data-hungry which -- given the lack of large-scale training data in languages other than English -- makes them rarely used in multilingual and cross-lingual retrieval settings. Current approaches therefore commonly transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders: they fine-tune all parameters of pretrained massively multilingual Transformers (MMTs, e.g., multilingual BERT) on English relevance judgments, and then deploy them in the target language(s). In this work, we show that two parameter-efficient approaches to cross-lingual transfer, namely Sparse Fine-Tuning Masks (SFTMs) and Adapters, allow for a more lightweight and more effective zero-shot transfer to multilingual and cross-lingual retrieval tasks. We first train language adapters (or SFTMs) via Masked Language Modelling and then train retrieval (i.e., reranking) adapters (SFTMs) on top, while keeping all other parameters fixed. At inference, this modular design allows us to compose the ranker by applying the (re)ranking adapter (or SFTM) trained with source language data together with the language adapter (or SFTM) of a target language. We carry out a large scale evaluation on the CLEF-2003 and HC4 benchmarks and additionally, as another contribution, extend the former with queries in three new languages: Kyrgyz, Uyghur and Turkish. The proposed parameter-efficient methods outperform standard zero-shot transfer with full MMT fine-tuning, while being more modular and reducing training times. The gains are particularly pronounced for low-resource languages, where our approaches also substantially outperform the competitive machine translation-based rankers.
Selective Token Generation for Few-shot Natural Language Generation
Jo, Daejin, Kwon, Taehwan, Kim, Eun-Sol, Kim, Sungwoong
Natural language modeling with limited training data is a challenging problem, and many algorithms make use of large-scale pretrained language models (PLMs) for this due to its great generalization ability. Among them, additive learning that incorporates a task-specific adapter on top of the fixed large-scale PLM has been popularly used in the few-shot setting. However, this added adapter is still easy to disregard the knowledge of the PLM especially for few-shot natural language generation (NLG) since an entire sequence is usually generated by only the newly trained adapter. Therefore, in this work, we develop a novel additive learning algorithm based on reinforcement learning (RL) that selectively outputs language tokens between the task-general PLM and the task-specific adapter during both training and inference. This output token selection over the two generators allows the adapter to take into account solely the task-relevant parts in sequence generation, and therefore makes it more robust to overfitting as well as more stable in RL training. In addition, to obtain the complementary adapter from the PLM for each few-shot task, we exploit a separate selecting module that is also simultaneously trained using RL. Experimental results on various few-shot NLG tasks including question answering, data-to-text generation and text summarization demonstrate that the proposed selective token generation significantly outperforms the previous additive learning algorithms based on the PLMs.
Interactions in Information Spread
Since the development of writing 5000 years ago, human-generated data gets produced at an ever-increasing pace. Classical archival methods aimed at easing information retrieval. Nowadays, archiving is not enough anymore. The amount of data that gets generated daily is beyond human comprehension, and appeals for new information retrieval strategies. Instead of referencing every single data piece as in traditional archival techniques, a more relevant approach consists in understanding the overall ideas conveyed in data flows. To spot such general tendencies, a precise comprehension of the underlying data generation mechanisms is required. In the rich literature tackling this problem, the question of information interaction remains nearly unexplored. First, we investigate the frequency of such interactions. Building on recent advances made in Stochastic Block Modelling, we explore the role of interactions in several social networks. We find that interactions are rare in these datasets. Then, we wonder how interactions evolve over time. Earlier data pieces should not have an everlasting influence on ulterior data generation mechanisms. We model this using dynamic network inference advances. We conclude that interactions are brief. Finally, we design a framework that jointly models rare and brief interactions based on Dirichlet-Hawkes Processes. We argue that this new class of models fits brief and sparse interaction modelling. We conduct a large-scale application on Reddit and find that interactions play a minor role in this dataset. From a broader perspective, our work results in a collection of highly flexible models and in a rethinking of core concepts of machine learning. Consequently, we open a range of novel perspectives both in terms of real-world applications and in terms of technical contributions to machine learning.
Artificial Intelligence for In Silico Clinical Trials: A Review
Wang, Zifeng, Gao, Chufan, Glass, Lucas M., Sun, Jimeng
A clinical trial is an essential step in drug development, which is often costly and time-consuming. In silico trials are clinical trials conducted digitally through simulation and modeling as an alternative to traditional clinical trials. AI-enabled in silico trials can increase the case group size by creating virtual cohorts as controls. In addition, it also enables automation and optimization of trial design and predicts the trial success rate. This article systematically reviews papers under three main topics: clinical simulation, individualized predictive modeling, and computer-aided trial design. We focus on how machine learning (ML) may be applied in these applications. In particular, we present the machine learning problem formulation and available data sources for each task. We end with discussing the challenges and opportunities of AI for in silico trials in real-world applications.
Learning the Quality of Machine Permutations in Job Shop Scheduling
Corsini, Andrea, Calderara, Simone, Dell'Amico, Mauro
In recent years, the power demonstrated by Machine Learning (ML) has increasingly attracted the interest of the optimization community that is starting to leverage ML for enhancing and automating the design of algorithms. One combinatorial optimization problem recently tackled with ML is the Job Shop scheduling Problem (JSP). Most of the works on the JSP using ML focus on Deep Reinforcement Learning (DRL), and only a few of them leverage supervised learning techniques. The recurrent reasons for avoiding supervised learning seem to be the difficulty in casting the right learning task, i.e., what is meaningful to predict, and how to obtain labels. Therefore, we first propose a novel supervised learning task that aims at predicting the quality of machine permutations. Then, we design an original methodology to estimate this quality, and we use these estimations to create an accurate sequential deep learning model (binary accuracy above 95%). Finally, we empirically demonstrate the value of predicting the quality of machine permutations by enhancing the performance of a simple Tabu Search algorithm inspired by the works in the literature.
New York Times op-ed says population decline may make 'climate change easier to combat'
During an appearance on "Fox and Friends First", Jimmy Failla shares his thoughts on Vice President Kamala Harris jet-setting to an event in Buffalo, York which focused on the Biden administration's green agenda. A New York Times newsletter published an article by staff editor Spencer Bokat-Lindell arguing that declining fertility rates and a shrinking population could be a good thing to help the world combat climate change. The article, titled "U.S. Population Growth Has Nearly Flatlined. Is That So Bad?", speculated that population decline may "actually bring welcome changes." "For a population to replenish itself in the absence of immigration, demographers estimate that there must be, on average, about 2.1 births per woman," Bokat-Lindell noted while pointing out that in the United States, the fertility rate has been below that since 2007.
Pentagon artificial intelligence boss says real data to guide work
After more than 100 days on the job, the U.S. Department of Defense's top artificial intelligence official said Thursday that the office's strategy will take a down-up approach and start with data. "It's true that most of my career has been in AI, but I don't think most of my work here at DoD will be about AI," Craig Martell, the Pentagon's first Chief Digital and AI Officer, said at the 2022 Intelligence & National Security Summit in National Harbor, Maryland. The Pentagon tapped Martell, a former head of machine learning at Lyft, to oversee its AI efforts this spring. The office subsumed multiple other defense digital authorities, including the Joint Artificial Intelligence Center and the Defense Digital Services, and reached full operational capability on June 1. While much of the talk about the CDAO has focused on its role as the Pentagon's leading AI authority, Martell said more attention and energy needs to be paid to data, which he said serves as the basis of the office's "hierarchy of needs."
The Artificial Intelligence and Cybersecurity Nexus: Taking Stock of the European Union's Approach
This publication is part of EU Cyber Direct – EU Cyber Diplomacy Initiative's New Tech in Review, a collection of commentaries that highlights key issues at the intersection of emerging technologies, cybersecurity, defense, and norms. Digital technologies increasingly complicate and transform present-day conflicts. The current war between Russia and Ukraine, for instance, is also played out in cyberspace, involving multiple public and private actors. This ranges from the formation of an IT army of Ukrainian volunteers, to the intensification of Kremlin-backed malicious cyber operations, to Western allied nations offering Ukraine assistance across "the full spectrum; offensive, defensive, [and] information operations." Such developments matter as they are consistent with a longer history of cyber conflicts running alongside and feeding into kinetic operations. Even if the conflict's cyber dimension is, as noted by experts, presently limited, there are serious concerns about the destabilization of the international security environment, including high risks of escalation. These risks are also exacerbated by the potential spillover of cyberattacks targeting Ukraine into other countries, which could cause systemic ripples in cyberspace and beyond.