Goto

Collaborating Authors

 Government


Detecting critical treatment effect bias in small subgroups

arXiv.org Machine Learning

Randomized trials have traditionally been the gold standard for informed decision-making in medicine, as they allow for unbiased estimation of treatment effects under mild assumptions. However, there is often a significant discrepancy between the patients observed in clinical practice and those enrolled in randomized trials, limiting the generalizability of the trial results [12, 43]. To address this issue, the U.S. Food and Drug Administration advocates for using observational data, as it is usually more representative of the patient population in clinical practice [30, 39]. Yet, a major caveat to this recommendation is that several sources of bias, including hidden confounding, can compromise the causal conclusions drawn from observational data. In light of the inherent limitations of randomized and observational data, it has become a popular strategy to benchmark observational studies against existing randomized trials to assess their quality [4, 13]. The main idea behind this approach is first to emulate the procedures adopted in the randomized trial within the observational study; see e.g.


Computational Job Market Analysis with Natural Language Processing

arXiv.org Artificial Intelligence

[Abridged Abstract] Recent technological advances underscore labor market dynamics, yielding significant consequences for employment prospects and increasing job vacancy data across platforms and languages. Aggregating such data holds potential for valuable insights into labor market demands, new skills emergence, and facilitating job matching for various stakeholders. However, despite prevalent insights in the private sector, transparent language technology systems and data for this domain are lacking. This thesis investigates Natural Language Processing (NLP) technology for extracting relevant information from job descriptions, identifying challenges including scarcity of training data, lack of standardized annotation guidelines, and shortage of effective extraction methods from job ads. We frame the problem, obtaining annotated data, and introducing extraction methodologies. Our contributions include job description datasets, a de-identification dataset, and a novel active learning algorithm for efficient model training. We propose skill extraction using weak supervision, a taxonomy-aware pre-training methodology adapting multilingual language models to the job market domain, and a retrieval-augmented model leveraging multiple skill extraction datasets to enhance overall performance. Finally, we ground extracted information within a designated taxonomy.


Ireland looking to send asylum seekers back to UK: Report

Al Jazeera

The Republic of Ireland is looking to amend the law to allow the return of asylum seekers to the United Kingdom, according to broadcaster RTE, after an influx over the border with Northern Ireland, which is part of the UK. Dublin's Minister of Justice Helen McEntee, who will visit London on Monday, told a parliamentary committee this week that she estimates 80 percent of those applying for asylum in the republic came over the land border with Northern Ireland. UK Prime Minister Rishi Sunak told Sky News it was evidence that London's plan to send asylum seekers to Rwanda is acting as a deterrent. "What it shows, I think, is that the deterrent is … already having an impact because people are worried about coming here," he said. In response, a spokesperson for Ireland's Prime Minister Simon Harris said the leader "does not comment on the migration policies of any other country but he is very clear about the importance of protecting the integrity of the migration system in Ireland", RTE reported.


What role do US tech giants play in powering Israeli war crimes?

Al Jazeera

After Amazon and Google signed a 1.2bn contract to launch Project Nimbus, providing cloud technology to the Israeli government and the military, tech workers started to notice more Israeli use of artificial intelligence against the Palestinian people. Many of those engineers have become activists for "No Tech for Genocide", including Zelda Montes, who was one of the dozens of Google staff who were recently fired for protesting against their company's involvement with Israel. Montes and tech entrepreneur Paul Biggar, who founded Tech for Palestine, tell host Steve Clemons why they refuse to build technology used for oppression, surveillance, warfare and apartheid.


Okinawa's Ishigaki uses drone for first time during marine survey around Senkakus

The Japan Times

The city government of Ishigaki, Okinawa Prefecture, has conducted its third marine survey around the Japanese-administered Senkaku Islands, with Jiji Press and other reporters allowed to board its survey ship. During the survey, which lasted from Thursday to Saturday, Chinese coast guard vessels entered Japanese waters around the islands and attempted to interfere with the city's ship, but Japan Coast Guard vessels blocked the move. The city survey ship's journey to the area was the first since January last year. The latest survey was joined by Ishigaki Mayor Yoshitaka Nakayama and some lawmakers, including former Defense Minister Tomomi Inada. The Japanese government did not allow the survey participants to land on the islands.


Machine Learning Techniques for Data Reduction of CFD Applications

arXiv.org Artificial Intelligence

We present an approach called guaranteed block autoencoder that leverages Tensor Correlations (GBATC) for reducing the spatiotemporal data generated by computational fluid dynamics (CFD) and other scientific applications. It uses a multidimensional block of tensors (spanning in space and time) for both input and output, capturing the spatiotemporal and interspecies relationship within a tensor. The tensor consists of species that represent different elements in a CFD simulation. To guarantee the error bound of the reconstructed data, principal component analysis (PCA) is applied to the residual between the original and reconstructed data. This yields a basis matrix, which is then used to project the residual of each instance. The resulting coefficients are retained to enable accurate reconstruction. Experimental results demonstrate that our approach can deliver two orders of magnitude in reduction while still keeping the errors of primary data under scientifically acceptable bounds. Compared to reduction-based approaches based on SZ, our method achieves a substantially higher compression ratio for a given error bound or a better error for a given compression ratio.


Research and application of artificial intelligence based webshell detection model: A literature review

arXiv.org Artificial Intelligence

Webshell, as the "culprit" behind numerous network attacks, is one of the research hotspots in the field of cybersecurity. However, the complexity, stealthiness, and confusing nature of webshells pose significant challenges to the corresponding detection schemes. With the rise of Artificial Intelligence (AI) technology, researchers have started to apply different intelligent algorithms and neural network architectures to the task of webshell detection. However, the related research still lacks a systematic and standardized methodological process, which is confusing and redundant. Therefore, following the development timeline, we carefully summarize the progress of relevant research in this field, dividing it into three stages: Start Stage, Initial Development Stage, and In-depth Development Stage. We further elaborate on the main characteristics and core algorithms of each stage. In addition, we analyze the pain points and challenges that still exist in this field and predict the future development trend of this field from our point of view. To the best of our knowledge, this is the first review that details the research related to AI-based webshell detection. It is also hoped that this paper can provide detailed technical information for more researchers interested in AI-based webshell detection tasks.


A Note on Asynchronous Challenges: Unveiling Formulaic Bias and Data Loss in the Hayashi-Yoshida Estimator

arXiv.org Machine Learning

The Hayashi-Yoshida (\HY)-estimator exhibits an intrinsic, telescoping property that leads to an often overlooked computational bias, which we denote,formulaic or intrinsic bias. This formulaic bias results in data loss by cancelling out potentially relevant data points, the nonextant data points. This paper attempts to formalize and quantify the data loss arising from this bias. In particular, we highlight the existence of nonextant data points via a concrete example, and prove necessary and sufficient conditions for the telescoping property to induce this type of formulaic bias.Since this type of bias is nonexistent when inputs, i.e., observation times, $\Pi^{(1)} :=(t_i^{(1)})_{i=0,1,\ldots}$ and $\Pi^{(2)} :=(t_j^{(2)})_{j=0,1,\ldots}$, are synchronous, we introduce the (a,b)-asynchronous adversary. This adversary generates inputs $\Pi^{(1)}$ and $\Pi^{(2)}$ according to two independent homogenous Poisson processes with rates a>0 and b>0, respectively. We address the foundational questions regarding cumulative minimal (or least) average data point loss, and determine the values for a and b. We prove that for equal rates a=b, the minimal average cumulative data loss over both inputs is attained and amounts to 25\%. We present an algorithm, which is based on our theorem, for computing the exact number of nonextant data points given inputs $\Pi^{(1)}$ and $\Pi^{(2)}$, and suggest alternative methods. Finally, we use simulated data to empirically compare the (cumulative) average data loss of the (\HY)-estimator.


Field Notes on Deploying Research Robots in Public Spaces

arXiv.org Artificial Intelligence

Human-robot interaction requires to be studied in the wild. In the summers of 2022 and 2023, we deployed two trash barrel service robots through the wizard-of-oz protocol in public spaces to study human-robot interactions in urban settings. We deployed the robots at two different public plazas in downtown Manhattan and Brooklyn for a collective of 20 hours of field time. To date, relatively few long-term human-robot interaction studies have been conducted in shared public spaces. To support researchers aiming to fill this gap, we would like to share some of our insights and learned lessons that would benefit both researchers and practitioners on how to deploy robots in public spaces. We share best practices and lessons learned with the HRI research community to encourage more in-the-wild research of robots in public spaces and call for the community to share their lessons learned to a GitHub repository.


EkoHate: Abusive Language and Hate Speech Detection for Code-switched Political Discussions on Nigerian Twitter

arXiv.org Artificial Intelligence

Nigerians have a notable online presence and actively discuss political and topical matters. This was particularly evident throughout the 2023 general election, where Twitter was used for campaigning, fact-checking and verification, and even positive and negative discourse. However, little or none has been done in the detection of abusive language and hate speech in Nigeria. In this paper, we curated code-switched Twitter data directed at three musketeers of the governorship election on the most populous and economically vibrant state in Nigeria; Lagos state, with the view to detect offensive speech in political discussions. We developed EkoHate -- an abusive language and hate speech dataset for political discussions between the three candidates and their followers using a binary (normal vs offensive) and fine-grained four-label annotation scheme. We analysed our dataset and provided an empirical evaluation of state-of-the-art methods across both supervised and cross-lingual transfer learning settings. In the supervised setting, our evaluation results in both binary and four-label annotation schemes show that we can achieve 95.1 and 70.3 F1 points respectively. Furthermore, we show that our dataset adequately transfers very well to three publicly available offensive datasets (OLID, HateUS2020, and FountaHate), generalizing to political discussions in other regions like the US.