Government
Improving the Efficiency of the PC Algorithm by Using Model-Based Conditional Independence Tests
Cai, Erica, McGregor, Andrew, Jensen, David
Learning causal structure is useful in many areas of artificial intelligence, including planning, robotics, and explanation. Constraint-based structure learning algorithms such as PC use conditional independence (CI) tests to infer causal structure. Traditionally, constraint-based algorithms perform CI tests with a preference for smaller-sized conditioning sets, partially because the statistical power of conventional CI tests declines rapidly as the size of the conditioning set increases. However, many modern conditional independence tests are model-based, and these tests use well-regularized models that maintain statistical power even with very large conditioning sets. This suggests an intriguing new strategy for constraint-based algorithms which may result in a reduction of the total number of CI tests performed: Test variable pairs with large conditioning sets first, as a pre-processing step that finds some conditional independencies quickly, before moving on to the more conventional strategy that favors small conditioning sets. We propose such a pre-processing step for the PC algorithm which relies on performing CI tests on a few randomly selected large conditioning sets. We perform an empirical analysis on directed acyclic graphs (DAGs) that correspond to real-world systems and both empirical and theoretical analyses for Erd\H{o}s-Renyi DAGs. Our results show that Pre-Processing Plus PC (P3PC) performs far fewer CI tests than the original PC algorithm, between 0.5% to 36%, and often less than 10%, of the CI tests that the PC algorithm alone performs. The efficiency gains are particularly significant for the DAGs corresponding to real-world systems.
Generating Textual Adversaries with Minimal Perturbation
Zhao, Xingyi, Zhang, Lu, Xu, Depeng, Yuan, Shuhan
Many word-level adversarial attack approaches for textual data have been proposed in recent studies. However, due to the massive search space consisting of combinations of candidate words, the existing approaches face the problem of preserving the semantics of texts when crafting adversarial counterparts. In this paper, we develop a novel attack strategy to find adversarial texts with high similarity to the original texts while introducing minimal perturbation. The rationale is that we expect the adversarial texts with small perturbation can better preserve the semantic meaning of original texts. Experiments show that, compared with state-of-the-art attack approaches, our approach achieves higher success rates and lower perturbation rates in four benchmark datasets.
Prior-mean-assisted Bayesian optimization application on FRIB Front-End tunning
Hwang, Kilean, Maruta, Tomofumi, Plastun, Alexander, Fukushima, Kei, Zhang, Tong, Zhao, Qiang, Ostroumov, Peter, Hao, Yue
The Facility for Rare Isotope Beams (FRIB) at Michigan State University (MSU) is designed for various kinds of rare isotope production. This involves the frequent switch of the ion source species. Therefore, fast tuning of the accelerator Front-End (FE) to maintain optimal beam optics is one of the key performance requirements. Breaking-through the tuning performance over the traditional black-box optimization algorithm may be possible if historical or simulated data can be incorporated into the optimization algorithm in a computationally feasible way. However, we experienced significant machine status changes (a.k.a. 'distribution shift' or'machine drift') whenever ion source species are switched or the ion source is re-started (after overnight turn-off).
Rethinking Log Odds: Linear Probability Modelling and Expert Advice in Interpretable Machine Learning
Dervovic, Danial, Marchesotti, Nicolas, Lecue, Freddy, Magazzeni, Daniele
We introduce a family of interpretable machine learning models, with two broad additions: Linearised Additive Models (LAMs) which replace the ubiquitous logistic link function in General Additive Models (GAMs); and SubscaleHedge, an expert advice algorithm for combining base models trained on subsets of features called subscales. LAMs can augment any additive binary classification model equipped with a sigmoid link function. Moreover, they afford direct global and local attributions of additive components to the model output in probability space. We argue that LAMs and SubscaleHedge improve the interpretability of their base algorithms. Using rigorous null-hypothesis significance testing on a broad suite of financial modelling data, we show that our algorithms do not suffer from large performance penalties in terms of ROC-AUC and calibration.
GeoAI for Knowledge Graph Construction: Identifying Causality Between Cascading Events to Support Environmental Resilience Research
Knowledge graph technology is considered a powerful and semantically enabled solution to link entities, allowing users to derive new knowledge by reasoning data according to various types of reasoning rules. However, in building such a knowledge graph, events modeling, such as that of disasters, is often limited to single, isolated events. The linkages among cascading events are often missing in existing knowledge graphs. This paper introduces our GeoAI (Geospatial Artificial Intelligence) solutions to identify causality among events, in particular, disaster events, based on a set of spatially and temporally-enabled semantic rules. Through a use case of causal disaster events modeling, we demonstrated how these defined rules, including theme-based identification of correlated events, spatiotemporal co-occurrence constraint, and text mining of event metadata, enable the automatic extraction of causal relationships between different events. Our solution enriches the event knowledge base and allows for the exploration of linked cascading events in large knowledge graphs, therefore empowering knowledge query and discovery.
Dark patterns in e-commerce: a dataset and its baseline evaluations
Yada, Yuki, Feng, Jiaying, Matsumoto, Tsuneo, Fukushima, Nao, Kido, Fuyuko, Yamana, Hayato
Dark patterns, which are user interface designs in online services, induce users to take unintended actions. Recently, dark patterns have been raised as an issue of privacy and fairness. Thus, a wide range of research on detecting dark patterns is eagerly awaited. In this work, we constructed a dataset for dark pattern detection and prepared its baseline detection performance with state-of-the-art machine learning methods. The original dataset was obtained from Mathur et al.'s study in 2019, which consists of 1,818 dark pattern texts from shopping sites. Then, we added negative samples, i.e., non-dark pattern texts, by retrieving texts from the same websites as Mathur et al.'s dataset. We also applied state-of-the-art machine learning methods to show the automatic detection accuracy as baselines, including BERT, RoBERTa, ALBERT, and XLNet. As a result of 5-fold cross-validation, we achieved the highest accuracy of 0.975 with RoBERTa. The dataset and baseline source codes are available at https://github.com/yamanalab/ec-darkpattern.
CoRAL: a Context-aware Croatian Abusive Language Dataset
Shekhar, Ravi, Karan, Mladen, Purver, Matthew
In light of unprecedented increases in the popularity of the internet and social media, comment moderation has never been a more relevant task. Semi-automated comment moderation systems greatly aid human moderators by either automatically classifying the examples or allowing the moderators to prioritize which comments to consider first. However, the concept of inappropriate content is often subjective, and such content can be conveyed in many subtle and indirect ways. In this work, we propose CoRAL -- a language and culturally aware Croatian Abusive dataset covering phenomena of implicitness and reliance on local and global context. We show experimentally that current models degrade when comments are not explicit and further degrade when language skill and context knowledge are required to interpret the comment.
Measuring Progress on Scalable Oversight for Large Language Models
Bowman, Samuel R., Hyun, Jeeyoon, Perez, Ethan, Chen, Edwin, Pettit, Craig, Heiner, Scott, Lukošiūtė, Kamilė, Askell, Amanda, Jones, Andy, Chen, Anna, Goldie, Anna, Mirhoseini, Azalia, McKinnon, Cameron, Olah, Christopher, Amodei, Daniela, Amodei, Dario, Drain, Dawn, Li, Dustin, Tran-Johnson, Eli, Kernion, Jackson, Kerr, Jamie, Mueller, Jared, Ladish, Jeffrey, Landau, Joshua, Ndousse, Kamal, Lovitt, Liane, Elhage, Nelson, Schiefer, Nicholas, Joseph, Nicholas, Mercado, Noemí, DasSarma, Nova, Larson, Robin, McCandlish, Sam, Kundu, Sandipan, Johnston, Scott, Kravec, Shauna, Showk, Sheer El, Fort, Stanislav, Telleen-Lawton, Timothy, Brown, Tom, Henighan, Tom, Hume, Tristan, Bai, Yuntao, Hatfield-Dodds, Zac, Mann, Ben, Kaplan, Jared
Developing safe and useful general-purpose AI systems will require us to make progress on scalable oversight: the problem of supervising systems that potentially outperform us on most skills relevant to the task at hand. Empirical work on this problem is not straightforward, since we do not yet have systems that broadly exceed our abilities. This paper discusses one of the major ways we think about this problem, with a focus on ways it can be studied empirically. We first present an experimental design centered on tasks for which human specialists succeed but unaided humans and current general AI systems fail. We then present a proof-of-concept experiment meant to demonstrate a key feature of this experimental design and show its viability with two question-answering tasks: MMLU and time-limited QuALITY. On these tasks, we find that human participants who interact with an unreliable large-language-model dialog assistant through chat -- a trivial baseline strategy for scalable oversight -- substantially outperform both the model alone and their own unaided performance. These results are an encouraging sign that scalable oversight will be tractable to study with present models and bolster recent findings that large language models can productively assist humans with difficult tasks.
Mixed Observable RRT: Multi-Agent Mission-Planning in Partially Observable Environments
Johansson, Kasper, Rosolia, Ugo, Ubellacker, Wyatt, Singletary, Andrew, Ames, Aaron D.
This paper considers centralized mission-planning for a heterogeneous multi-agent system with the aim of locating a hidden target. We propose a mixed observable setting, consisting of a fully observable state-space and a partially observable environment, using a hidden Markov model. First, we construct rapidly exploring random trees (RRTs) to introduce the mixed observable RRT for finding plausible mission plans giving way-points for each agent. Leveraging this construction, we present a path-selection strategy based on a dynamic programming approach, which accounts for the uncertainty from partial observations and minimizes the expected cost. Finally, we combine the high-level plan with model predictive control algorithms to evaluate the approach on an experimental setup consisting of a quadruped robot and a drone. It is shown that agents are able to make intelligent decisions to explore the area efficiently and to locate the target through collaborative actions.
Escaping Arrow's Theorem: The Advantage-Standard Model
Holliday, Wesley H., Kelley, Mikayla
There is an extensive literature in social choice theory studying the consequences of weakening the assumptions of Arrow's Impossibility Theorem. Much of this literature suggests that there is no escape from Arrow-style impossibility theorems, while remaining in an ordinal preference setting, unless one drastically violates the Independence of Irrelevant Alternatives (IIA). In this paper, we present a more positive outlook. We propose a model of comparing candidates in elections, which we call the Advantage-Standard (AS) model. The requirement that a collective choice rule (CCR) be rationalizable by the AS model is in the spirit of but weaker than IIA; yet it is stronger than what is known in the literature as weak IIA (two profiles alike on $x, y$ cannot have opposite strict social preferences on $x$ and $y$). In addition to motivating violations of IIA, the AS model makes intelligible violations of another Arrovian assumption: the negative transitivity of the strict social preference relation $P$. While previous literature shows that only weakening IIA to weak IIA or only weakening negative transitivity of $P$ to acyclicity still leads to impossibility theorems, we show that jointly weakening IIA to AS rationalizability and weakening negative transitivity of $P$ leads to no such impossibility theorems. Indeed, we show that several appealing CCRs are AS rationalizable, including even transitive CCRs.